JP2006203571A - Imaging apparatus, image processing apparatus, and image recording apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform optimum image processing corresponding to picked-up scene relative to picked-up image data without causing information losses of picked-up image information which is obtained by an imaging apparatus. <P>SOLUTION: The imaging apparatus 100 generates scene reference raw data relying on the characteristic of the imaging apparatus by picking up an image. An apparatus characteristic correcting data generator 13 generates correcting data to be required in imaging apparatus characteristic correcting processing in order to generate scene reference image data which is obtained by standardizing the scene reference raw data. A reduction raw data generator 21 generates reduction raw data obtained by reducing the image size of the scene reference raw data. A picked-up information data generator 12 generates picked-up information data which indicates imaging condition setting in picking up the image. A header information processor 8 generates an attached data file by attaching the correcting data, the reduction raw data, and the picked-up information data to the scene reference raw data. The data file is recorded in the recording medium of a storage device 9. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus such as a digital camera, and an image processing apparatus and an image recording apparatus that perform optimization processing on captured image data obtained by the imaging apparatus for forming an appreciation image on an output medium.

  In recent years, photographed image data obtained by an imaging device such as a digital camera is recorded on a recording medium such as a CD-R (Compact Disc Recordable), a floppy (registered trademark) disk, or a memory card, or via a communication network such as the Internet. It can be distributed on the Internet, displayed on a display device such as a CRT (Cathode Ray Tube), liquid crystal display, plasma display, or a small liquid crystal monitor on a mobile phone, or can be hardware using an output device such as a digital printer, an inkjet printer, or a thermal printer. There are a variety of output methods such as printing as a copy image.

  When outputting captured image data for viewing purposes, generally, gradation adjustment, brightness adjustment, color adjustment is performed on the captured image data so that a desired image quality can be obtained on a display monitor or hard copy used for viewing. Various image processes represented by balance adjustment and sharpness enhancement are performed.

  In response to these various display / printing methods, efforts have been made to increase the versatility of digital image data captured by an imaging apparatus. As part of this effort, there is an attempt to standardize the color space represented by the digital RGB (Red, Green, Blue) signal to a color space that does not depend on the characteristics of the imaging device. Currently, many digital image data are standardized color spaces. “SRGB” is adopted (see “Multimedia Systems and Equipment-Colour Measurement and Management-Part2-1: Color Management-Default RGB Color Space-sRGB” IEC “61966-2-1”). Are set corresponding to the color reproduction region of a standard CRT display monitor.

  Generally, a digital camera is a CCD type image pickup device having a photoelectric conversion function, which is provided with color sensitivity by combining a charge coupled device (CCD), a charge transfer mechanism, and a checkered color filter. (Hereinafter simply referred to as CCD). The digital image data output by the digital camera is subjected to correction of the photoelectric conversion function of the image sensor on the electrical original signal converted through the CCD so that it can be read and displayed by image editing software. The file has been subjected to file conversion / compression processing into a data format of a predetermined format standardized.

  Examples of the correction of the photoelectric conversion function of the image sensor include gradation correction, crosstalk correction of spectral sensitivity, dark current noise suppression, sharpening, white balance adjustment, saturation adjustment, and the like. In addition, as a standardized data format of a predetermined format, for example, “Baseline Tiff Rev. 6.0 RGB Full Color Image”, which is adopted as an uncompressed file of an Exif (Exchangeable Image File Format) file, JPEG (Joint Photographic Coding) A compressed data file format conforming to the Experts Group) format is known.

  The Exif file conforms to sRGB, and the correction of the photoelectric conversion function of the image sensor is set so as to obtain the most suitable image quality on a display monitor conforming to sRGB.

  For example, tag information or pixels indicating that any digital camera is displayed in a standard color space (hereinafter referred to as “monitor profile”) of a display monitor compliant with the sRGB signal, as in the Exif format. As long as a function of writing additional information indicating model-dependent information such as the number, pixel arrangement and the number of bits per pixel as metadata in the file header of digital image data and such a data format format are adopted, the digital image Tag information can be analyzed by image editing software (for example, Adobe Photoshop) that displays data on a display monitor, and the monitor profile can be prompted to change to sRGB or can be automatically changed. Therefore, it is possible to reduce the difference in device characteristics between different displays and to view digital image data captured by a digital camera in a suitable state on a display monitor.

  Further, as additional information written in the file header of digital image data, in addition to the above-mentioned model-dependent information, for example, information directly related to the camera type (model) such as camera name and code number, exposure time, shutter Speed, aperture value (F number), ISO sensitivity, brightness value, subject distance range, light source, presence / absence of strobe emission, subject area, white balance, zoom magnification, subject composition, shooting scene type, amount of reflected light from strobe light source, A tag (code) indicating shooting condition settings such as shooting saturation and information on the type of subject is used. The image editing software and the output device have a function of reading the additional information and making the image quality of the hard copy image more suitable.

In Patent Document 1, standardized scene reference image data such as sRGB is generated from raw scene reference data obtained from an imaging device (raw data that is not subjected to image processing and faithfully represents subject information). The correction data necessary for the correction process is attached to the file header of the scene reference raw data and recorded on a recording medium, and the correction data attached to the scene reference raw data is used on the image processing apparatus or the image recording apparatus side. A technique is disclosed that can create an optimized appreciation image by performing image processing without causing information loss of captured image information.
JP 2004-96500 A

  Depending on the shooting conditions (light source conditions, exposure conditions) at the time of shooting, the shooting scene may become an image unfavorable for viewing such as backlight, under, over, etc. In order to make such an undesired image suitable, for example, a shooting scene is determined, and image processing corresponding to the shooting scene is performed on the shooting image data. When performing a photographic scene discrimination process, it is preferable to use data with reduced image size (reduced data) because of processing speed. However, if the reduced image data is created after compressing the wide color gamut / luminance range information (photographed image information) acquired by the imaging device into appreciation image reference data optimized for the appreciation image, There has been a problem that information loss of image information becomes large.

  SUMMARY OF THE INVENTION An object of the present invention is to enable optimum image processing corresponding to a photographic scene with respect to photographic image data without accompanying information loss of photographic image information obtained by an imaging device.

In order to solve the above-described problem, the invention described in claim 1 includes scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
And recording control means for attaching the correction data and the reduced raw data to the scene reference raw data and recording them on a recording medium.

The invention according to claim 2 is a scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
A scene reference reduced image data generating means for generating standardized scene reference reduced image data by performing an imaging device characteristic correction process on the reduced raw data based on the correction data;
And recording control means for attaching the correction data and the scene reference reduced image data to the scene reference raw data and recording them on a recording medium.

The invention according to claim 3 is a scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
And recording control means for attaching the correction data, the reduced raw data, and the shooting information data to the scene reference raw data and recording them on a recording medium.

The invention according to claim 4 is a scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
A scene reference reduced image data generating means for generating standardized scene reference reduced image data by performing an imaging device characteristic correction process on the reduced raw data based on the correction data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
And recording control means for attaching the correction data, the scene reference reduced image data, and the shooting information data to the scene reference raw data and recording them on a recording medium.

According to the fifth aspect of the present invention, the scene reference raw data depending on the characteristics of the imaging device and the correction data necessary for the imaging device characteristic correction processing for generating the standardized scene reference image data from the scene reference raw data And input means for inputting reduced raw data generated from the scene reference raw data,
A standardized scene reference image data and scene reference reduced image data are generated by subjecting the input scene reference raw data and reduced raw data to a photographing device characteristic correction process based on the input correction data. And a scene reference image data generation means.

  According to a sixth aspect of the present invention, in the image processing apparatus according to the fifth aspect, the generated standardized scene reference image data is output on an output medium based on the generated scene reference reduced image data. The image processing apparatus includes an image reference data generation unit that generates image reference data to be subjected to image processing that is optimized for forming an image to be viewed.

According to the seventh aspect of the present invention, the scene reference raw data depending on the characteristics of the imaging device and the correction data necessary for the imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data And input means for inputting standardized scene reference reduced image data generated from the scene reference raw data,
Scene reference image data generating means for generating standardized scene reference image data by performing a photographing device characteristic correction process on the input scene reference raw data based on the input correction data; It is characterized by providing.

  According to an eighth aspect of the present invention, in the image processing apparatus according to the seventh aspect, the generated standardized scene reference image data is output on an output medium based on the input scene reference reduced image data. The image processing apparatus includes an image reference data generation unit that generates image reference data to be subjected to image processing that is optimized for forming an image to be viewed.

According to the ninth aspect of the present invention, there is provided scene reference raw data depending on characteristics of the imaging device, and correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data. Input means for inputting reduced raw data generated from the scene reference raw data and shooting information data indicating shooting condition settings at the time of shooting;
A standardized scene reference image data and scene reference reduced image data are generated by subjecting the input scene reference raw data and reduced raw data to a photographing device characteristic correction process based on the input correction data. And a scene reference image data generation means.

  According to a tenth aspect of the present invention, in the image processing apparatus according to the ninth aspect, the generated standardized scene reference is based on the generated scene reference reduced image data and the input photographing information data. The image data is provided with appreciation image reference data generating means for generating image appreciation image reference data by performing image processing optimized for appreciation image formation on the output medium.

According to the eleventh aspect of the present invention, the scene reference raw data depending on the characteristics of the imaging device and the correction data necessary for the imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data. Input means for inputting standardized scene reference reduced image data generated from the scene reference raw data and shooting information data indicating shooting condition settings at the time of shooting;
Scene reference image data generating means for generating standardized scene reference image data by performing a photographing device characteristic correction process on the input scene reference raw data based on the input correction data; It is characterized by providing.

  According to a twelfth aspect of the present invention, in the image processing device according to the eleventh aspect, the standardized scene reference generated based on the input scene reference reduced image data and the input photographing information data. The image data is provided with appreciation image reference data generating means for generating image appreciation image reference data by performing image processing optimized for appreciation image formation on the output medium.

A thirteenth aspect of the present invention is the image processing apparatus according to any one of the sixth, eighth, tenth, and twelfth aspects, wherein a scene for determining a photographing scene at the time of photographing is used using the scene reference reduced image data. With a discrimination means,
The appreciation image reference data generation unit generates the appreciation image reference data using a determination result obtained by the scene determination unit.

An invention according to claim 14 is an image processing apparatus according to any one of claims 6, 8, 10, 12, and 13,
Image forming means for forming an appreciation image on an output medium using appreciation image reference data generated by the appreciation image reference data generation means of the image processing apparatus.

<Explanation of terms>
Hereinafter, terms used in this specification will be described.

  In the description of the present specification, “generation” means that a program and a processing circuit that operate in the imaging apparatus, the image processing apparatus, and the image recording apparatus according to the present invention newly create an image signal and data. “Create” may be used as a synonym.

  In the description of this specification, an “imaging device” is a device including an imaging element (image sensor) having a photoelectric conversion function, and includes a so-called digital camera and scanner. As an example of the image pickup device, a CCD type image pickup device provided with a color sensitivity by combining a charge coupled device (CCD), a charge transfer mechanism, and a checkered color filter, or a complementary metal-oxide display (CMOS). Semiconductor) type imaging device. The output currents of these image sensors are digitized by an A / D converter. The contents of each color channel at this stage are signal intensities based on the spectral sensitivity unique to the image sensor.

  The “scene reference raw data depending on the characteristics of the imaging device” is a direct raw output signal of the imaging device in which information faithful to the subject is recorded, and the data itself digitized by the A / D converter, This means data obtained by performing noise correction such as fixed pattern noise and dark current noise on the data. This scene reference raw data is generated by image processing that modifies the contents of the data to improve the effects during image viewing such as gradation conversion, sharpness enhancement, and saturation enhancement, and each color channel based on the spectral sensitivity inherent to the image sensor. The process of mapping the signal intensity of the signal to a standardized color space such as RIMM RGB or sRGB is omitted. The information amount (for example, the number of gradations) of the scene reference raw data is preferably equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data in accordance with the performance of the A / D converter. For example, when the number of gradations of the viewing image reference data (described later) is 8 bits per channel, the number of gradations of the scene reference raw data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

  “Output media” refers to, for example, display devices such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), plasma display, and hard copy image generation such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. Paper.

  “Appreciation image reference data” refers to digital images used for display devices such as CRTs, LCDs, and plasma displays, or for generating hard copy images on output media such as silver halide photographic paper, ink jet paper, and thermal printer paper. Means data. The appreciation image reference data is subjected to "optimization processing" so that an optimal image can be obtained on a display device such as a CRT, LCD, or plasma display, or on an output medium such as silver salt photographic paper, inkjet paper, or thermal printer paper. It is different from the scene reference raw data.

  The “recording medium” is a storage medium used for storing “scene reference raw data” and “correction data necessary for the imaging device characteristic correction process” output from the imaging device, and is a compact flash (registered trademark). , Memory Stick (registered trademark), SmartMedia (registered trademark), multimedia card, hard disk, floppy (registered trademark) disk, magnetic storage medium (MO), CD-R, etc. In addition, the unit for writing to the recording medium is installed in an independent or remote place connected in a wireless state via a communication network such as the Internet, a writing unit connected by wire through a cord, even if it is integrated with the photographing apparatus. Any mode such as a unit may be used. The file format for recording on the recording medium is not a format specific to the imaging device, but standardized general-purpose such as TIFF (Tagged Image File Format), JPEG (Joint Photographic Coding Experts Group), and Exif (Exchangeable Image File Format) The file format is preferably recorded.

  The “shooting information data” is a record of shooting condition settings at the time of shooting, and may include the same tag information written in the header portion of the Exif file. Specifically, exposure time, shutter speed, aperture value (F number), ISO sensitivity, brightness value, subject distance range, light source, presence / absence of strobe emission, subject area, white balance, zoom magnification, shooting scene, strobe light source A tag (code) indicating the amount of reflected light, photographing saturation, type of subject, information on subject configuration, and the like.

  “Photographing information data” is a value obtained at the time of photographing by a sensor provided in the camera, data processed from the value of the sensor, or a value of the sensor for automating the exposure setting and focus function of the imaging device. Camera shooting conditions set based on the camera, but other than this, the shooting mode dial (for example, portrait, sports, macro shooting mode, etc.) and flash forced flash settings provided in the imaging device Information that the photographer manually sets the switch and the like is also included.

  In addition, “standardized scene reference image data” means that the signal intensity of each color channel based on at least the spectral sensitivity of the image sensor itself has been mapped to the standard color space such as RIM RGB, ERIMM RGB, and scRGB. It means image data in a state where image processing for modifying data contents is omitted in order to improve the effect at the time of image appreciation such as tone conversion, sharpness enhancement, and saturation enhancement. The scene reference image data is the photoelectric conversion characteristic of the imaging device (opto-electronic conversion function defined by ISO1452, for example, “Fine Imaging and Digital Photography” (see page 449 of the Japan Photographic Society Publishing Committee)). It is preferable that correction has been performed. The information amount (for example, the number of gradations) of the standardized scene reference image data conforms to the performance of the A / D converter and is equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data. Is preferred. For example, when the number of gradations of the viewing image reference data is 8 bits per channel, the number of gradations of the scene reference image data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

  In addition, “imaging device characteristic correction processing for generating standardized scene reference image data” means that “scene reference raw data depending on the characteristics of the imaging device” is converted to “standardized scene reference image data”. Means processing. The content of this processing depends on the state of the “scene reference raw data depending on the characteristics of the imaging device”, but at least the signal intensity of each color channel based on the spectral sensitivity specific to the imaging device is standard such as RIMM RGB, ERIMM RGB, scRGB, etc. Includes mapping to color space. For example, when the “scene reference raw data depending on the characteristics of the imaging device” is not subjected to the interpolation processing based on the color filter array, the processing needs to be additionally performed. (Details of the interpolation processing based on the color filter array are described in, for example, “Fine Imaging and Digital Photography” of Corona, Inc., page 51 of the Japan Photographic Society Publishing Committee.) As a result, “scene reference raw data” The “standardized scene reference image data” in which the difference in signal values between different “imaging devices” is corrected is obtained.

  In addition, “optimized image processing” means processing for obtaining an optimal image on a display device such as a CRT, LCD, or plasma display, or on an output medium such as silver salt photographic paper, inkjet paper, or thermal printer paper. For example, when it is assumed that the image is displayed on a CRT display monitor compliant with the sRGB standard, processing is performed so that optimum color reproduction is obtained within the color gamut of the sRGB standard. If output to silver salt photographic paper is assumed, processing is performed so that optimum color reproduction is obtained within the color gamut of silver salt photographic paper. In addition to color gamut compression, gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, and processing for handling output characteristics (LUT) of the output device are also included. Furthermore, it goes without saying that image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment or dodging is performed.

An example of optimizing appreciation image reference data using “photographing information data” is shown below.
Based on the “subject configuration” information, for example, it is possible to perform a saturation enhancement process partially, or to perform a dodging process in a scene with a wide dynamic range.
Based on the result of determining the shooting scene, for example, in night scene shooting, the degree of white balance adjustment can be relaxed and the color balance can be adjusted specially.
The distance between the photographer and the subject is estimated based on the “amount of reflected light from the strobe light source” information, and can be reflected in, for example, setting of image processing conditions that suppress whitening of the skin.
Based on the “subject type” information, for example, in human photography, the degree of sharpness is relaxed and the smoothing process is strengthened, so that wrinkles on the skin can be made inconspicuous.

  The image recording apparatus according to the present invention includes a color negative film, a color reversal film, a black and white negative film, a black and white film, in addition to a mechanism for performing image processing according to the present invention on digital image data acquired from the imaging apparatus according to the present invention. There may be provided a film scanner for inputting frame image information of a photographic photosensitive material recorded by an analog camera such as a reversal film, and a flatbed scanner for inputting image information reproduced on color paper which is silver salt photographic paper. Also obtained by a digital camera other than the imaging device of the present invention, CompactFlash (registered trademark), Memory Stick (registered trademark), SmartMedia (registered trademark), Multimedia card (registered trademark), floppy (registered trademark) disk, optical A means for reading digital image data stored in any known portable “recording medium” such as a magnetic storage medium (MO), CD-R, etc., or obtaining digital image data from a remote location via communication means, and CRT And display means such as LCD and plasma display, and processing means for forming an appreciation image on any known output medium such as hard copy image generation paper such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. May be.

  According to the present invention, the correction data and the reduced raw data necessary for the imaging device characteristic correction process are attached to the scene reference raw data depending on the characteristics of the imaging device, and shooting at the time of shooting based on the reduced raw data. Scene (front light, backlight, under, strobe) can be easily identified, and the standardized scene reference image data generated using the correction data from the scene reference raw data is optimal for the shooting scene. Image processing can be performed. Therefore, it is possible to obtain high-quality image data without causing information loss of captured image information.

  In addition to the correction data and the reduced raw data, by adding the shooting information data, which is the shooting condition setting at the time of shooting, to the scene reference raw data, the shooting scene as the determination result of the shooting scene determination process and the shooting information data are added. By collating the included shooting scenes, it is possible to improve the shooting scene discrimination accuracy.

  According to the present invention, the scene reference reduced image data is attached to the scene reference raw data depending on the characteristics of the imaging device by attaching correction data and scene reference reduced image data necessary for the imaging device characteristic correction processing. Based on the scene reference image data generated by using the correction data from the scene reference raw data, it is possible to easily determine the shooting scene (front light, backlight, under, strobe) at the time of shooting. It is possible to perform optimum image processing according to the scene. Therefore, it is possible to obtain high-quality image data without causing information loss of captured image information.

  Further, in addition to the correction data and the scene reference reduced image data, by adding shooting information data, which is a shooting condition setting at the time of shooting, to the scene reference raw data, a shooting scene as a determination result of the shooting scene determination process, and shooting information By collating the shooting scenes included in the data, it is possible to improve the shooting scene discrimination accuracy.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[Embodiment 1]
First, Embodiment 1 of the present invention will be described with reference to FIGS.

<Configuration of imaging device>
FIG. 1 shows a main part configuration of an imaging apparatus 100 according to Embodiment 1 of the present invention. As shown in FIG. 1, the imaging apparatus 100 includes a lens 1, an aperture 2, a CCD (Charge Coupled Device) 3, an analog processing circuit 4, an A / D converter 5, a temporary storage memory 6, an image processing unit 7, and header information. Processing unit 8, storage device 9, CCD drive circuit 10, control unit 11, photographing information data generation unit 12, device characteristic correction data generation unit 13, reduced raw data generation unit 21, operation unit 14, display unit 15, strobe drive circuit 16, a strobe 17, a focal length adjustment circuit 18, an automatic focus driving circuit 19, a motor 20, and the like.

The optical system of the imaging apparatus 100 includes a lens 1, a diaphragm 2, and a CCD 3.
The lens 1 adjusts the focus and forms an optical image of the subject. The diaphragm 2 adjusts the amount of light beam that has passed through the lens 1. The CCD 3 photoelectrically converts subject light imaged on the light receiving surface by the lens 1 into an electrical signal (imaging signal) of an amount corresponding to the amount of incident light for each sensor in the CCD 3. Then, the CCD 3 is sequentially controlled by the timing pulse input from the CCD driving circuit 10 to sequentially output the image pickup signal to the analog processing circuit 4.

  The analog processing circuit 4 performs RGB signal amplification, noise reduction processing, and the like on the imaging signal input from the CCD 3. The processing in the analog processing circuit 4 is switched ON / OFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The A / D converter 5 converts the imaging signal input from the analog processing circuit 4 into a digital signal and outputs it. Hereinafter, a digital signal obtained by the A / D converter 5 will be described as scene reference raw data.

  The temporary storage memory 6 is a buffer memory or the like, and temporarily stores the image data output from the A / D converter 5.

  The image processing unit 7 performs tone correction, spectral sensitivity crosstalk correction, dark current noise suppression, sharpening, white balance adjustment for display on the display unit 15 with respect to the image data stored in the temporary storage memory 6. In addition to image quality improvement processing such as saturation adjustment, processing such as image size change, trimming, and aspect conversion is performed. The processing in the image processing unit 7 is switched ON / OFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The header information processing unit 8 uses the shooting information data generated by the shooting information data generation unit 12 and the device characteristic correction data generation unit 13 in the file header (header area) of the scene reference raw data stored in the temporary storage memory 6. An attached data file is created by attaching the generated imaging device characteristic correction data and the reduced raw data generated by the reduced raw data generation unit 21. FIG. 2 shows the data structure of the attached data file. The reduced raw data, the imaging device characteristic correction data, and the like may be recorded as tag information written in the header portion in the image file, for example, a DNG tag according to the image format Digital Negative (DNG) recommended by Adobe Systems. preferable.

  The storage device 9 is configured by a nonvolatile semiconductor memory or the like, and stores a control program for the imaging apparatus 100. In addition, the storage device 9 includes a mounting unit for mounting a recording medium such as a memory card, and in accordance with a control signal input from the control unit 11, reading of recorded data on the recording medium mounted on the mounting unit and recording Write data to the media. For example, the storage device 9 records the attached data file (see FIG. 2) created by the header information processing unit 8 on a recording medium.

  The CCD drive circuit 10 outputs a timing pulse in accordance with a control signal input from the control unit 11 and performs drive control of the CCD 3.

  The control unit 11 is configured by a CPU (Central Processing Unit) or the like, reads a control program of the imaging device 100 stored in the storage device 9, and controls the entire imaging device 100 according to the read control program.

  For example, the control unit 11 controls the motor 20 for adjusting the focal length and focus (focus) of the lens 1 in accordance with an operation signal from the operation unit 14, an automatic focus drive circuit 19, a focal length adjustment circuit 18, a CCD By controlling the driving circuit 10, the analog processing circuit 4, the temporary storage memory 6, the image processing unit 7, the display unit 15, and the strobe driving circuit 16, the photographing operation is executed. In addition, the control unit 11 generates shooting information data, imaging device characteristic correction data, and reduced raw data to be attached to a file header of scene reference raw data obtained by shooting. A data file in which the correction data generation unit 13 and the reduced raw data generation unit 21 are instructed, and attached data (shooting information data, imaging device characteristic correction data, reduced raw data) is attached to the scene reference raw data for the storage device 9. Is recorded on the recording medium.

  The shooting information data generation unit 12 generates shooting information data that is a shooting condition setting at the time of shooting. This shooting information data includes, for example, information directly related to the camera type (model) such as camera name and code number, exposure time, shutter speed, aperture value (F number), ISO sensitivity, luminance value, subject distance range, Information on the light source, the presence / absence of strobe light emission, the subject area, white balance, zoom magnification, subject composition, shooting scene, the amount of reflected light from the strobe light source, shooting conditions such as shooting saturation, and information on the type of subject.

  The apparatus characteristic correction data generation unit 13 generates correction data (imaging apparatus characteristic correction data) necessary for imaging apparatus characteristic correction processing for generating standardized scene reference image data from scene reference raw data. As shown in FIG. 3, the imaging device characteristic correction processing includes filter interpolation calculation (a), matrix calculation (b), photoelectric conversion characteristic and gain correction (c) for scene reference raw data.

  The filter interpolation calculation shown in FIG. 3A is a process of interpolating image data having a filter arrangement of one pixel and one color into linear image data of one pixel and three colors (RGB). Here, the filter array is an array pattern of color filters for color discrimination of CCD, and an RGB primary color Bayer array is generally used. As a filter interpolation method, a nearest neighbor method, a bi-linear interpolation method, a bi-cubic convolution method, or the like can be applied.

  In the nearest neighbor method, a pixel nearest to the target pixel is selected, the pixel is enlarged or reduced as it is according to the size of enlargement / reduction, and the pixel is interpolated only at a necessary portion. In the bilinear method, linear density interpolation is performed from the density values of four pixels around the target pixel according to the coordinates (real values). In the bicubic method, in order to perform interpolation with higher accuracy than in the bilinear method, interpolation is performed using a cubic function from the density values of 16 pixels around the pixel of interest. The equation used for interpolation is sin (πx) / πx, which is the most complete density interpolation equation in theory (sampling theorem). This is approximated by a third-order term of x by Taylor expansion and used as an interpolation formula.

  The matrix calculation shown in FIG. 3B is a calculation process for correcting a difference in which the same subject color is recorded as a different signal due to a difference in spectral characteristics of the color filter and spectral sensitivity characteristics of the image sensor. Conversion from RGB to tristimulus values in the XYZ color system is performed by matrix calculation.

  FIG. 3C shows processing for correcting a difference due to photoelectric conversion characteristics (linear conversion characteristics) and gain correction (parallel movement) of the image sensor, and this processing is shown in FIG. As described above, logY (Y: stimulus value of XYZ color system) is linearly converted with respect to logE (E: exposure amount).

  By the imaging device characteristic correction processing as shown in FIGS. 3A to 3C, the scene reference raw data depending on the characteristics of the imaging device is converted into XYZ values in a standardized color space such as scene reference image data. The Note that the color filter array pattern, matrix calculation matrix coefficients a to i, coefficient values for correcting differences associated with photoelectric conversion characteristics and gain correction, which are applied in the imaging apparatus characteristic correction processing, are all different for each imaging apparatus. is there.

  The reduced raw data generation unit 21 generates reduced raw data by performing a process of reducing the image size on the scene reference raw data depending on the characteristics of the imaging device. A known method can be used to generate reduced raw data from scene reference raw data. For example, as disclosed in Japanese Patent Laid-Open No. 2003-346143 (see paragraph [0037], FIG. 4), the scene reference raw data is color-separated into RGB color signals, and the color-coded R signal, G The low-pass filter set for each color is applied to the signal and the B signal, and the R signal, the G signal, and the B signal obtained by applying the low-pass filter are rearranged into the pixel arrangement of the scene reference raw data. Then, a process of thinning out pixels at a certain rate is performed. The image size reduction rate is not particularly limited, but is preferably about 1/2 to 1/10 of the original image from the viewpoint of processing speed and processing accuracy.

  The operation unit 14 includes a shutter button, a power ON / OFF button, various function buttons such as a zoom button, a cursor key, and the like, and outputs an operation signal corresponding to each button and key to the control unit 11.

  The display unit 15 is configured by an LCD (Liquid Crystal Display), an EL (ElectroLuminescence) display, or the like, and performs a required display process in accordance with a display control signal input by the control unit 11. For example, the display unit 15 displays information for the user of the imaging apparatus 100 to confirm settings and conditions related to shooting, or the reduced raw data generated by the reduced raw data generation unit 21 is used as display image data. Convert and display. Further, the display unit 15 has a function as a finder for continuously displaying images captured by the CCD 3 in the photographing mode.

  The strobe drive circuit 16 drives and controls the strobe 17 to emit light when the subject brightness is low according to a control signal input from the control unit 11.

  The strobe 17 boosts the battery voltage to a predetermined high voltage and stores it as a charge in a capacitor. The strobe 17 is driven by the strobe drive circuit 16 to emit light from the X tube with the electric charge stored in the capacitor and irradiate the subject with auxiliary light.

The focal length adjustment circuit 18 controls the motor 20 for adjusting the focal length by moving the lens 1 in accordance with a control signal input from the control unit 11.
The automatic focus driving circuit 19 controls the motor 20 for adjusting the focus (focus) by moving the lens 1 in accordance with a control signal input from the control unit 11.

Next, the operation of the imaging apparatus 100 according to the first embodiment will be described.
With reference to the flowchart of FIG. 4, an image data recording process executed in the imaging apparatus 100 will be described.

  When the shutter button of the operation unit 14 is pressed and shooting is instructed (step S1), the image reference signal obtained from the CCD 3 is converted into a digital signal by the A / D converter 5 to generate scene reference raw data. At the same time (step S2), photographing information data is generated in the photographing information data generation unit 12 (step S3), and imaging device characteristic correction data is generated in the device characteristic correction data generation unit 13 (step S4). Next, reduced raw data is generated in the reduced raw data generation unit 21 by performing a process of reducing the image size of the scene reference raw data generated in step S2 (step S5).

  Next, in the header information processing unit 8, the shooting information data generated in step S3, the imaging device characteristic correction data generated in step S4, and the image data generated in step S5 are added to the file header of the scene reference raw data generated in step S2. The reduced raw data thus attached is attached as tag information (step S6), and an attached data file (see FIG. 2) is created (step S7). The attached data file is recorded and stored in the recording medium of the storage device 9 (step S8), and the image data recording process is completed.

  The recording media on which the attached data file is recorded is taken out from the main body of the image pickup device and mounted on an external device such as an image processing device or an image recording device. The external device is optimal for forming an appreciation image on an output medium. Appreciation image reference data is generated by performing image processing.

  As described above, according to the imaging apparatus 100 of the first embodiment, by adding the imaging apparatus characteristic correction data and the reduced raw data to the scene reference raw data depending on the characteristics of the imaging apparatus and recording them on the recording medium, When outputting the image data recorded on the recording medium to the output medium, it is possible to easily identify the shooting scene (front light, backlight, under, strobe) at the time of shooting using the reduced raw data. It is possible to perform optimum image processing corresponding to the shooting scene on the standardized scene reference image data generated from the raw data using the imaging device characteristic correction data. Therefore, it is possible to obtain high-quality image data without causing information loss of captured image information.

  In addition to the imaging device characteristic correction data and the reduced raw data, the shooting scene data based on the reduced raw data is recorded by attaching the shooting information data, which is the shooting condition setting at the time of shooting, to the scene reference raw data and recording it on the recording medium. By collating the photographic scene as the discrimination result of the discrimination process with the photographic scene included in the photographic information data, it is possible to improve the discrimination accuracy of the photographic scene.

Next, an image recording apparatus that performs image processing that optimizes image data recorded on a recording medium for the formation of an appreciation image on an output medium will be described.
<Configuration of image recording device>
FIG. 5 shows an external configuration of the image recording apparatus 200 according to Embodiment 1 of the present invention.

  The image recording apparatus 200 is provided with a magazine loading unit 203 on one side of a main body 202. In the main body 202, an exposure processing unit 204 that exposes a silver salt photographic paper as an output medium, and an exposed silver salt print. A print creating unit 205 is provided for creating a print by developing and drying the paper. The print created by the print creation unit 205 is discharged to a tray 206 provided on the other side of the main body 202. Further, a control unit 207 that controls each unit constituting the image recording apparatus 200 is provided inside the main body 202.

  In addition, a display unit 208, a film scanner unit 209 that is a transparent document reading device, a reflective document input device 210, and an operation unit 211 are disposed on the upper portion of the main body 202. Furthermore, the main body 202 is provided with an image reading unit 214 that can read image data recorded on various recording media, and an image writing unit 215 that writes image data on various recording media.

  There is a photographic photosensitive material as an original read from the film scanner unit 209 or the reflective original input device 210. Examples of the photographic light-sensitive material include a color negative film, a color reversal film, a black and white negative film, a black and white reversal film and the like, and frame image information captured by an analog camera is recorded. The film scanner unit 209 converts the frame image information recorded on the photographic photosensitive material into digital image data to obtain frame image data. Further, when the photographic photosensitive material is color paper which is silver salt photographic paper, the reflective original input device 210 converts the frame image information recorded on the silver salt photographic paper into frame image data by a flatbed scanner.

  The image reading unit 214 includes a PC card adapter 214a and a floppy (registered trademark) disk adapter 214b, into which a PC card 213a and a floppy (registered trademark) disk 213b can be inserted. The PC card 213a has, for example, a memory in which a plurality of frame image data is stored after being captured by a digital camera. For example, a plurality of frame image data captured by a digital camera is recorded on the floppy (registered trademark) disk 213b. Examples of recording media on which frame image data is recorded in addition to the PC card 213a and the floppy (registered trademark) disk 213b include a multimedia card (registered trademark), a memory stick (registered trademark), MD data, and a CD-ROM. It is done.

  The image writing unit 215 includes a floppy (registered trademark) disk adapter 215a, an MO adapter 215b, and an optical disk adapter 215c. ing. Examples of the optical disk 216c include a CD-R and a DVD-R.

  In FIG. 5, the operation unit 211, the display unit 208, the film scanner unit 209, the reflection original input device 210, and the image reading unit 214 are integrally provided in the main body 202. One or more of them may be provided separately.

  In the image recording apparatus 200 shown in FIG. 5, there is illustrated an apparatus that creates a print by exposing to a photosensitive material and developing it. However, the print creation method is not limited to this. You may make it use systems, such as a photography system, a heat sensitive system, and a sublimation system.

<Internal configuration of image recording device>
FIG. 6 shows the internal configuration of the image recording apparatus 200. As shown in FIG. 6, the image recording apparatus 200 includes a control unit 207, an exposure processing unit 204, a print creation unit 205, a film scanner unit 209, a reflection original input device 210, an image reading unit 214, an image writing unit 215, data The storage unit 271, the template storage unit 272, the operation unit 211, and the display unit 208 are configured.

  The control unit 207 is composed of a microcomputer, and cooperates with various control programs stored in a storage unit (not shown) such as a ROM (Read Only Memory) and a CPU (Central Processing Unit) (not shown). The operation of each part constituting the image recording apparatus 200 is controlled.

  Further, the control unit 207 includes an image processing unit 270, and based on an input signal from the information input unit 12 of the operation unit 211, image data read from the film scanner unit 209 or the reflective original input device 210, image reading The image data read from the unit 214 or image data input from an external device via communication means (not shown) is subjected to image processing to form exposure image information, which is output to the exposure processing unit 204. The image processing unit 270 performs conversion processing according to the output form on the image data that has been subjected to image processing, and outputs the image data to a designated output destination. Output destinations of the image processing unit 270 include a display unit 208, an image writing unit 215, communication means (not shown), and the like.

  The exposure processing unit 204 exposes an image on the photosensitive material and outputs the photosensitive material to the print creating unit 205. The print creating unit 205 develops and exposes the exposed photosensitive material to create prints P1, P2, and P3. The print P1 is a service size, high-definition size, panoramic size print, the print P2 is an A4 size print, and the print P3 is a business card size print.

  The film scanner unit 209 reads a frame image recorded on a transparent original such as a developed negative film N or a reversal film captured by an analog camera, and acquires a digital image signal of the frame image. The reflection original input device 210 reads an image on a print P (photo print, document, various printed materials) by a flat bed scanner, and acquires a digital image signal.

  The operation unit 211 is provided with information input means 212. The information input unit 212 is configured by a touch panel, for example, and outputs an operation signal of the information input unit 212 to the control unit 207. Further, the operation unit 211 may be configured to include a keyboard and a mouse.

  The image reading unit 214 reads frame image information recorded on the PC card 213 a and the floppy (registered trademark) disk 213 b and transfers the frame image information to the control unit 207. The image reading unit 214 includes, as the image transfer means 230, a PC card adapter 214a, a floppy (registered trademark) disk adapter 214b, and the like. The image reading unit 14 reads frame image information recorded on the PC card 213a inserted into the PC card adapter 214a or the floppy (registered trademark) disk 213b inserted into the floppy (registered trademark) disk adapter 214b. Transfer to the control unit 207. For example, a PC card reader or a PC card slot is used as the PC card adapter 214a.

  The image writing unit 215 includes a floppy (registered trademark) disk adapter 215 a, an MO adapter 215 b, and an optical disk adapter 215 c as the image transport unit 231. In accordance with a write signal input from the control unit 207, the image writing unit 215 includes a floppy (registered trademark) disk 216a inserted into the floppy (registered trademark) disk adapter 215a, an MO 216b inserted into the MO adapter 215b, The image data generated by the image processing method according to the present invention is written to the optical disk 216c inserted into the optical disk adapter 215c.

  The data storage means 271 stores and sequentially stores image information and corresponding order information (information on how many prints are created from images of which frames, print size information, etc.).

  The template storage unit 272 stores sample image data (data indicating a background image, an illustration image, etc.) corresponding to the sample identification information D1, D2, and D3, and a template for setting a synthesis area with the sample image data. Store at least one piece of data. Here, when a predetermined template is selected from a plurality of templates stored in advance in the template storage unit 272 by an operator's operation (the operator's operation is based on a client instruction), the control unit 207 displays frame image information When the sample identification information D1, D2, D3 is designated by the operator's operation (the operator's operation is based on the instruction of the client), the designated sample identification is performed. Sample image data is selected based on the information D1, D2, and D3, the selected sample image data is combined with image data and / or character data ordered by the client, and as a result, the sample image desired by the client is obtained. Create a print based on the data. The synthesis using this template is performed by a well-known chroma key method.

The sample identification information is not limited to the three types of sample identification information D1, D2, and D3, but may be more or less than three types.
The sample identification information D1, D2, and D3 for specifying the print sample is configured to be input from the operation unit 211. However, the sample identification information D1, D2, and D3 are input to the print sample or the order sheet. Since it is recorded, it can be read by reading means such as OCR (Optical Character Reader). Or an operator can also input from a keyboard.

  In this way, sample image data is recorded corresponding to the sample identification information D1 for designating the print sample, the sample identification information D1 for designating the print sample is input, and based on the input sample identification information D1. Select the sample image data, synthesize the selected sample image data with the image data and / or text data based on the order, and create a print based on the specified sample. Can actually place a print order and meet the diverse requirements of a wide range of users.

  Also, the first sample identification information D2 designating the first sample and the image data of the first sample are stored, and the second sample identification information D3 designating the second sample and the second sample The image data is stored, the sample image data selected based on the designated first and second sample identification information D2 and D3, and the image data and / or character data based on the order are synthesized, and according to the designation In order to create a print based on a sample, it is possible to synthesize a wider variety of images, and it is possible to create a print that meets a wider variety of user requirements.

  The display unit 208 is configured by a display such as a CRT or LCD, and performs display processing in accordance with a display control signal input from the control unit 207.

  The image processing unit 270 of the control unit 207 uses a communication unit (not shown) from another computer in the facility where the image recording apparatus 200 is installed or a remote computer via a communication network such as the Internet. It is also possible to receive image data representing a picked-up image and a work instruction such as printing, and to perform image processing or create a print by remote operation.

  Further, the image processing unit 270 uses communication means (not shown) to send image data representing the captured image after the image processing of the present invention and the accompanying order information to another computer in the facility, the Internet, etc. It is also possible to send to a distant computer via

  As described above, the image recording apparatus 200 includes an input unit that captures image information obtained by dividing and metering images of various recording media and an image original, and image information of the input image that is input from the input unit is “size of output image”. Image processing means for performing processing so as to obtain an image that gives a favorable impression when acquiring or estimating the information “size” and “the size of the main subject in the output image” and observing the image on the output medium; To send image data and accompanying order information to another computer in the facility or to a distant computer via the Internet, etc. Means.

  FIG. 7 shows an internal configuration when the image recording apparatus 200 is divided into an image processing apparatus 301 and an output unit 302 that outputs image data processed by the image processing apparatus 301.

  As illustrated in FIG. 7, the image processing apparatus 301 includes an input unit 303, a header information analysis unit 304, an imaging device characteristic correction processing unit 305, and an optimization processing unit 306.

  The input unit 303 includes the image reading unit 214 in FIG. 6 and includes a mounting unit for mounting a recording medium. When the recording medium is loaded in the loading unit, the input unit 303 reads the data file recorded on the recording medium and outputs the data file to the header information analysis unit 304. In the present embodiment, the input unit 303 is described as reading a data file from the attached recording medium. However, the input unit 303 includes a wired or wireless communication unit, and the data file is input via the communication unit. May be.

  The header information analysis unit 304 analyzes the data file input from the input unit 303, divides the data file into scene reference raw data, imaging device characteristic correction data, reduced raw data, and shooting information data. Is output to the scene reference image data generation unit 309 in the imaging device characteristic correction processing unit 305, the imaging device characteristic correction data is output to the device characteristic correction processing unit 307, and the reduced raw data is output to the scene reference reduced image data generation unit 310. The imaging information data is output to the imaging information data processing unit 313 in the optimization processing unit 306.

  The imaging device characteristic correction processing unit 305 includes a device characteristic correction processing unit 307, a processing condition table 308, a scene reference image data generation unit 309, a scene reference reduced image data generation unit 310, and a temporary storage memory 311.

  The device characteristic correction processing unit 307 determines the generation conditions of the scene reference image data and the scene reference reduced image data by referring to the imaging device characteristic correction data input from the header information analysis unit 304 and the processing condition table 308. The processing condition table 308 stores processing conditions for generating scene reference image data and scene reference reduced image data for each characteristic of the imaging apparatus.

  The scene reference image data generation unit 309 performs imaging device characteristic correction processing on the scene reference raw data input from the header information analysis unit 304 in accordance with the generation conditions determined by the device characteristic correction processing unit 307, and Standardized scene reference image data independent of characteristics is generated and output to the temporary storage memory 311. Specifically, in the imaging device characteristic correction processing, the signal intensity of each color channel based on at least the spectral sensitivity inherent to the imaging device of the imaging device that generated the scene reference raw data is set to a standard such as RIM RGB, ERIMM RGB, or scRGB. Includes mapping to color space.

  The scene reference reduced image data generation unit 310 is standardized by subjecting the reduced raw data input from the header information analysis unit 304 to imaging device characteristic correction processing according to the generation conditions determined by the device characteristic correction processing unit 307. The reduced scene reference reduced image data is generated and output to the scene discrimination processing unit 312 in the optimization processing unit 306.

  The temporary storage memory 311 temporarily stores the scene reference image data generated by the scene reference image data generation unit 309.

  FIG. 9 shows a data file that is the processing result of the imaging device characteristic correction processing in the imaging device characteristic correction processing unit 305. As shown in FIG. 9, the scene reference raw data is converted into scene reference image data and the reduced raw data is converted into scene reference reduced image data in the data file shown in FIG.

  The optimization processing unit 306 includes a scene determination processing unit 312, a shooting information data processing unit 313, an appreciation image reference data generation unit 314, a temporary storage memory 315, and a setting input unit 316.

  The scene discrimination processing unit 312 discriminates a shooting scene at the time of shooting using the scene reference reduced image data generated by the scene reference reduced image data generation unit 310 and outputs the determination result to the appreciation image reference data generation unit 314. . The shooting scene determination process performed by the scene determination processing unit 312 will be described in detail later with reference to FIGS.

  The shooting information data processing unit 313 determines a generation condition for generating appreciation image reference data corresponding to the shooting condition based on the shooting information data input from the header information analysis unit 304.

  The appreciation image reference data generation unit 314 reads the scene reference image data from the temporary storage memory 311, the generation conditions of the appreciation image reference data determined by the shooting information data processing unit 313, the determination result of the scene determination processing unit 312, Optimization for obtaining an optimal image at the output destination for the scene reference image data based on the information of the output destination (storage device 318, output device 317, display unit 208) designated from the setting input unit 316 Processing is performed to generate appreciation image reference data, which is output to the temporary storage memory 315 together with operation information of the setting input unit 316. The optimization processing includes, for example, gradation conversion processing according to the shooting scene determined by the scene determination processing unit 312, compression to the output destination color gamut, gradation compression from 16 bits to 8 bits, and reduction of the number of output pixels , Processing for handling output characteristics (LUT) of output devices and display devices, and the like are included. Furthermore, image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment, and dodging processing is included. A method for determining a gradation conversion curve representing a gradation conversion process corresponding to a shooting scene will be described in detail later with reference to FIGS.

  The temporary storage memory 315 temporarily stores the appreciation image reference data generated by the appreciation image reference data generation unit 314, and the output destination (storage device 318, output device 317, display unit 208) set by the setting input unit 316. Any of the above).

  The setting input unit 316 is an input device for designating an output destination of the appreciation image reference data generated by the appreciation image reference data generation unit 314, and corresponds to the operation unit 211 of FIG.

  Of the components of the image processing apparatus 301, components other than the input unit 303 and the setting input unit 316 are included in the image processing unit 270 illustrated in FIG.

  The output unit 302 includes a display unit 208, an output device 317 corresponding to the exposure processing unit 204 and the print creation unit 205 in FIG. 6, and a storage device 318 corresponding to the image writing unit 215 in FIG.

Next, the operation of the image recording apparatus 200 according to the first embodiment will be described.
With reference to FIG. 8, image processing executed in the image recording apparatus 200 will be described.

  When data is input to the input unit 303 (that is, when a recording medium is mounted on the mounting unit) (step S20), the data file recorded on the recording medium is read, and the header information analysis unit 304 The contents of the data file are analyzed (step S21), and the data file is divided into scene reference raw data, imaging device characteristic correction data, reduced raw data, and photographing information data.

  The shooting information data processing unit 313 determines generation conditions for generating appreciation image reference data corresponding to shooting conditions based on the shooting information data obtained in step S21 (step S22). The apparatus characteristic correction processing unit 307 determines the generation conditions of the scene reference image data and the scene reference reduced image data by referring to the imaging apparatus characteristic correction data obtained in step S21 and the processing condition table 308, and is obtained in step S21. The scene reference raw data and the reduced raw data are subjected to imaging device characteristic correction processing according to the determined generation conditions (step S23), and scene reference image data is generated (step S24), and the scene reference reduced image is also generated. Data is generated (step S25).

  When the scene reference reduced image data is generated in step S25, a shooting scene determination process (see FIGS. 10 to 21) for determining a shooting scene at the time of shooting is performed based on the scene reference reduced image data (step S26). ). Next, optimization processing is performed on the scene reference image data generated in step S24 based on the generation conditions of the viewing image reference data determined in step S22 and the determination result of the shooting scene determination processing in step S26. (Step S27) Appreciation image reference data is generated (Step S28).

  The appreciation image reference data is subjected to processing specific to the output destination set in the setting input unit 316 (operation unit 211) (step S29), and is output from the output destination device (step S29). S30), the image processing ends.

<Shooting scene discrimination processing>
Next, with reference to the flowchart of FIG. 10, a shooting scene determination process executed in the scene determination processing unit 312 will be described.

  First, the captured image data (scene reference reduced image data) is divided into predetermined image areas, and occupancy ratios (first occupancy ratio, second occupancy ratio) indicating the ratio of each divided area to the entire captured image data are set. An occupation ratio calculation process to be calculated is performed (step T1). Details of the first occupancy rate calculation process and the second occupancy rate calculation process will be described later with reference to FIGS. 11 and 17, respectively.

  Next, the occupancy ratio (first occupancy ratio, second occupancy ratio) calculated in step T1 is set in advance according to at least the average luminance value of the skin color in the screen center portion of the captured image data and the imaging conditions. Based on the obtained coefficients, indices (indexes 1 to 6) for specifying the shooting scene (representing the light source condition and the exposure condition quantitatively) are calculated (step T2). Next, a shooting scene is determined based on the index calculated in step T2 (step T3), and the main shooting scene determination process ends. The index calculation process in step T2 and the shooting scene determination method in step T3 will be described in detail later.

  Next, the first occupation ratio calculation process will be described in detail with reference to the flowchart of FIG.

  First, RGB (Red, Green, Blue) values of photographed image data (scene reference reduced image data) are converted into the HSV color system (step T10). FIG. 12 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 12, the values of digital image data as input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step T11). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is found that the contribution to the discrimination of the shooting scene is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh color hue regions (H = 0 to 39, 330 to 359), a hue color '(H) that satisfies the following equation (1) is defined as a flesh color region (H1), and a region that does not satisfy the equation (1) is ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance (Y) = InR x 0.30 + InG x 0.59 + InB x 0.11 (A)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (1)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formulas (A) and (1).

When the two-dimensional histogram is created, a first occupancy ratio indicating a ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step T12), and the first The occupation rate calculation process ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

<Calculation method of index 1 and index 2>
Next, a method for calculating the index 1 and the index 2 will be described.
Table 2 shows the first coefficient necessary for calculating the index 1 that quantitatively indicates the accuracy of strobe shooting, that is, the brightness state of the face area at the time of strobe shooting, for each divided area. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy ratio Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 13 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 13, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 15 shows a curve (coefficient curve) in which the first coefficient in the skin color area (H1) and the first coefficient in the other areas (green hue area (H3)) continuously change over the entire brightness. It is shown. According to Table 2 and FIG. 15, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(2-1)〜(2-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) (2-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) (2-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) (2-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) (2-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index 1 is defined as in Expression (2).

Accordingly, the sum of the H1 to H4 regions is expressed as the following formulas (2-1) to (2-4).
H1 area sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) (2-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) (2-2)
Sum of H3 area = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) (2-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) (2-4)

The index 1 is defined as Expression (3) using the sum of the H1 to H4 regions shown in Expressions (2-1) to (2-4).
Index 1 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 4.424 (3)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index 2 that quantitatively indicates the accuracy as backlight photographing, that is, the brightness state of the face region at the time of backlight photographing. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy rate Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 14 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupation ratio calculated from the region (r4) distributed in the intermediate lightness of the flesh color hue region in FIG. 14, and the low lightness (shadow) region ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 16 shows the second coefficient in the skin color area (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 16, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(4-1)〜(4-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) (4-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) (4-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 (4-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 (4-4) When the second coefficient in the lightness area vi and the hue area Hj is Dij, the sum of the Hk areas for calculating the index 2 is defined as in Expression (4).

Therefore, the sum of the H1 to H4 regions is expressed by the following equations (4-1) to (4-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) (4-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (4-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (4-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (4-4)

The index 2 is defined as in Expression (5) using the sum of the H1 to H4 regions shown in Expressions (4-1) to (4-4).
Index 2 = sum of H1 area + sum of H2 area + sum of H3 area + sum of H4 area + 1.554 (5)
Since the index 1 and the index 2 are calculated based on the brightness of the captured image data and the distribution amount of the hue, the index 1 and the index 2 are effective for determining a captured scene when the captured image data is a color image.

  Next, the second occupancy rate calculation process executed to calculate the index 3 will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data (scene reference reduced image data) are converted into the HSV color system (step T20). Next, the photographed image data is divided into regions composed of combinations of the distance from the outer edge of the photographed image screen and the brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step T21). Hereinafter, the area division of the captured image data will be described in detail.

  18A to 18D show four regions n1 to n4 that are divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 18A is an outer frame, a region n2 shown in FIG. 18B is a region inside the outer frame, and a region n3 shown in FIG. An inner area, an area n4 shown in FIG. 18D, is an area at the center of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating a ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire photographed image) is calculated (step T22). The occupation rate calculation process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１９は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 <Calculation method of index 3>
Next, a method for calculating the index 3 will be described.
Table 5 shows the third coefficient necessary for calculating the index 3 for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy rate Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the shooting conditions.

FIG. 19 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、n1〜n4領域の和は、下記の式(6-1)〜(6-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 (6-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 (6-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 (6-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) (6-4) If the third coefficient in the brightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index 3 is defined as in Expression (6).

Therefore, the sum of the n1 to n4 regions is expressed by the following formulas (6-1) to (6-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (6-1)
Sum of n2 area = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (6-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (6-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) (6-4)

The index 3 is defined as Expression (7) using the sum of the n1 to n4 regions shown in Expressions (6-1) to (6-4).
Index 3 = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (7)
The index 3 is calculated based on a compositional characteristic (distance from the outer edge of the screen of the captured image data) based on the lightness distribution position of the captured image data, so that it can determine not only a color image but also a monochrome image capturing scene. It is also effective.

<Calculation method of index 4>
Next, the calculation process of the index 4 will be described with reference to the flowchart of FIG.

  First, the luminance Y is calculated from the RGB (Red, Green, Blue) values of the photographed image data (scene reference reduced image data) using Expression (A). Next, the average luminance value x1 of the skin color area at the center of the screen of the photographed image data is calculated (step T23). Here, the center of the screen is, for example, an area configured by an area n3 and an area n4 in FIG. Next, a difference value x2 between the maximum luminance value and the average luminance value of the captured image data is calculated (step T24).

Next, the standard deviation x3 of the luminance of the captured image data is calculated (step T25), and the average luminance value x4 at the center of the screen is calculated (step T26). Next, a comparison value x5 between the difference value between the maximum luminance value Yskin_max and the minimum luminance value Yskin_min of the skin color area in the photographed image data and the average luminance value Yskin_ave of the skin color area is calculated (step T27). This comparison value x5 is expressed as the following equation (8).
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (8)

Next, the index 4 is calculated by multiplying each of the values x1 to x5 calculated in steps T23 to S27 by a fourth coefficient set in advance according to the shooting conditions and taking the sum (step T28). ). The index 4 is defined as the following formula (9).
Index 4 = 0.06 x x 1 + 1.13 x x 2 + 0.02 x x 3 + (-0.01) x x 4 + 0.03 x x 5-6.50 (9)
This index 4 has not only the compositional characteristics of the screen of the photographed image data but also luminance histogram distribution information, and is particularly effective for discriminating between the flash photography scene and the under photography scene.

The average luminance value of the skin color area at the center of the screen of the photographed image data is set as an index 4 ′. Here, the center of the screen is, for example, an area composed of the area n2, the area n3, and the area n4 in FIG. At this time, the index 5 is defined as in Expression (10) using the index 1, index 3, and index 4 ′, and the index 6 is defined by Expression (11) using the index 2, index 3, and index 4 ′. Is defined as
Index 5 = 0.46 × Index 1 + 0.61 × Index 3 + 0.01 × Index 4′−0.79 (10)
Indicator 6 = 0.58 × Indicator 2 + 0.18 × Indicator 3 + (-0.03) × Indicator 4 '+ 3.34 (11)
Here, the weighting coefficient by which each index is multiplied in Expression (10) and Expression (11) is set in advance according to the shooting conditions.

  Next, a method for discriminating a shooting scene based on the values of the indices 4 to 6 will be described.

  In FIG. 21A, 60 images are shot in each of the shooting scenes of the front light, the backlight, and the strobe, and the index 5 and the index 6 are calculated for a total of 180 digital image data. The values of index 6 are plotted. According to FIG. 21A, when the value of index 5 is greater than 0.5, there are many flash photography scenes, the value of index 5 is 0.5 or less, and the value of index 6 is greater than −0.5. It can be seen that there are many backlight scenes. In this way, the shooting scene can be determined quantitatively based on the values of the index 5 and the index 6.

  Furthermore, by adding the index 4 to the index 5 and the index 6 that can discriminate each shooting scene of the front light, the backlight, and the strobe, the shooting scene can be discriminated three-dimensionally, and the discrimination accuracy of the shooting scene is further improved. Is possible. The index 4 is particularly effective for discriminating between a flash photography scene in which gradation adjustment is performed to darken the entire image and an under photography scene in which gradation adjustment is performed to brighten the entire image.

FIG. 21B is a graph in which index 4 and index 5 of an image with index 5 greater than 0.5 are calculated and plotted among 60 captured images of the strobe shooting scene and the under shooting scene. According to FIG. 21B, it can be seen that when the value of index 4 is larger than 0, there are many flash shooting scenes, and when the value of index 4 is 0 or less, there are many under shooting scenes. Table 6 shows the details of the shooting scene discrimination based on the values of index 4, index 5, and index 6.

<Determination method of gradation conversion curve>
When the shooting scene is determined, a gradation adjustment method for the shooting image data (scene reference image data) is determined according to the determined shooting scene. As shown in FIG. 22, the gradation adjustment method A (FIG. 22A) is selected when the shooting scene is a follow light, and the gradation adjustment method B (FIG. 22B) is selected when the shooting scene is a backlight. If it is selected and it is a strobe, the gradation adjustment method C (FIG. 22C) is selected, and if it is under, the gradation adjustment method B (FIG. 22B) is selected.

  When the gradation adjustment method is determined, parameters (gradation adjustment parameters) necessary for gradation adjustment are calculated based on the calculated index. Hereinafter, a method for calculating the gradation adjustment parameter will be described. In the following description, it is assumed that the 8-bit captured image data is converted in advance to 16 bits, and the unit of the value of the captured image data is 16 bits.

As the tone adjustment parameters, the following parameters P1 to P9 are calculated.
P1: Average brightness of the entire shooting screen
P2: Block division average brightness
P3: Average brightness of skin tone area (H1)
P4: Brightness correction value 1 = P1-P2
P5: Reproduction target correction value = Brightness reproduction target value (30360)-P4
P6: Offset value 1 = P5-P1
P7: Key correction value
P7 ': Key correction value 2
P8: Brightness correction value 2
P9: Offset value 2 = P5-P8-P1

Here, a calculation method of the parameter P2 (block division average luminance) will be described with reference to FIGS.
First, a CDF (cumulative density function) is created in order to normalize captured image data. Next, the maximum value and the minimum value are determined from the obtained CDF. The maximum value and the minimum value are obtained for each RGB. Here, the maximum value and the minimum value obtained for each RGB are Rmax, Rmin, Gmax, Gmin, Bmax, and Bmin, respectively.

Next, normalized image data for any pixel (Rx, Gx, Bx) of the captured image data is calculated. When Rx normalization data in the R plane is R _point , Gx normalization data in the G plane is G _point , and Bx normalization data in the B plane is B _point , the normalization data R _point , G _point , B _point are , Respectively, are expressed as in Expression (12) to Expression (14).
R _point = {(Rx−Rmin) / (Rmax−Rmin)} × 65535 (12)
G _point = {(Gx−Gmin) / (Gmax−Gmin)} × 65535 (13)
B _point = {(Bx−Bmin) / (Bmax−Bmin)} × 65535 (14)
Next, the luminance N _point of the pixel (Rx, Gx, Bx) is calculated by Expression (15).
N _point = (B _point + G _point + R _point ) / 3 (15)

  FIG. 23A shows a frequency distribution (histogram) of luminance of RGB pixels before normalization. In FIG. 23A, the horizontal axis represents luminance, and the vertical axis represents pixel frequency. This histogram is created for each RGB. When the luminance histogram is created, normalization is performed for each plane with respect to the captured image data according to equations (12) to (14). FIG. 23B shows a histogram of luminance calculated by the equation (15). Since the captured image data is normalized by 65535, each pixel takes an arbitrary value between the maximum value 65535 and the minimum value 0.

  When the luminance histogram shown in FIG. 23B is divided into blocks divided by a predetermined range, a frequency distribution as shown in FIG. 23C is obtained. In FIG. 23C, the horizontal axis represents the block number (luminance) and the vertical axis represents the frequency.

  Next, processing for deleting highlight and shadow areas is performed from the luminance histogram shown in FIG. This is because the average brightness is very high in white walls and snow scenes, and the average brightness is very low in dark scenes, so highlights and shadow areas adversely affect average brightness control. . Therefore, by limiting the highlight area and shadow area of the luminance histogram shown in FIG. 23C, the influence of both areas is reduced. If the high luminance region (highlight region) and the low luminance region (shadow region) are deleted from the luminance histogram shown in FIG. 24A (or FIG. 23C), the result is as shown in FIG.

  Next, as shown in FIG. 24C, an area having a frequency greater than a predetermined threshold is deleted from the luminance histogram. This is because if there is a part having an extremely high frequency, the data in this part strongly affects the average luminance of the entire captured image, and thus erroneous correction is likely to occur. Therefore, as shown in FIG. 24C, the number of pixels equal to or larger than the threshold is limited in the luminance histogram. FIG. 24D shows a luminance histogram after the pixel number limiting process is performed.

  Each block number of the luminance histogram (FIG. 24 (d)) obtained by deleting the high luminance region and the low luminance region from the normalized luminance histogram and further limiting the number of accumulated pixels, and the respective frequencies The parameter P2 is obtained by calculating the average luminance based on the above.

The parameter P1 is an average value of the luminance of the entire captured image data, and the parameter P3 is an average value of the luminance of the skin color area (H1) in the captured image data. The key correction value of the parameter P7, the key correction value 2 of the parameter P7 ′, and the luminance correction value 2 of the parameter P8 are defined as Expression (16), Expression (17), and Expression (18), respectively.
P7 (key correction value) = {P3-((index 6/6) x 18000 + 22000)} / 24.78 (16)
P7 '(key correction value 2) = {P3-((index 4/6) x 10000 + 30000)} / 24.78 (17)
P8 (Luminance correction value 2) = (Indicator 5/6) x 17500 (18)

Next, a method for determining a gradation conversion curve corresponding to each shooting scene based on the gradation adjustment parameter will be described.
<For direct light>
When the shooting scene is front light, offset correction (parallel shift of 8-bit value) for matching the parameter P1 with P5 is performed by the following equation (19).
RGB value of output image = RGB value of input image + P6 (19)
Therefore, when the photographic scene is front light, a gradation conversion curve corresponding to Expression (19) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (19).

<Backlight>
When the shooting scene is backlit, a gradation conversion curve corresponding to the parameter P7 (key correction value) shown in Expression (16) is selected from the plurality of gradation conversion curves shown in FIG. A specific example of the gradation conversion curve of FIG. 22B is shown in FIG. The correspondence between the value of parameter P7 and the selected gradation conversion curve is shown below.
When -50 <P7 <+50 → L3
When +50 ≤ P7 <+150 → L4
When +150 ≤ P7 <+250 → L5
When -150 <P7 ≤ -50 → L2
When -250 <P7 ≤ -150 → L1
When the shooting scene is backlit, it is preferable to perform a dodging process together with the gradation conversion process. In this case, it is desirable that the degree of the dodging process be adjusted according to the index 6 indicating the backlight intensity.

<In case of under>
When the shooting scene is under, a gradation conversion curve corresponding to the parameter P7 ′ (key correction value 2) shown in Expression (17) is selected from the plurality of gradation conversion curves shown in FIG. The Specifically, the gradation conversion curve corresponding to the value of the parameter P7 ′ is selected from the gradation conversion curves shown in FIG. 25 in the same manner as the method for selecting the gradation conversion curve when the shooting scene is backlit. The When the shooting scene is under, the dodging process as shown in the case of backlight is not performed.

<When using a flash>
When the shooting scene is a strobe, offset correction (parallel shift of 8-bit value) is performed by Expression (20).
RGB value of output image = RGB value of input image + P9 (20)
Therefore, when the shooting scene is a strobe, a gradation conversion curve corresponding to Expression (20) is selected from a plurality of gradation conversion curves shown in FIG. Alternatively, the gradation conversion curve may be calculated (determined) based on Expression (20). When the value of parameter P9 in equation (20) exceeds a predetermined value α, a curve corresponding to the key correction value P9-α is selected from curves L1 to L5 shown in FIG. Is done.

  In the present embodiment, when the gradation conversion process is actually performed on the captured image data, the above-described image processing conditions are changed from 16 bits to 8 bits.

  Also, if the gradation adjustment method differs greatly between the following light, backlight, strobe, and under, there is a concern about the effect on the image quality when the shooting scene is misidentified. It is desirable to set an intermediate region where the tone adjustment method gradually changes.

  As described above, according to the image recording apparatus 200 of the first embodiment, the image recording apparatus 200 is optimized based on the imaging apparatus characteristic correction data and the imaging information data attached to the file header of the scene reference raw data output from the imaging apparatus. By generating appreciation image reference data, it is possible to obtain appreciation image reference data of higher quality than the image data obtained by the imaging apparatus without any information loss of the captured image information.

  Also, standardized scene reference reduced image data is generated from the reduced raw data attached to the file header of the scene reference raw data output from the imaging device, and at the time of shooting based on the generated scene reference reduced image data By discriminating the shooting scene, it is possible to perform optimum gradation conversion processing according to the shooting scene. In particular, the accuracy of shooting scene discrimination can be improved by collating a shooting scene as a determination result of the shooting scene determination processing with a shooting scene included in shooting information data.

  In the first embodiment, as shown in FIG. 2, the case is shown where shooting information data indicating shooting condition settings at the time of shooting is attached to the scene reference raw data, as shown in FIG. Instead of attaching the shooting information data to the file header of the scene reference raw data, only the imaging device characteristic correction data and the reduced raw data may be attached to create a data file. In this case, the data file which is the processing result of the imaging device characteristic correction processing in the imaging device characteristic correction processing unit 305 does not include the shooting information data as shown in FIG. 26B. In the image processing of FIG. The process of step S22 is not performed. Even if there is no shooting information data, the shooting scene determination process based on the scene reference reduced image data can be used to determine the shooting scene with sufficient accuracy, and the optimum gradation conversion process according to the shooting scene is performed. It is possible.

[Embodiment 2]
A second embodiment of the present invention will be described with reference to FIGS.

<Configuration of imaging device>
FIG. 27 shows a main part configuration of an imaging apparatus 101 according to Embodiment 2 of the present invention. The imaging apparatus 101 according to the second embodiment has a configuration in which a scene reference reduced image data generation unit 22 is added to the imaging apparatus 100 according to the first embodiment. Below, the same code | symbol is attached | subjected to the component same as the imaging device 100 of Embodiment 1, and only a different part from the imaging device 100 is demonstrated.

  The scene reference reduced image data generation unit 22 uses the imaging device characteristic correction data generated by the device characteristic correction data generation unit 13 for the reduced raw data generated by the reduced raw data generation unit 21, and performs imaging device characteristic correction processing. To generate standardized scene reference reduced image data. The scene reference reduced image data is converted into display image data and displayed on the display unit 15.

  The header information processing unit 8 uses the shooting information data generated by the shooting information data generation unit 12 and the device characteristic correction data generation unit 13 in the file header (header area) of the scene reference raw data stored in the temporary storage memory 6. An attached data file is created by attaching the generated imaging device characteristic correction data and the scene reference reduced image data data generated by the scene reference reduced image data generation unit 22. FIG. 28 shows the data structure of the attached data file. The scene reference reduced image data, the imaging device characteristic correction data, and the like are recorded as tag information to be written in the header portion in the image file, such as a DNG tag according to an Adobe system recommended image format Digital Negative (DNG), for example. It is preferable.

Next, the operation of the imaging apparatus 101 according to the second embodiment will be described.
With reference to the flowchart of FIG. 29, the image data recording process performed in the imaging device 101 will be described.

  When the shutter button of the operation unit 14 is pressed and shooting is instructed (step S40), the image reference signal obtained from the CCD 3 is converted into a digital signal by the A / D converter 5, thereby generating scene reference raw data. At the same time (step S41), shooting information data is generated in the shooting information data generation unit 12 (step S42), and imaging device characteristic correction data is generated in the device characteristic correction data generation unit 13 (step S43).

  Next, reduced raw data is generated in the reduced raw data generation unit 21 by performing a process of reducing the image size of the scene reference raw data generated in step S41 (step S44). Next, the scene reference reduced image data generation unit 22 performs an imaging device characteristic correction process on the reduced raw data generated in step S44 using the imaging device characteristic correction data generated in step S43. Standardized scene reference reduced image data is generated (step S45).

  Next, in the header information processing unit 8, the shooting reference data generated in step S42, the imaging device characteristic correction data generated in step S43, and the generation in step S45 are added to the file header of the scene reference raw data generated in step S41. The reduced scene reference reduced image data is attached as tag information (step S46), and an attached data file (see FIG. 28) is created (step S47). The attached data file is recorded and stored in the recording medium of the storage device 9 (step S48), and the image data recording process is completed.

  The recording media on which the attached data file is recorded is taken out from the main body of the image pickup device and mounted on an external device such as an image processing device or an image recording device. The external device is optimal for forming an appreciation image on an output medium. Appreciation image reference data is generated by performing image processing.

  As described above, according to the imaging apparatus 101 of the second embodiment, the imaging apparatus characteristic correction data and the scene reference reduced image data are attached to the scene reference raw data depending on the characteristics of the imaging apparatus and recorded on the recording medium. Thus, when outputting the image data recorded on the recording medium to the output medium, it is possible to easily determine the shooting scene (front light, backlight, under, strobe) at the time of shooting using the scene reference reduced image data. In addition, it is possible to perform optimum image processing corresponding to the shooting scene on the standardized scene reference image data generated from the scene reference raw data using the imaging device characteristic correction data. Therefore, it is possible to obtain high-quality image data without causing information loss of captured image information.

  In addition to the imaging device characteristic correction data and the scene reference reduced image data, shooting information data, which is a shooting condition setting at the time of shooting, is attached to the scene reference raw data and recorded on a recording medium, whereby the scene reference reduced image data is obtained. By collating the photographic scene as the discrimination result of the photographic scene discrimination process based on the photographic scene data included in the photographic information data, it is possible to improve the photographic scene discrimination accuracy.

Next, an image recording apparatus that performs image processing that optimizes image data recorded on a recording medium for the formation of an appreciation image on an output medium will be described.
<Configuration of image recording device>
The external configuration and the internal configuration of the image recording apparatus 201 of the second embodiment are the same as those shown in FIGS. 5 and 6 in the first embodiment, and therefore the same reference numerals are used and the illustration and description of the configuration are omitted. .

  FIG. 30 shows an internal configuration when the image recording apparatus 201 according to the second embodiment is divided into an image processing apparatus 320 and an output unit 302 that outputs image data processed by the image processing apparatus 320. In FIG. 30, the same reference numerals are used for the same components as those of the image recording apparatus 200 according to the first embodiment shown in FIG. Only differences from the image recording apparatus 200 of the first embodiment will be described below.

  As illustrated in FIG. 30, the image processing device 320 includes an input unit 303, a header information analysis unit 304, an imaging device characteristic correction processing unit 321, and an optimization processing unit 322.

  The header information analysis unit 304 analyzes the data file input from the input unit 303, divides the data file into scene reference raw data, imaging device characteristic correction data, scene reference reduced image data, and shooting information data, and refers to the scene. The raw data is output to the scene reference image data generation unit 309 in the imaging device characteristic correction processing unit 321, the imaging device characteristic correction data is output to the device characteristic correction processing unit 307, and the scene reference reduced image data is optimized. To the scene discrimination processing unit 312a, and the shooting information data is output to the shooting information data processing unit 313.

  The imaging device characteristic correction processing unit 321 includes a device characteristic correction processing unit 307, a processing condition table 308, a scene reference image data generation unit 309, and a temporary storage memory 311.

  The device characteristic correction processing unit 307 determines the generation conditions of the scene reference image data by referring to the imaging device characteristic correction data input from the header information analysis unit 304 and the processing condition table 308. The processing condition table 308 is a table that stores processing conditions for generating scene reference image data for each characteristic of the imaging apparatus.

  The data file that is the processing result of the imaging device characteristic correction processing in the imaging device characteristic correction processing unit 321 is the same as that in FIG.

  The optimization processing unit 322 includes a scene determination processing unit 312a, a shooting information data processing unit 313, an appreciation image reference data generation unit 314, a temporary storage memory 315, and a setting input unit 316.

  The scene discrimination processing unit 312a discriminates a shooting scene at the time of shooting using the scene reference reduced image data input from the header information analysis unit 304 (see FIGS. 10 to 21), and generates a viewing image reference data. To the unit 314.

  The appreciation image reference data generation unit 314 reads the scene reference image data from the temporary storage memory 311, the generation conditions of the appreciation image reference data determined by the shooting information data processing unit 313, the determination result of the scene determination processing unit 312 a, Optimization for obtaining an optimal image at the output destination for the scene reference image data based on the information of the output destination (storage device 318, output device 317, display unit 208) designated from the setting input unit 316 Processing is performed to generate appreciation image reference data, which is output to the temporary storage memory 315 together with operation information of the setting input unit 316. The optimization processing includes, for example, gradation conversion processing (FIGS. 22 to 25) according to the determined shooting scene, compression to the output destination color gamut, gradation compression from 16 bits to 8 bits, and the number of output pixels. Reduction, processing for handling output characteristics (LUT) of output devices and display devices, and the like are included. Furthermore, image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment, and dodging processing is included.

  Of the components of the image processing apparatus 320, components other than the input unit 303 and the setting input unit 316 are included in the image processing unit 270 illustrated in FIG.

Next, the operation of the image recording apparatus 201 according to the second embodiment will be described.
With reference to FIG. 31, image processing executed in the image recording apparatus 201 will be described.

  When data is input to the input unit 303 (that is, when a recording medium is mounted on the mounting unit) (step S50), the data file recorded on the recording medium is read, and the header information analysis unit 304 The contents of the data file are analyzed (step S51), and the data file is divided into scene reference raw data, imaging device characteristic correction data, scene reference reduced image data, and shooting information data.

  The shooting information data processing unit 313 determines a generation condition for generating appreciation image reference data corresponding to the shooting condition based on the shooting information data obtained in step S51 (step S52). The apparatus characteristic correction processing unit 307 determines the generation conditions of the scene reference image data by referring to the imaging apparatus characteristic correction data obtained in step S51 and the processing condition table 308, and generates the scene reference raw data obtained in step S51. On the other hand, imaging device characteristic correction processing is performed according to the determined generation condition (step S53), and scene reference image data is generated (step S54).

  In the scene determination processing unit 312a, a shooting scene determination process (see FIGS. 10 to 21) for determining a shooting scene at the time of shooting is performed based on the scene reference reduced image data obtained in step S51 (step S55).

  Next, optimization processing is performed on the scene reference image data generated in step S54 based on the generation conditions of the viewing image reference data determined in step S52 and the determination result of the shooting scene determination processing in step S55. (Step S56), appreciation image reference data is generated (step S57).

  The appreciation image reference data is subjected to processing specific to the output destination set in the setting input unit 316 (operation unit 211) (step S58), and is output from the output destination device (step S58). S59), the main image processing ends.

  As described above, according to the image recording apparatus 201 of the second embodiment, the image recording apparatus 201 is optimized based on the imaging apparatus characteristic correction data and the imaging information data attached to the file header of the scene reference raw data output from the imaging apparatus. By generating appreciation image reference data, it is possible to obtain appreciation image reference data of higher quality than the image data obtained by the imaging apparatus without any information loss of the captured image information.

  Further, by determining the shooting scene at the time of shooting based on the scene reference reduced image data attached to the file header of the scene reference raw data output from the imaging device, the optimum gradation conversion processing according to the shooting scene is performed. Can be done. In particular, the accuracy of shooting scene discrimination can be improved by collating a shooting scene as a determination result of the shooting scene determination processing with a shooting scene included in shooting information data. As described above, the scene reference reduced image data is attached to the file header of the scene reference raw data output from the imaging device, so that it is necessary to generate standardized scene reference reduced image data in the image processing device 320. Thus, the processing efficiency of the image processing apparatus 320 can be improved.

  In the second embodiment, as shown in FIG. 28, the case where shooting information data indicating shooting condition setting at the time of shooting is attached to the scene reference raw data is shown. However, as shown in FIG. The data file may be created by attaching only the imaging device characteristic correction data and the scene reference reduced image data without attaching the shooting information data to the file header of the raw data. In this case, the data file that is the processing result of the imaging device characteristic correction processing in the imaging device characteristic correction processing unit 321 does not include the shooting information data as in FIG. 26B. In the image processing of FIG. The process of S52 is not performed. Even if there is no shooting information data, the shooting scene determination process based on the scene reference reduced image data can be used to determine the shooting scene with sufficient accuracy, and the optimum gradation conversion process according to the shooting scene is performed. It is possible.

  Further, the data attached to the scene reference raw data is not limited to reduced raw data or scene reference reduced image data, imaging device characteristic correction data, and shooting information data. For example, restoration information for restoring appreciation image reference data in the imaging device in order to stably reproduce the appreciation image reference data optimized for display in the imaging device by the image processing device or the image recording device (Appreciation image reference data restoration information) may be attached.

  FIG. 33A shows the tone conversion characteristics of scene reference image data. As shown in FIG. 33A, the gradation conversion characteristic is a characteristic that logY (Y: stimulus value of XYZ color system) is linearly converted with respect to logE (E: exposure amount).

  FIG. 33 (b) shows L * and logE (E: exposure) of the L * a * b * color system, which is a uniform color space, as gradation conversion characteristics in the sRGB color space of scene reference image data and appreciation image reference data. (Quantity) relationship. In the gradation conversion characteristic shown in FIG. 33 (b), this time the viewing image reference data has a characteristic that L * changes linearly with respect to logE.

  FIG. 34 shows the conversion characteristics (curve in the figure) of L * from the scene reference image data to the viewing image reference data. The conversion characteristic shown in FIG. 34 is the viewing image reference data restoration information.

1 is a block diagram showing a main part configuration of an imaging apparatus according to Embodiment 1 of the present invention. The figure which shows the data structure of the data file recorded on the recording medium of the storage device of FIG. The figure for demonstrating an imaging device characteristic correction process. 3 is a flowchart illustrating image data recording processing executed in the imaging apparatus according to the first embodiment. 1 is an external view of an image recording apparatus according to Embodiments 1 and 2 of the present invention. FIG. 6 is a block diagram showing an internal configuration of the image recording apparatus in FIG. 5. FIG. 2 is a block diagram illustrating an internal configuration of an image processing apparatus and an output unit in the image recording apparatus according to the first embodiment. 3 is a flowchart illustrating image processing executed in the image recording apparatus according to the first embodiment. The figure which shows the process result of an imaging device characteristic correction process. The flowchart which shows the imaging | photography scene discrimination | determination process performed in a scene discrimination | determination processing part. The flowchart which shows the 1st occupation rate calculation process which calculates the 1st occupation rate for every area | region of lightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient by which the 1st occupation rate for calculating the parameter | index 1 is multiplied. The figure which shows the curve showing the 2nd coefficient by which the 1st occupation rate for calculating the parameter | index 2 is multiplied. The flowchart which shows the 2nd occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient for multiplying the 2nd occupation rate for calculating the parameter | index 3 according to area | region (n1-n4). The flowchart which shows the parameter | index 4 calculation process performed in a scene discrimination | determination process part. The plot figure which shows the relationship between a photography scene (forward light, strobe, backlight, under) and the indicators 4-6. The figure which shows the gradation conversion curve corresponding to each gradation adjustment method. The figure which shows the frequency distribution (histogram) (a) of brightness | luminance, the normalized histogram (b), and the block-divided histogram (c). The figure ((a) and (b)) explaining deletion of the low-intensity area | region and the high-intensity area | region from the brightness | luminance histogram, and the figure ((c) and (d)) explaining the restriction | limiting of the frequency of a brightness | luminance. The figure which shows the gradation conversion curve showing the image processing conditions (gradation conversion conditions) in case a imaging | photography scene is backlight or under. The modification (a) of the data structure of the data file recorded on the recording medium of the storage device of the imaging device according to the first embodiment, and the imaging device characteristic correction processing on the reduced raw data and scene reference raw data of the data file The figure which shows the result of giving (b). The block diagram which shows the principal part structure of the imaging device which concerns on Embodiment 2 of this invention. The figure which shows the data structure of the data file recorded on the recording medium of the storage device of FIG. 9 is a flowchart illustrating image data recording processing executed in the imaging apparatus according to the second embodiment. FIG. 4 is a block diagram illustrating an internal configuration of an image processing apparatus and an output unit in an image recording apparatus according to a second embodiment. 9 is a flowchart illustrating image processing executed in the image recording apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a modification of the data structure of a data file recorded on a recording medium of a storage device of the imaging apparatus according to the second embodiment. The figure which shows the gradation conversion characteristic of scene reference image data, and the figure which shows the gradation conversion characteristic of scene reference image data and appreciation image reference data (b). The figure which shows the conversion characteristic from scene reference image data to appreciation image reference data.

Explanation of symbols

1 Lens 2 Aperture 3 CCD
4 Analog processing circuit 5 A / D converter 6 Temporary storage memory 7 Image processing unit 8 Header information processing unit 9 Storage device 10 CCD drive circuit 11 Control unit 12 Imaging information data generation unit 13 Device characteristic correction data generation unit 14 Operation unit 15 Display unit 16 Strobe drive circuit 17 Strobe 18 Focal length adjustment circuit 19 Automatic focus drive circuit 20 Motor 21 Reduced raw data generation unit 22 Scene reference reduced image data generation unit 100, 101 Imaging device 200, 201 Image recording device 202 Main body 204 Exposure processing Unit 205 print creation unit 207 control unit 208 display unit 209 film scanner unit 210 reflective original input device 211 operation unit 214 image reading unit 215 image writing unit 270 image processing unit 301, 320 image processing unit 302 output unit 303 input unit 304 header Information analysis units 305 and 32 Imaging device characteristic correction processing units 306 and 322 Optimization processing unit 307 Device characteristic correction processing unit 308 Processing condition table 309 Scene reference image data generation unit 310 Scene reference reduced image data generation unit 311 Temporary storage memories 312 and 312a Scene determination processing unit 313 Shooting information data processing unit 314 Appreciation image reference data generation unit 315 Temporary storage memory 316 Setting input unit 317 Output device 318 Storage device

Claims (14)

Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
Recording control means for recording on a recording medium by attaching the correction data and the reduced raw data to the scene reference raw data;
An imaging apparatus comprising: Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
A scene reference reduced image data generating means for generating standardized scene reference reduced image data by performing an imaging device characteristic correction process on the reduced raw data based on the correction data;
Recording control means for recording on the recording medium with the correction data and the scene reference reduced image data attached to the scene reference raw data;
An imaging apparatus comprising: Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
Recording control means for recording the correction data, the reduced raw data, and the shooting information data on the scene reference raw data and recording them on a recording medium;
An imaging apparatus comprising: Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Correction data generation means for generating correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the raw scene reference data;
Reduced raw data generating means for generating reduced raw data from the scene reference raw data;
A scene reference reduced image data generating means for generating standardized scene reference reduced image data by performing an imaging device characteristic correction process on the reduced raw data based on the correction data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
Recording control means for recording the correction data, the scene reference reduced image data and the shooting information data on the recording media by attaching the correction data, the scene reference reduced image data, and the scene reference raw data;
An imaging apparatus comprising: Generated from scene reference raw data depending on the characteristics of the imaging device, correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data, and the scene reference raw data Input means for inputting the reduced reduced data,
Standardized scene reference image data and scene reference reduced image data are generated by subjecting the input scene reference raw data and reduced raw data to a photographing device characteristic correction process based on the input correction data. Scene reference image data generating means for performing,
An image processing apparatus comprising:   On the basis of the generated scene reference reduced image data, the generated standardized scene reference image data is subjected to image processing that is optimized for the formation of an appreciation image on an output medium, and the appreciation image reference data The image processing apparatus according to claim 5, further comprising: an appreciation image reference data generation unit that generates Generated from scene reference raw data depending on the characteristics of the imaging device, correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data, and the scene reference raw data Input means for inputting the standardized scene reference reduced image data,
Scene reference image data generating means for generating standardized scene reference image data by performing a photographing apparatus characteristic correction process on the input scene reference raw data based on the input correction data;
An image processing apparatus comprising:   Based on the inputted scene reference reduced image data, the generated standardized scene reference image data is subjected to image processing that is optimized for formation of an appreciation image on an output medium, and the appreciation image reference data The image processing apparatus according to claim 7, further comprising: an appreciation image reference data generation unit that generates Generated from scene reference raw data depending on the characteristics of the imaging device, correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data, and the scene reference raw data Input means for inputting the reduced reduced raw data and photographing information data indicating photographing condition setting at the time of photographing;
A standardized scene reference image data and scene reference reduced image data are generated by subjecting the input scene reference raw data and reduced raw data to a photographing device characteristic correction process based on the input correction data. Scene reference image data generating means for performing,
An image processing apparatus comprising:   Based on the generated scene reference reduced image data and the input shooting information data, an image to be optimized for viewing image formation on an output medium with respect to the generated standardized scene reference image data The image processing apparatus according to claim 9, further comprising an appreciation image reference data generation unit that performs processing to generate appreciation image reference data. Generated from the scene reference raw data depending on the characteristics of the imaging device, correction data necessary for imaging device characteristic correction processing for generating standardized scene reference image data from the scene reference raw data, and the scene reference raw data Input means for inputting the standardized reduced scene reference image data and shooting information data indicating shooting condition settings at the time of shooting;
Scene reference image data generating means for generating standardized scene reference image data by performing a photographing apparatus characteristic correction process on the input scene reference raw data based on the input correction data;
An image processing apparatus comprising:   An image to be optimized for viewing image formation on an output medium with respect to the generated standardized scene reference image data based on the input scene reference reduced image data and the input shooting information data The image processing apparatus according to claim 11, further comprising an appreciation image reference data generation unit that performs processing to generate appreciation image reference data. Using the scene reference reduced image data, comprising a scene discrimination means for discriminating a shooting scene at the time of shooting;
The image according to any one of claims 6, 8, 10, and 12, wherein the appreciation image reference data generation unit generates the appreciation image reference data using a determination result by the scene determination unit. Processing equipment. An image processing apparatus according to any one of claims 6, 8, 10, 12, and 13,
Image forming means for forming an appreciation image on an output medium using appreciation image reference data generated by the appreciation image reference data generation means of the image processing apparatus;
An image recording apparatus comprising:

Images