JP2006203567A - Imaging apparatus, image processing apparatus, and image recording apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image reading apparatus capable of generating appreciation image reference data with the quality equal to and higher than the quality of image data obtained by an imaging apparatus and protecting information associated with production of the appreciation image reference data. <P>SOLUTION: The imaging apparatus 100 generates scene reference raw data depending on the characteristic of the imaging apparatus through imaging, a reproduction auxiliary data generating section 12 applies optimal image processing to the scene reference raw data for generating an appreciation image on an output medium to produce a reproduction auxiliary data in the case of generating the appreciation image reference data, a coding/authentication information setting section 21 sets authentication information (password) for authenticating use of the reproduction auxiliary data, and a header information processing section 8 attaches the reproduction auxiliary data and its authentication information to a file header (header area) of the scene reference raw data to produce an attached data file and records the attached data file to a recording medium of a storage device 9. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus such as a digital camera, and an image processing apparatus and an image recording apparatus that perform optimization processing on captured image data obtained by the imaging apparatus for forming an appreciation image on an output medium.

  In recent years, photographed image data obtained by an imaging device such as a digital camera is recorded on a recording medium such as a CD-R (Compact Disc Recordable), a floppy (registered trademark) disk, or a memory card, or via a communication network such as the Internet. It can be distributed on the Internet, displayed on a display device such as a CRT (Cathode Ray Tube), liquid crystal display, plasma display, or a small liquid crystal monitor on a mobile phone, or can be hardware using an output device such as a digital printer, an inkjet printer, or a thermal printer. There are a variety of output methods such as printing as a copy image.

When outputting captured image data for viewing purposes, generally, gradation adjustment, brightness adjustment, color adjustment is performed on the captured image data so that a desired image quality can be obtained on a display monitor or hard copy used for viewing. Various image processes represented by balance adjustment and sharpness enhancement are performed. Patent Document 1 discloses a correction process for generating standardized scene reference image data from scene reference raw data obtained from an imaging device (raw data that is not subjected to image processing and faithfully represents subject information). Auxiliary data necessary for the recording is attached to the scene reference raw data and recorded on a recording medium, and the image processing apparatus or the image recording apparatus performs image processing using the auxiliary data attached to the scene reference raw data. In addition, a technique that can create an optimized viewing image without any information loss of captured image information is disclosed.
JP 2004-96500 A

  However, in the above-described conventional image processing, information loss of captured image data is suppressed and the degree of freedom in image quality adjustment is improved. However, viewing image reference data optimized in the imaging apparatus is used as the image processing apparatus or image. There was a problem that the recording apparatus could not reproduce it stably. In particular, the history information of various sensors and arithmetic processes provided in the imaging device up to the actual photographing with the imaging device is useful information at the time of image quality adjustment, but the image processing device or the image recording device. Since this information cannot be obtained on the side, appreciation image reference data having higher quality than image data obtained by the imaging device cannot be generated.

  Further, in recent years, while an image format for converting scene reference raw data unique to an imaging device into general-purpose image data independent of the imaging device has become widespread, technical information such as internal processing of the imaging device and captured image data In order to protect the image, there are not a few requests by the manufacturer and user (purchaser) of the imaging apparatus to provide a protective measure such as limiting the use of the captured image data.

  An object of the present invention is to suppress information loss of captured image information, and on the image processing device or image recording device side, it is possible to generate high-quality appreciation image reference data that is higher than image data obtained by an imaging device, It is to protect information related to generation of appreciation image reference data.

In order to solve the above-described problem, the invention described in claim 1 includes scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Reproduction auxiliary data generating means for generating reproduction auxiliary data when generating image appreciation image reference data by performing image processing optimized for appreciation image formation on an output medium with respect to the scene reference raw data;
Authentication information setting means for setting authentication information for authenticating the use of the generated reproduction auxiliary data;
And recording control means for attaching the reproduction auxiliary data and the authentication information to the scene reference raw data and recording the data on a recording medium.

Invention of Claim 2 is provided with the encoding means which encodes the generated reproduction auxiliary data by a predetermined encoding method in the imaging apparatus of Claim 1,
The recording control means attaches the encoded reproduction auxiliary data and the authentication information to the scene reference raw data.

The invention according to claim 3 is a scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging,
Reproduction auxiliary data generating means for generating reproduction auxiliary data when generating image appreciation image reference data by performing image processing optimized for appreciation image formation on an output medium with respect to the scene reference raw data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
Authentication information setting means for setting authentication information for authenticating the use of the generated reproduction auxiliary data;
And recording control means for attaching the reproduction assistance data, the shooting information data, and the authentication information to the scene reference raw data and recording the data on a recording medium.

According to a fourth aspect of the present invention, in the imaging apparatus according to the third aspect of the present invention, the imaging apparatus includes an encoding unit that encodes the auxiliary reproduction data using a predetermined encoding method.
The recording control means attaches the encoded reproduction auxiliary data, the photographing information data, and the authentication information to the scene reference raw data.

  According to a fifth aspect of the present invention, in the imaging apparatus according to any one of the first to fourth aspects, the reproduction auxiliary data is for generating scene reference image data standardized from scene reference raw data. It is characterized in that correction information necessary for the imaging device characteristic correction processing is included.

  According to a sixth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, the reproduction auxiliary data is obtained by the imaging apparatus when generating appreciation image reference data on an output medium. The restoration information for restoring the appreciation image reference data is included.

  According to a seventh aspect of the present invention, in the imaging device according to any one of the first to sixth aspects, the reproduction assisting data is generated by the imaging device when generating appreciation image reference data on an output medium. Reproduction information for reproducing the generation process of the appreciation image reference data is included.

  According to an eighth aspect of the present invention, in the imaging device according to the seventh aspect, the reproduction information includes an index value for judging the validity of the photographing condition.

  According to a ninth aspect of the present invention, in the imaging device according to the eighth aspect, the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions during shooting. It is said.

  According to a tenth aspect of the present invention, in the imaging apparatus according to the ninth aspect, the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

According to the eleventh aspect of the present invention, the scene reference raw data depending on the characteristics of the imaging device, and the scene reference raw data are subjected to image processing that is optimized for forming a viewing image on the output medium. Input means for inputting reproduction auxiliary data for generating image reference data and authentication information for authenticating the use of the reproduction auxiliary data;
When the use of the reproduction auxiliary data is authenticated based on the input authentication information, the input scene reference raw data is subjected to an optimization process based on the input reproduction auxiliary data to be viewed image And an appreciation image reference data generating means for generating reference data.

According to a twelfth aspect of the present invention, in the image processing apparatus according to the eleventh aspect, the input unit inputs reproduction auxiliary data encoded by a predetermined encoding method,
When use of the reproduction auxiliary data is authenticated based on the input authentication information, the decoding auxiliary means includes decoding means for decoding the encoded reproduction auxiliary data,
The appreciation image reference data generating means generates the appreciation image reference data by performing optimization processing on the input scene reference raw data based on the decoded reproduction auxiliary data.

According to the thirteenth aspect of the present invention, the scene reference raw data depending on the characteristics of the imaging device, and the scene reference raw data are subjected to image processing that is optimized for forming a viewing image on the output medium. Input means for inputting auxiliary reproduction data when generating image reference data, photographing information data indicating photographing condition settings at the time of photographing, and authentication information for authenticating the use of the auxiliary reproduction data,
When the use of the reproduction assistance data is authorized based on the inputted authentication information, the optimization processing is performed on the inputted scene reference raw data based on the inputted reproduction assistance data and the photographing information data. And an appreciation image reference data generating means for generating appreciation image reference data.

According to a fourteenth aspect of the present invention, in the image processing device according to the thirteenth aspect, the input means inputs reproduction auxiliary data encoded by a predetermined encoding method,
When use of the reproduction auxiliary data is authenticated based on the input authentication information, the decoding auxiliary means includes decoding means for decoding the encoded reproduction auxiliary data,
The appreciation image reference data generating means generates an appreciation image reference data by performing an optimization process on the input scene reference raw data based on the decoded reproduction auxiliary data and the shooting information data. It is a feature.

  According to a fifteenth aspect of the present invention, in the image processing device according to any one of the eleventh to fourteenth aspects, in order to generate scene reference image data standardized from scene reference raw data as the reproduction assistance data. Correction information necessary for the imaging apparatus characteristic correction process is included.

  According to a sixteenth aspect of the present invention, there is provided the image processing apparatus according to any one of the eleventh to fifteenth aspects, wherein the reproduction auxiliary data includes an imaging device when generating appreciation image reference data on an output medium. The restoration information for restoring the viewing image reference data is included.

  The invention according to claim 17 is the image processing apparatus according to any one of claims 11 to 16, wherein the reproduction auxiliary data is an imaging device when generating appreciation image reference data on an output medium. Reproduction information for reproducing the generation process of the appreciation image reference data in the video is included.

  According to an eighteenth aspect of the present invention, in the image processing device according to the seventeenth aspect, the reproduction information includes an index value for judging validity of the photographing condition in the imaging device. .

  According to a nineteenth aspect of the present invention, in the image processing device according to the eighteenth aspect, the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions at the time of photographing by the imaging device. It is characterized by that.

  According to a twentieth aspect of the present invention, in the image processing apparatus according to the nineteenth aspect, the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

The invention according to claim 21 is the image processing apparatus according to any one of claims 11 to 20,
Image forming means for forming an appreciation image on an output medium using appreciation image reference data generated by the appreciation image reference data generation means of the image processing apparatus.

<Explanation of terms>
Hereinafter, terms used in this specification will be described.

  In the description of the present specification, “generation” means that a program and a processing circuit that operate in the imaging apparatus, the image processing apparatus, and the image recording apparatus according to the present invention newly create an image signal and data. “Create” may be used as a synonym.

  In the description of this specification, an “imaging device” is a device including an imaging element (image sensor) having a photoelectric conversion function, and includes a so-called digital camera and scanner. As an example of the image pickup device, a CCD type image pickup device provided with a color sensitivity by combining a charge coupled device (CCD), a charge transfer mechanism, and a checkered color filter, or a complementary metal-oxide display (CMOS). Semiconductor) type imaging device. The output currents of these image sensors are digitized by an A / D converter. The contents of each color channel at this stage are signal intensities based on the spectral sensitivity unique to the image sensor.

  The “scene reference raw data depending on the characteristics of the imaging device” is a direct raw output signal of the imaging device in which information faithful to the subject is recorded, and the data itself digitized by the A / D converter, This means data obtained by performing noise correction such as fixed pattern noise and dark current noise on the data. This scene reference raw data is generated by image processing that modifies the contents of the data to improve the effects during image viewing such as gradation conversion, sharpness enhancement, and saturation enhancement, and each color channel based on the spectral sensitivity inherent to the image sensor. The process of mapping the signal intensity of the signal to a standardized color space such as RIMM RGB or sRGB is omitted. The information amount (for example, the number of gradations) of the scene reference raw data is preferably equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data in accordance with the performance of the A / D converter. For example, when the number of gradations of the viewing image reference data (described later) is 8 bits per channel, the number of gradations of the scene reference raw data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

  “Output media” refers to, for example, display devices such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), plasma display, and hard copy image generation such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. Paper.

  “Appreciation image reference data” refers to digital images used for display devices such as CRTs, LCDs, and plasma displays, or for generating hard copy images on output media such as silver halide photographic paper, ink jet paper, and thermal printer paper. Means data. The appreciation image reference data is subjected to "optimization processing" so that an optimal image can be obtained on a display device such as a CRT, LCD, or plasma display, or on an output medium such as silver salt photographic paper, inkjet paper, or thermal printer paper. It is different from the scene reference raw data.

  Further, the “recording medium” is a storage medium used for storing “scene reference raw data” and “reproduction auxiliary data” output by the imaging device, and includes CompactFlash (registered trademark), Memory Stick (registered trademark), Smart media (registered trademark), multimedia card, hard disk, floppy (registered trademark) disk, magnetic storage medium (MO), CD-R, etc. may be used. In addition, the unit for writing to the recording medium is installed in an independent or remote place connected in a wireless state via a communication network such as the Internet, a writing unit connected by wire through a cord, even if it is integrated with the photographing apparatus. Any mode such as a unit may be used. The file format for recording on the recording medium is not a format specific to the imaging device, but standardized general-purpose such as TIFF (Tagged Image File Format), JPEG (Joint Photographic Coding Experts Group), and Exif (Exchangeable Image File Format) The file format is preferably recorded.

  The “shooting information data” is a record of shooting condition settings at the time of shooting, and may include the same tag information written in the header portion of the Exif file. Specifically, exposure time, shutter speed, aperture value (F number), ISO sensitivity, brightness value, subject distance range, light source, presence / absence of strobe emission, subject area, white balance, zoom magnification, shooting scene, strobe light source A tag (code) indicating the amount of reflected light, photographing saturation, type of subject, information on subject configuration, and the like.

  “Photographing information data” is a value obtained at the time of photographing by a sensor provided in the camera, data processed from the value of the sensor, or a value of the sensor for automating the exposure setting and focus function of the imaging device. Camera shooting conditions set based on the camera, but other than this, the shooting mode dial (for example, portrait, sports, macro shooting mode, etc.) and flash forced flash settings provided in the imaging device Information that the photographer manually sets the switch and the like is also included.

  In addition, “standardized scene reference image data” means that the signal intensity of each color channel based on at least the spectral sensitivity of the image sensor itself has been mapped to the standard color space such as RIM RGB, ERIMM RGB, and scRGB. It means image data in a state where image processing for modifying data contents is omitted in order to improve the effect at the time of image appreciation such as tone conversion, sharpness enhancement, and saturation enhancement. The scene reference image data is a correction of the photoelectric conversion characteristics of the imaging device (opto-electronic conversion function defined by ISO1452, for example, “Fine Imaging and Digital Photography” of Corona Co., Ltd., page 449 of the Japan Photographic Society Publishing Committee). It is preferable to have performed. The information amount (for example, the number of gradations) of the standardized scene reference image data conforms to the performance of the A / D converter and is equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data. Is preferred. For example, when the number of gradations of the viewing image reference data is 8 bits per channel, the number of gradations of the scene reference image data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

  In addition, “imaging device characteristic correction processing for generating standardized scene reference image data” refers to the above “scene reference raw data depending on the characteristics of the imaging device” as “standardized scene reference image data”. Means processing to convert. The content of this processing depends on the state of the “scene reference raw data depending on the characteristics of the imaging device”, but at least the signal intensity of each color channel based on the spectral sensitivity specific to the imaging device is standard such as RIMM RGB, ERIMM RGB, scRGB, etc. Includes mapping to color space. For example, when the “scene reference raw data depending on the characteristics of the imaging device” is not subjected to the interpolation processing based on the color filter array, the processing needs to be additionally performed. (Details of the interpolation processing based on the color filter array are described in, for example, “Fine Imaging and Digital Photography” of Corona, Inc., page 51 of the Japan Photographic Society Publishing Committee.) As a result, “scene reference raw data” The “standardized scene reference image data” in which the difference in signal values between different “imaging devices” is corrected is obtained.

  The “optimization process” is a process for obtaining an optimal image on a display device such as a CRT, LCD, or plasma display, or on an output medium such as silver salt photographic paper, inkjet paper, or thermal printer paper. For example, when it is assumed that the image is displayed on a CRT display monitor compliant with the sRGB standard, processing is performed so that optimum color reproduction is obtained within the color gamut of the sRGB standard. If output to silver salt photographic paper is assumed, processing is performed so that optimum color reproduction is obtained within the color gamut of silver salt photographic paper. In addition to color gamut compression, gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, and processing for handling output characteristics (LUT) of the output device are also included. Furthermore, it goes without saying that image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment or dodging is performed.

An example of optimizing appreciation image reference data using “photographing information data” is shown below.
Based on the “subject configuration” information, for example, it is possible to perform a saturation enhancement process partially, or to perform a dodging process in a scene with a wide dynamic range.
Based on the result of determining the shooting scene, for example, in night scene shooting, the degree of white balance adjustment can be relaxed and the color balance can be adjusted specially.
The distance between the photographer and the subject is estimated based on the “amount of reflected light from the strobe light source” information, and can be reflected in, for example, setting of image processing conditions that suppress whitening of the skin.
Based on the “subject type” information, for example, in human photography, the degree of sharpness is relaxed and the smoothing process is strengthened, so that wrinkles on the skin can be made inconspicuous.

  The image recording apparatus according to the present invention includes a color negative film, a color reversal film, a black and white negative film, a black and white film, in addition to a mechanism for performing image processing according to the present invention on digital image data acquired from the imaging apparatus according to the present invention. There may be provided a film scanner for inputting frame image information of a photographic photosensitive material recorded by an analog camera such as a reversal film, and a flatbed scanner for inputting image information reproduced on color paper which is silver salt photographic paper. Also obtained by a digital camera other than the imaging device of the present invention, CompactFlash (registered trademark), Memory Stick (registered trademark), SmartMedia (registered trademark), Multimedia card (registered trademark), floppy (registered trademark) disk, optical A means for reading digital image data stored in any known portable “recording medium” such as a magnetic storage medium (MO), CD-R, etc., or obtaining digital image data from a remote location via communication means, and CRT And display means such as LCD and plasma display, and processing means for forming an appreciation image on any known output medium such as hard copy image generation paper such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. May be.

  According to the present invention, reproduction auxiliary data for generating appreciation image reference data is generated on the imaging apparatus side, and the reproduction auxiliary data is attached to the scene reference raw data depending on the characteristics of the imaging apparatus on the recording medium. By recording, when data recorded on the recording medium is output to an output medium, viewing image reference data is generated using the reproduction auxiliary data, and information loss of captured image information is suppressed, and an imaging apparatus It is possible to obtain appreciation image reference data of higher quality than the image data obtained in (1).

  Also, by setting authentication information for authenticating the use of reproduction information data, it becomes possible to limit the use of image data obtained by the imaging apparatus, and protect information related to generation of appreciation image reference data. be able to.

  Furthermore, in addition to the reproduction assistance data, shooting information data, which is the shooting condition setting at the time of shooting, is attached to the scene reference raw data, so that the image processing apparatus or the image recording apparatus uses the reproduction assistance data and the shooting information data. Thus, appreciation image reference data is generated, and higher-quality appreciation image reference data can be obtained.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
First, the configuration in the present embodiment will be described.

<Configuration of imaging device>
FIG. 1 shows a main part configuration of an imaging apparatus 100 according to an embodiment of the present invention. As shown in FIG. 1, the imaging apparatus 100 includes a lens 1, an aperture 2, a CCD (Charge Coupled Device) 3, an analog processing circuit 4, an A / D converter 5, a temporary storage memory 6, an image processing unit 7, and header information. Processing unit 8, storage device 9, CCD drive circuit 10, control unit 11, reproduction auxiliary data generation unit 12, encoding / authentication information setting unit 21, operation unit 14, display unit 15, strobe drive circuit 16, strobe 17, focus A distance adjusting circuit 18, an automatic focus driving circuit 19, a motor 20 and the like are provided.

The optical system of the imaging apparatus 100 includes a lens 1, a diaphragm 2, and a CCD 3.
The lens 1 adjusts the focus and forms an optical image of the subject. The diaphragm 2 adjusts the amount of light beam that has passed through the lens 1. The CCD 3 photoelectrically converts subject light imaged on the light receiving surface by the lens 1 into an electrical signal (imaging signal) of an amount corresponding to the amount of incident light for each sensor in the CCD 3. Then, the CCD 3 is sequentially controlled by the timing pulse input from the CCD driving circuit 10 to sequentially output the image pickup signal to the analog processing circuit 4.

  The analog processing circuit 4 performs RGB signal amplification, noise reduction processing, and the like on the imaging signal input from the CCD 3. The processing in the analog processing circuit 4 is switched ON / OFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The A / D converter 5 converts the imaging signal input from the analog processing circuit 4 into a digital signal and outputs it. Hereinafter, a digital signal obtained by the A / D converter 5 will be described as scene reference raw data.

  The temporary storage memory 6 is a buffer memory or the like, and temporarily stores the image data output from the A / D converter 5.

  The image processing unit 7 performs tone correction, spectral sensitivity crosstalk correction, dark current noise suppression, sharpening, white balance adjustment for display on the display unit 15 with respect to the image data stored in the temporary storage memory 6. In addition to image quality improvement processing such as saturation adjustment, processing such as image size change, trimming, and aspect conversion is performed. The processing in the image processing unit 7 is switched ON / OFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

  The header information processing unit 8 encodes encoded reproduction auxiliary data (for details, input from the encoding / authentication information setting unit 21 to the file header (header area) of the scene reference raw data stored in the temporary storage memory 6. Attached data file (see FIG. 3) is created by attaching the authentication information described later.

  The storage device 9 is configured by a nonvolatile semiconductor memory or the like, and stores a control program for the imaging apparatus 100. In addition, the storage device 9 includes a mounting unit for mounting a recording medium such as a memory card, and in accordance with a control signal input from the control unit 11, reading of recorded data on the recording medium mounted on the mounting unit and recording Write data to the media.

  The CCD drive circuit 10 outputs a timing pulse in accordance with a control signal input from the control unit 11 and performs drive control of the CCD 3.

  The control unit 11 is configured by a CPU (Central Processing Unit) or the like, reads a control program of the imaging device 100 stored in the storage device 9, and controls the entire imaging device 100 according to the read control program.

  For example, the control unit 11 controls the motor 20 for adjusting the focal length and focus (focus) of the lens 1 in accordance with an operation signal from the operation unit 14, an automatic focus drive circuit 19, a focal length adjustment circuit 18, a CCD By controlling the driving circuit 10, the analog processing circuit 4, the temporary storage memory 6, the image processing unit 7, the display unit 15, and the strobe driving circuit 16, the photographing operation is executed. In addition, the control unit 11 instructs the reproduction auxiliary data generation unit 12 to generate reproduction auxiliary data (details will be described later) to be attached to the file header of the scene reference raw data obtained by shooting. An instruction is given to record on the recording medium a data file in which reproduction auxiliary data is attached to the reference raw data.

  The reproduction auxiliary data generation unit 12 generates reproduction auxiliary data necessary for generating the viewing image reference data by performing processing for optimizing the formation of the viewing image on the output medium, and the generated reproduction assistance data. Is output to the encoding / authentication information setting unit 21. The internal configuration of the reproduction auxiliary data generation unit 12 will be described in detail later with reference to FIG.

  The encoding / authentication information setting unit 21 encodes the reproduction auxiliary data generated by the reproduction auxiliary data generation unit 12 using a predetermined encoding method (compression method). In addition, authentication information (password) for authenticating the use of reproduction auxiliary data is set by an input operation from the operation unit 14, and the encoded reproduction auxiliary data and its authentication information are output to the header information processing unit 8. .

  Encoding schemes (compression schemes) applied in the present embodiment can be broadly divided into two types: lossy schemes and lossless schemes. The lossy method is a method that does not return to the original data at the time of decoding, and the lossless method is a method that completely returns to the original data at the time of decoding. There are LZH and ZIP as reversible compressed files. Hereinafter, the lossless method that is the basis of the compression algorithm will be described.

  A reversible compression algorithm is generally composed of two stages (pre-process and post-process) (the same applies to the non-reversible system). Here, the pre-process is referred to as “modeling” and the post-process is referred to as “encoding”. In the modeling of the previous process, data conversion for extracting data features is performed to reduce the data amount. As modeling, run length, forward movement, LZ (LZ77 system, LZ78 system), BlockSorting, PPM, and the like are known, and LZ is currently most widely used. In LZ, a table of character strings generally called a “dictionary” is created and data compression is performed. A dictionary is a table in which character strings are registered. For example, in a character string “ABURAKATABURA”, a character string “ABURA” appears twice. Thus, by registering frequently used character strings in the dictionary and converting the registered character strings into dictionary codes, the amount of information can be reduced.

  In encoding (encoding) in the subsequent process, the data converted in the previous process is converted to bit data. This bit conversion of information amount is also referred to as entropy coding. As entropy coding, Huffman coding, arithmetic coding, RangeCoder, and the like are known, and Huffman coding is currently the most widely used. Huffman coding is based on the idea that “bits are allocated in consideration of the appearance frequency of characters”. In Huffman coding, bits are assigned from characters with a low appearance frequency. Specifically, two low appearance frequencies are searched and 0 and 1 bits are assigned.

  The operation unit 14 includes a shutter button, a power ON / OFF button, various function buttons such as a zoom button, a cursor key, and the like, and outputs an operation signal corresponding to each button and key to the control unit 11.

  The display unit 15 is configured by an LCD (Liquid Crystal Display), an EL (ElectroLuminescence) display, or the like, and performs a required display process in accordance with a display control signal input by the control unit 11. For example, the display unit 15 displays information for the user of the image capturing apparatus 100 to confirm settings and conditions regarding shooting, or displays appreciation image reference data generated for display on the display unit 15. . Further, the display unit 15 has a function as a finder for continuously displaying images captured by the CCD 3 in the photographing mode.

  The strobe drive circuit 16 drives and controls the strobe 17 to emit light when the subject brightness is low according to a control signal input from the control unit 11.

  The strobe 17 boosts the battery voltage to a predetermined high voltage and stores it as a charge in a capacitor. The strobe 17 is driven by the strobe drive circuit 16 to emit light from the X tube with the electric charge stored in the capacitor and irradiate the subject with auxiliary light.

The focal length adjustment circuit 18 controls the motor 20 for adjusting the focal length by moving the lens 1 in accordance with a control signal input from the control unit 11.
The automatic focus driving circuit 19 controls the motor 20 for adjusting the focus (focus) by moving the lens 1 in accordance with a control signal input from the control unit 11.

  FIG. 2 shows an internal configuration of the reproduction auxiliary data generation unit 12. As shown in FIG. 2, the auxiliary reproduction data generation unit 12 includes an imaging device characteristic correction information generation unit 121, an appreciation image reference data restoration information generation unit 122, and a processing process reproduction information generation unit 123.

  The imaging device characteristic correction information generation unit 121 generates information (imaging device characteristic correction information) necessary for imaging device characteristic correction processing for generating standardized scene reference image data from scene reference raw data. As shown in FIG. 4, the imaging device characteristic correction processing includes filter interpolation calculation (a), matrix calculation (b), photoelectric conversion characteristics and gain correction (c) for scene reference raw data.

  The filter interpolation calculation shown in FIG. 4A is a process of interpolating image data having a filter arrangement of one pixel and one color into linear image data of one pixel and three colors (RGB). Here, the filter array is an array pattern of color filters for color discrimination of CCD, and an RGB primary color Bayer array is generally used. As a filter interpolation method, a nearest neighbor method, a bi-linear interpolation method, a bi-cubic convolution method, or the like can be applied.

  In the nearest neighbor method, a pixel nearest to the target pixel is selected, the pixel is enlarged or reduced as it is according to the size of enlargement / reduction, and the pixel is interpolated only at a necessary portion. In the bilinear method, linear density interpolation is performed from the density values of four pixels around the target pixel according to the coordinates (real values). In the bicubic method, in order to perform interpolation with higher accuracy than in the bilinear method, interpolation is performed using a cubic function from the density values of 16 pixels around the pixel of interest. The equation used for interpolation is sin (πx) / πx, which is the most complete density interpolation equation in theory (sampling theorem). This is approximated by a third-order term of x by Taylor expansion and used as an interpolation formula.

  The matrix calculation shown in FIG. 4B is a calculation process for correcting a difference in which the same subject color is recorded as a different signal due to a difference in spectral characteristics of the color filter and spectral sensitivity characteristics of the image sensor. Conversion from RGB to tristimulus values in the XYZ color system is performed by matrix calculation.

  FIG. 4C shows processing for correcting a difference due to photoelectric conversion characteristics (linear conversion characteristics) and gain correction (parallel movement) of the image sensor. This processing is shown in FIG. As described above, logY (Y: stimulus value of XYZ color system) is linearly converted with respect to logE (E: exposure amount).

  By the imaging device characteristic correction processing as shown in FIGS. 4A to 4C, the scene reference raw data depending on the characteristics of the imaging device is converted into XYZ values in a standardized color space such as scene reference image data. The Note that the color filter array pattern, matrix calculation matrix coefficients a to i, coefficient values for correcting differences associated with photoelectric conversion characteristics and gain correction, which are applied in the imaging apparatus characteristic correction processing, are all different for each imaging apparatus. is there.

  The appreciation image reference data restoration information generation unit 122 generates restoration information (appreciation image reference data restoration information) for restoring appreciation image reference data in the imaging apparatus 100 when generating appreciation image reference data from scene reference image data. Generate encoding information.

  FIG. 5A shows the tone conversion characteristics of the scene reference image data. As shown in FIG. 5A, the gradation conversion characteristic is a characteristic that logY (Y: stimulus value of XYZ color system) is linearly converted with respect to logE (E: exposure amount).

  FIG. 5B illustrates the L * and logE (E: exposure) of the L * a * b * color system, which is a uniform color space, as gradation conversion characteristics in the sRGB color space of the scene reference image data and the viewing image reference data. (Quantity) relationship. In the gradation conversion characteristic shown in FIG. 5B, this time, the viewing image reference data has a characteristic that L * changes linearly with respect to logE.

  FIG. 6A shows the conversion characteristics (curve in the figure) of scene reference image data to appreciation image reference data. The conversion characteristics shown in FIG. 6A can be corrected according to the shooting scene at the time of shooting (front light, backlight, under, flash proximity, etc.). FIG. 6B shows the gradation conversion characteristics of the appreciation image reference data for correcting the light source conditions for backlighting, flash close-up photography, and under / over exposure conditions. A combination of the conversion characteristics shown in FIG. 6A and the conversion characteristics shown in FIG. 6B is the viewing image reference data restoration information.

  The processing process reproduction information generating unit 123 generates information (processing process reproduction information) for reproducing the generation process of the appreciation image reference data in the imaging apparatus 100 when generating the appreciation image reference data from the scene reference image data. .

  FIG. 7 shows an example of processing process reproduction information. The process process reproduction information indicates history information of shooting conditions from a main shooting to a predetermined time before (for example, 2500 ms). Specifically, as shown in FIG. Indicators 1 to 3 indicating the processing result of the discrimination processing of the shooting scenes (forward light, backlight, under, flash proximity, etc.), total indicator 4 calculated from the indicators 1 to 3, and evaluation value 1 for the calculated indicator The history information about each item of the camera shake level evaluation value 2 and the photographing condition validity is shown.

  In the processing process reproduction information, the camera shake level indicates the degree of camera shake in, for example, 10 levels (an integer of 1 to 10). The higher the degree of camera shake, the higher the value of the camera shake level.

  The indices 1 to 3 are numerical values for specifying a shooting scene at the time of shooting (forward light, backlight, under, flash proximity, etc.). FIG. 8 shows a discrimination map for discriminating the shooting scene. In this embodiment, as shown in FIG. 8, the index 1 is the strobe degree (strobe degree), the index 2 is the backlight degree (backlight degree), and the index 3 is the under degree (under degree). As shown in FIG. 8A, back light and forward light are discriminated from the area determined by the index 1 (strobe degree) and the index 2 (back light intensity), and the index 1 (strobe degree) and the index 3 (under degree). Under and flash proximity are discriminated from the area determined by. The shooting scene determination process will be described in detail later with reference to FIGS.

The definition of the total index 4 is shown in Formula (1).
Total index 4 = 10 − [{(index 1 + index 2 + index 3 + 18) / 36} × 10] (1)
Evaluation value 1 and evaluation value 2 indicate a light source condition / exposure condition evaluation value and a camera shake evaluation value, respectively, and are defined as in equations (2) and (3).
Evaluation value 1 = Total index 4 x 3.4-5 (2)
Evaluation value 2 = -0.54 x camera shake level +8.5 (3)
Here, in the formulas (2) and (3), among the indexes 1 to 3, a value of −6 or less is −6, and a value of +6 or more is +6.

The validity of the shooting condition is a numerical value indicating the degree of shooting that has been performed correctly, such as the environment at the time of shooting and the technical level of the photographer, and is defined as shown in Expression (4) using the evaluation value 1 and the evaluation value 2. The
Shooting condition correctness = evaluation value 1 × 0.8 + evaluation value 2 × 0.2 (4)
The correctness of the shooting conditions increases as the shooting is performed correctly. As shown in the processing process reproduction information of FIG. 7, the photographing conditions (light source conditions, exposure conditions, etc.) are adjusted so that the numerical value of the photographing condition validity increases as time passes.

  8A and 8B, the histories of the indices 1 to 3 indicated by the processing process reproduction information shown in FIG. 7 are indicated by diamonds. The history information on the discrimination map shown in FIG. 8 may be displayed on the display unit 15.

  In the above description, the camera shake level is an integer value of 1 to 10, and the numerical range of the indices 1 to 3 is −6 or more and +6 or less, but these numerical ranges are not particularly limited. With the change of the numerical value range, the definitions of the total index 4, the evaluation value 1, and the evaluation value 2 are changed. Further, various user mode settings may be used as user characteristics other than camera shake. The history information recorded in the processing process reproduction information may include a shutter speed, an aperture value, and the like.

  The recording of history information as processing process reproduction information starts after the shooting mode is designated by the operation unit 14, and the time when the shutter button is pressed may be the time of actual shooting or the shutter button of the operation unit 14. Recording of history information may be started after the button is half-pressed, and the time when the shutter button is pressed hard may be set as the actual shooting.

  FIG. 3 shows the data structure of the data file recorded on the recording medium of the storage device 9. Reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration information and processing process reproduction information) encoded by the encoding / authentication information setting unit 21 and its authentication information include header information as shown in FIG. The processing unit 8 creates an attached data file attached to the file header of the scene reference raw data, and the attached data file is recorded on the recording medium of the storage device 9. Note that decoding information for decoding the encoded auxiliary reproduction data may also be attached to the file header of the scene reference raw data.

  Note that the appreciation image reference data generated in the imaging apparatus 100 is not only used for display on the display unit 15 but also recorded on a recording medium in association with the scene reference image data and the appreciation image reference data. The thumbnail image of the viewing image reference data may be attached as metadata of the scene reference raw data and recorded on the recording medium.

Next, the operation in the imaging apparatus 100 will be described.
With reference to the flowchart of FIG. 9, an image data recording process executed in the imaging apparatus 100 will be described.

  When the shooting mode is designated by the operation unit 14, preliminary imaging is performed (step S <b> 1), and an appreciation image reference for displaying on the display unit 15 an image obtained by preliminary imaging (hereinafter referred to as a preliminary captured image) is displayed. Data is formed (step S2), and the formed appreciation image reference data is displayed on the display unit 15 (step S3). In step S1, preliminary imaging may be performed when the shutter button is pressed halfway by the operation unit 14.

  Next, in the processing process reproduction information generation unit 123, a shooting scene discrimination process (see FIGS. 10 to 20) is performed, and the validity of the shooting conditions is determined from the camera shake level and the indices 1 to 3 as the processing result of the shooting scene discrimination process. Is calculated, and the validity of the imaging condition is determined from the calculation result (step S4). Then, the shooting condition is adjusted so that the value of the shooting condition validity is increased (step S5). The processing in steps S1 to S5 is repeated until the actual shooting is instructed by pressing the shutter button of the operation unit 14, and history information of shooting conditions up to a predetermined time before the main shooting is generated as processing process reproduction information. .

  When the shutter button of the operation unit 14 is pressed and main shooting is instructed (step S6; YES), the image reference signal obtained from the CCD 3 is converted into a digital signal by the A / D converter 5, thereby generating a scene reference source. At the same time that data is generated (step S7), the reproduction auxiliary data generation unit 12 generates imaging device characteristic correction information, appreciation image reference data restoration information, and processing process reproduction information as reproduction auxiliary data (step S8). . The process process reproduction information generated in step S8 is obtained by adding information at the time of main shooting to the history information before main shooting generated by the processing of steps S1 to S5. Note that a step of creating appreciation image reference data for recording on a recording medium may be added after the main shooting.

  When the reproduction auxiliary data is generated, the reproduction auxiliary data is encoded by a predetermined encoding method (step S9), and a password input screen for authenticating the use of the reproduction auxiliary data is displayed on the display unit 15. The When a password is input on the password input screen by operating the operation unit 14, the input password is set as authentication information of the reproduction assistance data generated in step S8 (step S10).

  Next, in the header information processing unit 8, the reproduction auxiliary data encoded in step S9 and the authentication information (password) set in step S10 are added to the file header of the scene reference raw data generated in step S7 as tag information. The attached data file (see FIG. 3) is created (step S12). The attached data file is recorded and stored in the recording medium of the storage device 9 (step S13), and the image data recording process is completed.

<Shooting scene discrimination processing>
Next, with reference to the flowchart of FIG. 10, a shooting scene determination process necessary for determining the validity of shooting conditions will be described.

  First, photographic image data (for example, scene reference raw data) is divided into predetermined image areas, and an occupancy ratio (first occupancy ratio, second occupancy ratio) indicating a ratio of each divided area to the entire captured image data. Occupancy rate calculation processing is performed (step T1). Details of the occupation rate calculation process will be described later with reference to FIGS. 11 and 17.

  Next, a bias amount calculation process is performed for calculating a bias amount indicating a bias in the gradation distribution of the photographed image data (step T2). The bias amount calculation process in step T2 will be described in detail later with reference to FIG.

  Next, indices 1 to 3 for specifying a shooting scene are calculated based on the occupation ratio calculated in step T1 and a coefficient set in advance according to shooting conditions (step T3), and the shooting scene determination process is performed. finish. The index calculation method in step T3 will be described in detail later.

  Next, the first occupation ratio calculation process will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step T10). FIG. 12 shows an example of a conversion program (HSV conversion program) that obtains a hue value, a saturation value, and a lightness value by converting from RGB to the HSV color system using program code (c language). In the HSV conversion program shown in FIG. 12, the values of digital image data as input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255.

  Next, the photographed image data is divided into regions composed of combinations of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step T11). Hereinafter, the area division of the captured image data will be described in detail.

The lightness value (V) is 0-25 (v1), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 (v6) , 225 to 255 (v7). Hue (H) is a skin hue hue area (H1 and H2) with a hue value of 0 to 39, 330 to 359, a green hue area (H3) with a hue value of 40 to 160, and a blue hue area with a hue value of 161 to 250 It is divided into four areas (H4) and a red hue area (H5). Note that the red hue region (H5) is not used in the following calculation because it is known that the contribution to the determination of the imaging condition is small. The flesh-color hue area is further divided into a flesh-color area (H1) and other areas (H2). Hereinafter, among the flesh-colored hue areas (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following formula (5) is defined as the flesh-colored area (H1), and the area that does not satisfy the formula (5) ( H2).
10 <Saturation (S) <175,
Hue '(H) = Hue (H) + 60 (when 0 ≤ Hue (H) <300),
Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),
Luminance (Y) = InR x 0.30 + InG x 0.59 + InB x 0.11 (A)
As
Hue '(H) / Luminance (Y) <3.0 × (Saturation (S) / 255) +0.7 (5)
Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In addition, it is also possible to use the brightness (V) in the formula (5).

When the two-dimensional histogram is created, a first occupancy ratio indicating a ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step T12), and the first The occupation rate calculation process ends. Assuming that the first occupancy ratio calculated in the divided area composed of the combination of the lightness area vi and the hue area Hj is Rij, the first occupancy ratio in each divided area is expressed as shown in Table 1.

Next, an index calculation method will be described.
Table 2 shows the first coefficient necessary for calculating the accuracy α for strobe shooting, that is, the index α that quantitatively indicates the lightness state of the face area at the time of strobe shooting, for each divided area. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy ratio Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 13 shows a lightness (v) -hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (r1) distributed in the skin color hue region of high brightness in FIG. 13, and blue other than that is blue. A negative (−) coefficient is used for the first occupancy calculated from the hue region (r2). FIG. 15 shows a curve (coefficient curve) in which the first coefficient in the skin color area (H1) and the first coefficient in the other areas (green hue area (H3)) continuously change over the entire brightness. It is shown. According to Table 2 and FIG. 15, in the region of high brightness (V = 170 to 224), the sign of the first coefficient in the skin color region (H1) is positive (+), and other regions (for example, the green hue region) The sign of the first coefficient in (H3)) is negative (-), and it can be seen that the signs of the two are different.

従って、H1〜H4領域の和は、下記の式(6-1)〜式(6-4)のように表される。
H1領域の和＝R11×(-44.0)＋R21×(-16.0)＋（中略）...＋R71×(-11.3) (6-1)
H2領域の和＝R12×0.0＋R22×8.6＋（中略）... ＋R72×(-11.1) (6-2)
H3領域の和＝R13×0.0＋R23×(-6.3)＋（中略）...＋R73×(-10.0) (6-3)
H4領域の和＝R14×0.0＋R24×(-1.8)＋（中略）...＋R74×(-14.6) (6-4) When the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index α is defined as in Expression (6).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (6-1) to (6-4).
H1 area sum = R11 x (-44.0) + R21 x (-16.0) + (omitted) ... + R71 x (-11.3) (6-1)
Sum of H2 area = R12 x 0.0 + R22 x 8.6 + (omitted) ... + R72 x (-11.1) (6-2)
Sum of H3 area = R13 x 0.0 + R23 x (-6.3) + (omitted) ... + R73 x (-10.0) (6-3)
Sum of H4 area = R14 x 0.0 + R24 x (-1.8) + (omitted) ... + R74 x (-14.6) (6-4)

The index α is defined as in Expression (7) using the sum of the H1 to H4 regions shown in Expressions (6-1) to (6-4).
Index α = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 4.424 (7)

Table 3 shows, for each divided region, the second coefficient necessary for calculating the index β that quantitatively indicates the accuracy as backlight photographing, that is, the brightness state of the face region at the time of backlight photographing. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy rate Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

  FIG. 14 shows a lightness (v) -hue (H) plane. According to Table 3, a negative (-) coefficient is used for the occupation ratio calculated from the region (r4) distributed in the intermediate lightness of the flesh color hue region in FIG. 14, and the low lightness (shadow) region ( A positive (+) coefficient is used for the occupation ratio calculated from r3). FIG. 16 shows the second coefficient in the skin color area (H1) as a curve (coefficient curve) that continuously changes over the entire brightness. According to Table 3 and FIG. 16, the sign of the second coefficient of the intermediate lightness region of the flesh color hue region having the lightness value of 85 to 169 (v4) is negative (−), and the lightness value is 26 to 84 (v2, It can be seen that the sign of the second coefficient in the low brightness (shadow) region of v3) is positive (+), and the sign of the coefficient in both regions is different.

従って、H1〜H4領域の和は、下記の式(8-1)〜式(8-4)のように表される。
H1領域の和＝R11×(-27.0)＋R21×4.5＋（中略）...＋R71×(-24.0) (8-1)
H2領域の和＝R12×0.0＋R22×4.7＋（中略）... ＋R72×(-8.5) (8-2)
H3領域の和＝R13×0.0＋R23×0.0＋（中略）...＋R73×0.0 (8-3)
H4領域の和＝R14×0.0＋R24×(-5.1)＋（中略）...＋R74×7.2 (8-4) When the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index β is defined as in Expression (8).

Therefore, the sum of the H1 to H4 regions is expressed by the following formulas (8-1) to (8-4).
H1 area sum = R11 x (-27.0) + R21 x 4.5 + (omitted) ... + R71 x (-24.0) (8-1)
Sum of H2 area = R12 x 0.0 + R22 x 4.7 + (omitted) ... + R72 x (-8.5) (8-2)
Sum of H3 area = R13 x 0.0 + R23 x 0.0 + (omitted) ... + R73 x 0.0 (8-3)
H4 area sum = R14 x 0.0 + R24 x (-5.1) + (omitted) ... + R74 x 7.2 (8-4)

The index β is defined as in Expression (9) using the sum of the H1 to H4 regions shown in Expressions (8-1) to (8-4).
Index β = sum of H1 region + sum of H2 region + sum of H3 region + sum of H4 region + 1.554 (9)
Since the index α and the index β are calculated based on the brightness of the captured image data and the distribution amount of the hue, the index α and the index β are effective for determining a captured scene when the captured image data is a color image.

  Next, the second occupancy rate calculation process executed to calculate the index γ will be described in detail with reference to the flowchart of FIG.

  First, the RGB values of the photographed image data are converted into the HSV color system (step T20). Next, the photographed image data is divided into regions composed of combinations of the distance from the outer edge of the photographed image screen and the brightness, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step T21). Hereinafter, the area division of the captured image data will be described in detail.

  18A to 18D show four regions n1 to n4 that are divided according to the distance from the outer edge of the screen of the captured image data. A region n1 shown in FIG. 18A is an outer frame, a region n2 shown in FIG. 18B is a region inside the outer frame, and a region n3 shown in FIG. An inner area, an area n4 shown in FIG. 18D, is an area at the center of the captured image screen. Further, the lightness is divided into seven regions v1 to v7 as described above. Therefore, when the captured image data is divided into regions composed of combinations of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

When the two-dimensional histogram is created, a second occupancy ratio indicating a ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire photographed image) is calculated (step T22). The occupation rate calculation process ends. If the second occupancy calculated in the divided area composed of the combination of the brightness area vi and the screen area nj is Qij, the second occupancy in each divided area is expressed as shown in Table 4.

図１９は、画面領域ｎ１〜ｎ４における第３の係数を、明度全体に渡って連続的に変化する曲線（係数曲線）として示したものである。 Next, a method for calculating the index γ calculated using the second occupation ratio will be described.
Table 5 shows the third coefficient necessary for calculating the index γ for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy rate Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the shooting conditions.

FIG. 19 shows the third coefficient in the screen areas n1 to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

従って、ｎ１〜ｎ４領域の和は、下記の式(10-1)〜式(10-4)のように表される。
n1領域の和＝Q11×40.1＋Q21×37.0＋（中略）...＋Q71×22.0 (10-1)
n2領域の和＝Q12×(-14.8)＋Q22×(-10.5)＋（中略）...＋Q72×0.0 (10-2)
n3領域の和＝Q13×24.6＋Q23×12.1＋（中略）...＋Q73×10.1 (10-3)
n4領域の和＝Q14×1.5＋Q24×(-32.9)＋（中略）...＋Q74×(-52.2) (10-4) When the third coefficient in the lightness area vi and the screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index γ is defined as in Expression (10).

Accordingly, the sum of the n1 to n4 regions is expressed by the following formulas (10-1) to (10-4).
n1 area sum = Q11 x 40.1 + Q21 x 37.0 + (omitted) ... + Q71 x 22.0 (10-1)
n2 area sum = Q12 x (-14.8) + Q22 x (-10.5) + (omitted) ... + Q72 x 0.0 (10-2)
n3 area sum = Q13 x 24.6 + Q23 x 12.1 + (omitted) ... + Q73 x 10.1 (10-3)
n4 area sum = Q14 x 1.5 + Q24 x (-32.9) + (omitted) ... + Q74 x (-52.2) (10-4)

The index γ is defined as in Expression (11) using the sum of the n1 to n4 regions shown in Expressions (10-1) to (10-4).
Index γ = sum of n1 regions + sum of n2 regions + sum of n3 regions + sum of n4 regions−12.6201 (11)
Since the index γ is calculated based on a compositional feature (distance from the outer edge of the screen of the captured image data) based on the brightness distribution position of the captured image data, the imaging condition of the monochrome image as well as the color image is determined. It is also effective.

  Next, the bias amount calculation processing (step T2 in FIG. 10) will be described with reference to the flowchart in FIG.

式（１２）において、画素輝度値とは、撮影画像データの各画素の輝度であり、平均輝度値とは、撮影画像データの輝度の平均値である。また、全体画素数とは、撮影画像データ全体の画素数である。 First, the luminance Y (brightness) of each pixel is calculated from the RGB (Red, Green, Blue) values of the photographed image data using Equation (A), and the luminance standard deviation (x1) is calculated (step 1). T23). The standard deviation (x1) of brightness is defined as shown in Equation (12).

In Expression (12), the pixel luminance value is the luminance of each pixel of the captured image data, and the average luminance value is the average value of the luminance of the captured image data. The total number of pixels is the number of pixels of the entire captured image data.

Next, as shown in Expression (13), a luminance difference value (x2) is calculated (step T24).
Difference in luminance value (x2) = (maximum luminance value−average luminance value) / 255 (13)
In equation (13), the maximum luminance value is the maximum luminance value of the captured image data.

  Next, the average luminance value (x3) of the flesh color region in the center of the screen of the captured image data is calculated (step T25), and further, the average luminance value (x4) in the center of the screen is calculated (step T26). Here, the center of the screen is, for example, an area configured by an area n3 and an area n4 in FIG.

Next, the flesh color luminance distribution value (x5) is calculated (step T27), and this deviation amount calculation processing is completed. When the maximum luminance value of the skin color area of the photographed image data is Yskin_max, the minimum luminance value of the skin color area is Yskin_min, and the average luminance value of the skin color area is Yskin_ave, the skin color luminance distribution value (x5) is defined as shown in Expression (14). Is done.
x5 = (Yskin_max−Yskin_min) / 2−Yskin_ave (14)

Let x6 be the average luminance value of the skin color area in the center of the screen of the captured image data. Here, the center of the screen is, for example, an area composed of the area n2, the area n3, and the area n4 in FIG. At this time, the index 1 shown in FIG. 7 and FIG. 8 is defined as the formula (15) using the index α, the index γ, and x6, and the index 2 is defined using the formula (15) using the index β, the index γ, and x6. 16).
Index 1 = 0.46 × Index α + 0.61 × Index γ + 0.01 × x6−0.79 (15)
Indicator 2 = 0.58 × Indicator β + 0.18 × Indicator γ + (-0.03) × x6 + 3.34 (16)
Here, the weighting coefficient by which each index is multiplied in Expression (15) and Expression (16) is set in advance according to the shooting conditions.

指標３は、式（１７）のように定義される。
指標３＝x1×0.02＋x2×1.13＋x3×0.06＋x4×(-0.01)＋x5×0.03−6.49 （１７）
この指標３は、撮影画像データの画面の構図的な特徴だけでなく、輝度ヒストグラム分布情報を持ち合わせており、特に、ストロボ（フラッシュ近接）撮影シーンとアンダー撮影シーンの判別に有効である。 The index 3 shown in FIGS. 7 and 8 is obtained by multiplying the deviation amounts (x1) to (x5) calculated in the deviation amount calculation process by a fourth coefficient set in advance according to the shooting conditions. . Table 6 shows a fourth coefficient which is a weighting coefficient by which each deviation amount is multiplied.

The index 3 is defined as in Expression (17).
Index 3 = x1 x 0.02 + x2 x 1.13 + x3 x 0.06 + x4 x (-0.01) + x5 x 0.03-6.49 (17)
This index 3 has not only the compositional characteristics of the screen of the photographed image data but also the luminance histogram distribution information, and is particularly effective for discriminating between the strobe (flash proximity) photographing scene and the under photographing scene.

  As described above, according to the imaging apparatus 100 of the present embodiment, reproduction auxiliary data for generating appreciation image reference data is generated on the imaging apparatus side, and the reproduction auxiliary data depends on the characteristics of the imaging apparatus. By attaching to the reference raw data and recording on the recording medium, when the data recorded on the recording medium is output on the output medium, the viewing image reference data is generated using the reproduction auxiliary data, and the captured image is generated. In addition to suppressing information loss of information, it is possible to obtain appreciation image reference data with higher quality than image data obtained by the imaging device.

  Also, by setting authentication information for authenticating the use of reproduction information data, it becomes possible to limit the use of image data obtained by the imaging apparatus 100, and protect information related to generation of appreciation image reference data. can do.

<Modification of imaging device>
Next, in order to obtain a more preferable image at the output destination of the image data recorded on the recording medium of the imaging apparatus, the imaging apparatus configured by adding the imaging information data generation unit 13 to the imaging apparatus 100 of FIG. 101 will be described. FIG. 21 shows a configuration of an imaging apparatus 101 as a modification of the imaging apparatus 100. In the imaging apparatus 101 of FIG. 21, the same components as those of the imaging apparatus 100 of FIG.

  The shooting information data generation unit 13 generates shooting information data that is a shooting condition setting at the time of shooting. This shooting information data includes, for example, information directly related to the camera type (model) such as camera name and code number, exposure time, shutter speed, aperture value (F number), ISO sensitivity, luminance value, subject distance range, Information on the light source, the presence / absence of strobe light emission, the subject area, white balance, zoom magnification, subject composition, shooting scene, the amount of reflected light from the strobe light source, shooting conditions such as shooting saturation, and information on the type of subject.

  FIG. 22 shows the data structure of a data file recorded on the recording medium of the storage device 9 of the imaging apparatus 101. Reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration information and processing process reproduction information) encoded by the encoding / authentication information setting unit 21 and authentication information for authenticating the use of the reproduction auxiliary data Then, the shooting information data generated by the shooting information data generation unit 13 is attached to the file header of the scene reference raw data by the header information processing unit 8 to create an attached data file, and this attached data file Is recorded on the recording medium of the storage device 9. Note that decoding information for decoding the encoded auxiliary reproduction data may also be attached to the file header of the scene reference raw data.

Next, the operation in the imaging apparatus 101 will be described.
With reference to the flowchart of FIG. 23, the image data recording process performed in the imaging device 101 will be described.

  When the photographing mode is designated by the operation unit 14, preliminary imaging is performed (step S20), and an image to be viewed for displaying on the display unit 15 an image obtained by preliminary imaging (hereinafter referred to as a preliminary captured image) is referred to. Data is formed (step S21), and the formed appreciation image reference data is displayed on the display unit 15 (step S22). In step S20, preliminary imaging may be performed when the shutter button is pressed halfway by the operation unit 14.

  Next, the processing process reproduction information generation unit 123 calculates the value of the shooting condition validity from the camera shake level, the indices 1 to 3 as the scene determination processing result, and the validity of the shooting condition is determined from the calculation result ( Step S23). Then, the shooting condition is adjusted so that the value of the shooting condition validity is increased (step S24). The processing in steps S20 to S24 is repeated until the actual shooting is instructed by pressing the shutter button of the operation unit 14, and history information of shooting conditions up to a predetermined time before the main shooting is generated as processing process reproduction information. .

  When the shutter button of the operation unit 14 is pressed and main shooting is instructed (step S25; YES), the image reference signal obtained from the CCD 3 is converted into a digital signal by the A / D converter 5, thereby generating a scene reference source. At the same time that the data is generated (step S26), the reproduction auxiliary data generation unit 12 generates imaging device characteristic correction information, appreciation image reference data restoration information, and processing process reproduction information as reproduction auxiliary data (step S28). The shooting information data generation unit 13 generates shooting information data (step S27). The process process reproduction information generated in step S28 is obtained by adding information at the time of main photographing to the history information before main photographing generated by the processing of steps S20 to S24. Note that a step of creating appreciation image reference data for recording on a recording medium may be added after the main shooting.

  When the reproduction auxiliary data is generated, the reproduction auxiliary data is encoded by a predetermined encoding method (step S29), and a password input screen for authenticating the use of the reproduction auxiliary data is displayed on the display unit 15. The When a password is input on the password input screen by the operation of the operation unit 14, the input password is set as authentication information of the reproduction auxiliary data encoded in step S29 (step S30).

  Next, in the header information processing unit 8, the reproduction auxiliary data encoded in step S 29, the authentication information (password) set in step S 30 and the step S 27 in the file header of the scene reference raw data generated in step S 26. The generated photographing information data is attached as tag information (step S31), and an attached data file (see FIG. 22) is created (step S32). The attached data file is recorded and stored in the recording medium of the storage device 9 (step S33), and the image data recording process is completed.

  As described above, according to the imaging apparatus 101 shown in FIG. 21, in addition to the reproduction assistance data, the shooting information data that is the shooting condition setting at the time of shooting is attached to the file header of the scene reference raw data and recorded on the recording medium. Thus, when outputting the data recorded on the recording medium onto the output medium, it is possible to generate appreciation image reference data according to the shooting situation.

  The recording media on which the attached data file is recorded is taken out from the main body of the image pickup device and mounted on an external device such as an image processing device or an image recording device. The external device is optimal for forming an appreciation image on an output medium. Appreciation image reference data is generated by performing image processing.

Next, an image recording apparatus that performs image processing that optimizes image data recorded on a recording medium for the formation of an appreciation image on an output medium will be described.
<Configuration of image recording device>
FIG. 24 shows an external configuration of the image recording apparatus 201 according to the embodiment of the present invention.

  The image recording apparatus 201 is provided with a magazine loading unit 203 on one side of a main body 202. In the main body 202, an exposure processing unit 204 that exposes a silver salt printing paper as an output medium, and an exposed silver salt printing. A print creating unit 205 is provided for creating a print by developing and drying the paper. The print created by the print creation unit 205 is discharged to a tray 206 provided on the other side of the main body 202. Further, a control unit 207 that controls each unit constituting the image recording apparatus 201 is provided inside the main body 202.

  In addition, a display unit 208, a film scanner unit 209 that is a transparent document reading device, a reflective document input device 210, and an operation unit 211 are disposed on the upper portion of the main body 202. Furthermore, the main body 202 is provided with an image reading unit 214 that can read image data recorded on various recording media, and an image writing unit 215 that writes image data on various recording media.

  There is a photographic photosensitive material as an original read from the film scanner unit 209 or the reflective original input device 210. Examples of the photographic light-sensitive material include a color negative film, a color reversal film, a black and white negative film, a black and white reversal film and the like, and frame image information captured by an analog camera is recorded. The film scanner unit 209 converts the frame image information recorded on the photographic photosensitive material into digital image data to obtain frame image data. Further, when the photographic photosensitive material is color paper which is silver salt photographic paper, the reflective original input device 210 converts the frame image information recorded on the silver salt photographic paper into frame image data by a flatbed scanner.

  The image reading unit 214 includes a PC card adapter 214a and a floppy (registered trademark) disk adapter 214b, into which a PC card 213a and a floppy (registered trademark) disk 213b can be inserted. The PC card 213a has, for example, a memory in which a plurality of frame image data is stored after being captured by a digital camera. For example, a plurality of frame image data captured by a digital camera is recorded on the floppy (registered trademark) disk 213b. Examples of recording media on which frame image data is recorded in addition to the PC card 213a and the floppy (registered trademark) disk 213b include a multimedia card (registered trademark), a memory stick (registered trademark), MD data, and a CD-ROM. It is done.

  The image writing unit 215 includes a floppy (registered trademark) disk adapter 215a, an MO adapter 215b, and an optical disk adapter 215c. ing. Examples of the optical disk 216c include a CD-R and a DVD-R.

  In FIG. 24, the operation unit 211, the display unit 208, the film scanner unit 209, the reflection original input device 210, and the image reading unit 214 are integrally provided in the main body 202. One or more of them may be provided separately.

  In the image recording apparatus 201 shown in FIG. 24, an apparatus that exposes a photosensitive material and develops it to create a print is exemplified. However, the print creation method is not limited to this. You may make it use systems, such as a photography system, a heat sensitive system, and a sublimation system.

<Internal configuration of image recording device>
FIG. 25 shows the internal configuration of the image recording apparatus 201. As shown in FIG. 25, the image recording apparatus 201 includes a control unit 207, an exposure processing unit 204, a print creation unit 205, a film scanner unit 209, a reflective original input device 210, an image reading unit 214, an image writing unit 215, data The storage unit 271, the template storage unit 272, the operation unit 211, and the display unit 208 are configured.

  The control unit 207 is composed of a microcomputer, and cooperates with various control programs stored in a storage unit (not shown) such as a ROM (Read Only Memory) and a CPU (Central Processing Unit) (not shown). The operation of each part constituting the image recording apparatus 201 is controlled.

  Further, the control unit 207 includes an image processing unit 270, and based on an input signal from the information input unit 12 of the operation unit 211, image data read from the film scanner unit 209 or the reflective original input device 210, image reading The image data read from the unit 214 or image data input from an external device via communication means (not shown) is subjected to image processing to form exposure image information, which is output to the exposure processing unit 204. The image processing unit 270 performs conversion processing according to the output form on the image data that has been subjected to image processing, and outputs the image data to a designated output destination. Output destinations of the image processing unit 270 include a display unit 208, an image writing unit 215, communication means, and the like.

  The exposure processing unit 204 exposes an image on the photosensitive material and outputs the photosensitive material to the print creating unit 205. The print creating unit 205 develops and exposes the exposed photosensitive material to create prints P1, P2, and P3. The print P1 is a service size, high-definition size, panoramic size print, the print P2 is an A4 size print, and the print P3 is a business card size print.

  The film scanner unit 209 reads a frame image recorded on a transparent original such as a developed negative film N or a reversal film captured by an analog camera, and acquires a digital image signal of the frame image. The reflection original input device 210 reads an image on a print P (photo print, document, various printed materials) by a flat bed scanner, and acquires a digital image signal.

  The operation unit 211 is provided with information input means 212. The information input unit 212 is configured by a touch panel, for example, and outputs an operation signal of the information input unit 212 to the control unit 207. Further, the operation unit 211 may be configured to include a keyboard and a mouse.

  The image reading unit 214 reads frame image information recorded on the PC card 213 a and the floppy (registered trademark) disk 213 b and transfers the frame image information to the control unit 207. The image reading unit 214 includes, as the image transfer means 230, a PC card adapter 214a, a floppy (registered trademark) disk adapter 214b, and the like. The image reading unit 14 reads frame image information recorded on the PC card 213a inserted into the PC card adapter 214a or the floppy (registered trademark) disk 213b inserted into the floppy (registered trademark) disk adapter 214b. Transfer to the control unit 207. For example, a PC card reader or a PC card slot is used as the PC card adapter 214a.

  The image writing unit 215 includes a floppy (registered trademark) disk adapter 215 a, an MO adapter 215 b, and an optical disk adapter 215 c as the image transport unit 231. In accordance with a write signal input from the control unit 207, the image writing unit 215 includes a floppy (registered trademark) disk 216a inserted into the floppy (registered trademark) disk adapter 215a, an MO 216b inserted into the MO adapter 215b, The image data generated by the image processing method according to the present invention is written to the optical disk 216c inserted into the optical disk adapter 215c.

  The data storage means 271 stores and sequentially stores image information and corresponding order information (information on how many prints are created from images of which frames, print size information, etc.).

  The template storage unit 272 stores sample image data (data indicating a background image, an illustration image, etc.) corresponding to the sample identification information D1, D2, and D3, and a template for setting a synthesis area with the sample image data. Store at least one piece of data. Here, when a predetermined template is selected from a plurality of templates stored in advance in the template storage unit 272 by an operator's operation (the operator's operation is based on a client instruction), the control unit 207 displays frame image information When the sample identification information D1, D2, D3 is designated by the operator's operation (the operator's operation is based on the instruction of the client), the designated sample identification is performed. Sample image data is selected based on the information D1, D2, and D3, the selected sample image data is combined with image data and / or character data ordered by the client, and as a result, the sample image desired by the client is obtained. Create a print based on the data. The synthesis using this template is performed by a well-known chroma key method.

The sample identification information is not limited to the three types of sample identification information D1, D2, and D3, but may be more or less than three types.
The sample identification information D1, D2, and D3 for specifying the print sample is configured to be input from the operation unit 211. However, the sample identification information D1, D2, and D3 are input to the print sample or the order sheet. Since it is recorded, it can be read by reading means such as OCR (Optical Character Reader). Or an operator can also input from a keyboard.

  In this way, sample image data is recorded corresponding to the sample identification information D1 for designating the print sample, the sample identification information D1 for designating the print sample is input, and based on the input sample identification information D1. Select the sample image data, synthesize the selected sample image data with the image data and / or text data based on the order, and create a print based on the specified sample. Can actually place a print order and meet the diverse requirements of a wide range of users.

  Also, the first sample identification information D2 designating the first sample and the image data of the first sample are stored, and the second sample identification information D3 designating the second sample and the second sample The image data is stored, the sample image data selected based on the designated first and second sample identification information D2 and D3, and the image data and / or character data based on the order are synthesized, and according to the designation In order to create a print based on a sample, it is possible to synthesize a wider variety of images, and it is possible to create a print that meets a wider variety of user requirements.

  The display unit 208 is configured by a display such as a CRT or LCD, and performs display processing in accordance with a display control signal input from the control unit 207.

  The image processing unit 270 of the control unit 207 uses communication means (not shown) from another computer in the facility where the image recording apparatus 201 is installed, or from a distant computer via a communication network such as the Internet. It is also possible to receive image data representing a picked-up image and a work instruction such as printing, and to perform image processing or create a print by remote operation.

  Further, the image processing unit 270 uses communication means (not shown) to send image data representing the captured image after the image processing of the present invention and the accompanying order information to another computer in the facility, the Internet, etc. It is also possible to send to a distant computer via

  As described above, the image recording apparatus 201 is configured to input image information obtained by dividing and metering images of various recording media and image originals, and image information of the input image input from the input unit is “the size of the output image”. Image processing means for performing processing so as to obtain an image that gives a favorable impression when acquiring or estimating the information “size” and “the size of the main subject in the output image” and observing the image on the output medium; To send image data and accompanying order information to another computer in the facility or to a distant computer via the Internet, etc. Means.

  FIG. 26 shows an internal configuration when the image recording apparatus 201 is divided into an image processing apparatus 301 and an output unit 302 that outputs image data processed by the image processing apparatus 301.

  As shown in FIG. 26, the image processing apparatus 301 includes an input unit 303, a header information analysis unit 304, an imaging device characteristic correction processing unit 305, an appreciation image reference data restoration condition generation unit 306, a processing process reproduction unit 307, and an optimization process. A unit 308 and a communication unit 320 are included.

  The input unit 303 includes the image reading unit 214 in FIG. 25 and includes a mounting unit for mounting a recording medium. When the recording medium is loaded in the loading unit, the input unit 303 reads the data file recorded on the recording medium and outputs the data file to the header information analysis unit 304. In the present embodiment, the input unit 303 is described as reading a data file from the attached recording medium, but the data file may be input via the communication unit 320.

  The header information analysis unit 304 analyzes the data file input from the input unit 303 and uses the data file as scene reference raw data, encoded reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration information). , Processing process reproduction information), authentication information and photographing information data. When the authentication information (password) for authenticating the use of the auxiliary reproduction data is input from the operation unit 211, the header information analysis unit 304 includes the input authentication information and the file Check the authentication information set in the header. As a result of the collation, when both pieces of authentication information match, the decoding information of the auxiliary reproduction data is acquired from the server (the server owned by the manufacturer of the imaging device) via the communication unit 320, and the decoding information acquired by the acquired decoding information. Decode reproduction auxiliary data. Then, the raw scene reference data is output to the scene reference image data generation unit 311 in the imaging device characteristic correction processing unit 305, the imaging device characteristic correction information is output to the device characteristic correction processing unit 309, and the viewing image reference data restoration information is output. The image data is output to the appreciation image reference data restoration condition generation unit 306, the process process reproduction information is output to the process process reproduction unit 307, and the shooting information data is output to the shooting information data processing unit 313 in the optimization processing unit 308.

  The imaging device characteristic correction processing unit 305 includes an apparatus characteristic correction processing unit 309, a processing condition table 310, a scene reference image data generation unit 311, and a temporary storage memory 312.

  The device characteristic correction processing unit 309 determines a scene reference image data generation condition based on the imaging device characteristic correction information input from the header information analysis unit 304 and the processing condition table 310. The processing condition table 310 is a table that stores processing conditions for generating scene reference image data for each characteristic of the imaging apparatus.

  The scene reference image data generation unit 311 performs imaging device characteristic correction processing on the scene reference raw data input from the header information analysis unit 304 in accordance with the generation conditions determined by the device characteristic correction processing unit 309, so that the imaging device Standardized scene reference image data independent of characteristics is generated and output to the temporary storage memory 312. Specifically, in the imaging device characteristic correction processing, the signal intensity of each color channel based on at least the spectral sensitivity inherent to the imaging device of the imaging device that generated the scene reference raw data is set to a standard such as RIM RGB, ERIMM RGB, or scRGB. Includes mapping to color space.

  The temporary storage memory 312 temporarily stores the scene reference image data generated by the scene reference image data generation unit 311.

  The appreciation image reference data restoration condition generation unit 306 determines a restoration condition for restoring appreciation image reference data in the imaging device based on the appreciation image reference data restoration information input from the header information analysis unit 304.

  Based on the process process reproduction information input from the header information analysis unit 304, the process process reproduction unit 307 determines a reproduction condition for reproducing the appreciation image reference data generation process in the imaging apparatus.

  The optimization processing unit 308 includes a shooting information data processing unit 313, an appreciation image reference data generation unit 314, a temporary storage memory 315, and a setting input unit 316.

  The shooting information data processing unit 313 determines a generation condition for generating appreciation image reference data corresponding to the shooting condition based on the shooting information data input from the header information analysis unit 304.

  The appreciation image reference data generation unit 314 reads the scene reference image data from the temporary storage memory 312, the appreciation image reference data generation condition determined by the shooting information data processing unit 313, and the appreciation image reference data restoration condition generation unit 306. Based on the determined restoration condition, the reproduction condition determined by the processing process reproduction unit 307, and the information of the output destination (storage device 318, output device 317, display unit 208) designated from the setting input unit 316, The scene reference image data is subjected to optimization processing for obtaining an optimum image at the output destination to generate appreciation image reference data, which is output to the temporary storage memory 315 together with the operation information of the setting input unit 316. The optimization processing includes, for example, compression to the output destination color gamut, gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, processing to cope with output characteristics (LUT) of output devices and display devices, and the like. It is. Furthermore, image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment, and dodging processing is included.

  The temporary storage memory 315 temporarily stores the appreciation image reference data generated by the appreciation image reference data generation unit 314, and the output destination (storage device 318, output device 317, display unit 208) set by the setting input unit 316. Any of the above).

  The setting input unit 316 is an input device for designating an output destination of the appreciation image reference data generated by the appreciation image reference data generation unit 314, and corresponds to the operation unit 211 of FIG.

  The communication unit 320 controls communication with an external device connected to a communication network such as the Internet. As an external device connected to this communication network, for example, there is a server (a server owned by the manufacturer of the imaging device) that stores decoding information for decoding (decoding) auxiliary reproduction data encoded on the imaging device side. is there.

  Of the components of the image processing apparatus 301, components other than the input unit 303, the setting input unit 316, and the communication unit 320 are included in the image processing unit 270 illustrated in FIG.

  The output unit 302 includes a display unit 208, an output device 317 corresponding to the exposure processing unit 204 and the print creation unit 205 in FIG. 25, and a storage device 318 corresponding to the image writing unit 215 in FIG.

Next, the operation in the image recording apparatus 201 will be described.
With reference to FIG. 27, the image processing executed in the image recording apparatus 201 will be described.

  When data is input to the input unit 303 (that is, when a recording medium is mounted on the mounting unit) (step S40), the data file recorded on the recording medium is read out, and the header information analysis unit 304 The content of the data file is analyzed (step S41), scene reference raw data, encoded reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration information, processing process reproduction information), authentication information, and shooting information data It is divided into.

  Next, a password input screen prompting input of authentication information (password) for authenticating the use of the reproduction auxiliary data is displayed on the display unit 208. When authentication information (password) is input to the password input screen by the operation unit 211 (step S42; YES), the input authentication information is collated with the authentication information read from the recording medium in step S41. (Step S43). As a result of the collation in step S43, when both pieces of authentication information match (step S44; YES), the decoding information of the reproduction auxiliary data is acquired from the server (the server owned by the manufacturer of the imaging device) via the communication unit 320, The reproduction auxiliary data read in step S41 is decoded by the acquired decoding information (step S45). As a result of the collation in step S43, if the two pieces of authentication information do not match (step S44; NO), the image processing ends.

  The shooting information data processing unit 313 determines generation conditions for generating appreciation image reference data corresponding to the shooting conditions based on the shooting information data (step S46), and the appreciation image reference data restoration condition generation unit 306 Based on the decoded appreciation image reference data restoration information, a restoration condition for restoring the appreciation image reference data in the imaging apparatus is determined (step S48), and the processing process reproduction unit 307 decodes the decoded process process reproduction information. Based on the above, the reproduction condition for reproducing the generation process of the appreciation image reference data in the imaging device is determined (step S49).

  In the apparatus characteristic correction processing unit 309, a generation condition for scene reference image data is determined by referring to the decoded imaging apparatus characteristic correction information and the processing condition table 310, and the scene reference raw data is determined according to the determined generation condition. Imaging device characteristic correction processing is performed (step S47), and scene reference image data is generated (step S50).

  Next, the scene reference image data generated in step S50 is optimized based on various image processing conditions (appearance image reference data generation conditions, restoration conditions, reproduction conditions) determined in steps S46, S48, and S49. Processing is performed (step S51), and appreciation image reference data is generated (step S52).

  The appreciation image reference data is subjected to processing specific to the output destination set in the setting input unit 316 (operation unit 211) (step S53), and is output from the output destination device (step S53). (S54), the image processing ends.

  In the flowchart of FIG. 27, the image processing for the data file including the shooting information data as shown in FIG. 22 has been described. However, in the image processing for the data file not including the shooting information data as shown in FIG. The process of step S46 of 27 is not performed.

  If the decoding information for decoding the encoded reproduction auxiliary data is attached to the data file read from the recording medium in step S41, the decoding information attached to the data file is displayed in step S45. The reproduction assistance data is decoded using the data.

  As described above, according to the image recording apparatus 201 of the present embodiment, the viewing image optimized based on the reproduction information data and the shooting information data attached to the file header of the scene reference raw data output from the imaging apparatus. By generating the reference data, it is possible to obtain high-quality appreciation image reference data that is higher than the image data obtained by the imaging device without any information loss of the captured image information.

  In particular, only when the use of the encoded reproduction information data recorded on the recording medium is authenticated, the encoded reproduction information data can be used by decoding and obtained by the imaging apparatus. The use of image data can be restricted, and information related to the generation of appreciation image reference data can be protected.

  Even if the reproduction assistance data does not contain processing process reproduction information, sufficiently high quality viewing image reference data can be obtained. However, if this processing process reproduction information is included, higher quality viewing image reference can be obtained. Data can be obtained.

1 is a block diagram showing a main part configuration of an imaging apparatus according to an embodiment of the present invention. The block diagram which shows the internal structure of a reproduction auxiliary data generation part. The figure which shows the data structure of the data file recorded on the recording medium of the storage device of FIG. The figure for demonstrating the content of imaging device characteristic correction information. The figure which shows the gradation conversion characteristic of scene reference image data, and the figure which shows the gradation conversion characteristic of scene reference image data and appreciation image reference data (b). The figure which shows the conversion characteristic from scene reference image data to appreciation image reference data, and the figure (b) which shows the gradation conversion characteristic of appreciation image reference data for every light source condition and exposure condition. The figure which shows an example of the data structure of processing process reproduction information. The figure which shows the discrimination | determination map for discriminating a photography scene. 5 is a flowchart illustrating image data recording processing executed in the imaging apparatus according to the embodiment. 6 is a flowchart illustrating a shooting scene determination process executed in the imaging apparatus. The flowchart which shows the 1st occupation rate calculation process which calculates the 1st occupation rate for every area | region of lightness and hue. The figure which shows an example of the program which converts from RGB to HSV color system. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r1 and the area | region r2 on a VH plane. The figure which shows the brightness | luminance (V) -hue (H) plane, and the area | region r3 and the area | region r4 on a VH plane. The figure which shows the curve showing the 1st coefficient for multiplying the 1st occupation rate for calculating parameter | index (alpha). The figure which shows the curve showing the 2nd coefficient which multiplies the 1st occupation rate for calculating parameter | index (beta). The flowchart which shows the 2nd occupation rate calculation process which calculates a 2nd occupation rate based on the composition of picked-up image data. The figure which shows the area | regions n1-n4 determined according to the distance from the outer edge of the screen of picked-up image data. The figure which shows the curve showing the 3rd coefficient multiplied by the 2nd occupation rate for calculating parameter | index (gamma) according to area | region (n1-n4). The flowchart which shows a deviation amount calculation process. The block diagram which shows the principal part structure of the imaging device in the modification of this embodiment. The figure which shows the data structure of the data file recorded on the recording medium of the storage device of FIG. 9 is a flowchart showing image data recording processing executed in an imaging apparatus according to a modified example of the embodiment. 1 is an external view of an image recording apparatus according to an embodiment of the present invention. 1 is a block diagram showing an internal configuration of an image recording apparatus. The block diagram which shows the internal structure of an image processing apparatus and an output part. 6 is a flowchart showing image processing executed in the image recording apparatus.

Explanation of symbols

1 Lens 2 Aperture 3 CCD
4 Analog processing circuit 5 A / D converter 6 Temporary storage memory 7 Image processing unit 8 Header information processing unit 9 Storage device 10 CCD drive circuit 11 Control unit 12 Reproduction auxiliary data generation unit 121 Imaging device characteristic correction information generation unit 122 Appreciation image Reference data restoration information generation unit 123 Processing process reproduction information generation unit 13 Shooting information data generation unit 14 Operation unit 15 Display unit 16 Strobe drive circuit 17 Strobe 18 Focal length adjustment circuit 19 Automatic focus drive circuit 20 Motor 21 Encoding / authentication information setting Numerals 100 and 101 Imaging device 201 Image recording device 202 Main body 204 Exposure processing unit 205 Print creation unit 207 Control unit 208 Display unit 209 Film scanner unit 210 Reflected original input device 211 Operation unit 214 Image reading unit 215 Image writing unit 270 Image processing Unit 301 Image processing apparatus 302 Input unit 303 input unit 304 header information analysis unit 305 imaging device characteristic correction processing unit 306 appreciation image reference data restoration condition generation unit 307 processing process reproduction unit 308 optimization processing unit 309 device characteristic correction processing unit 310 processing condition table 311 scene reference image Data generation unit 312 Temporary storage memory 313 Shooting information data processing unit 314 Appreciation image reference data generation unit 315 Temporary storage memory 316 Setting input unit 317 Output device 318 Storage device 320 Communication unit

Claims (21)

Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Reproduction auxiliary data generating means for generating reproduction auxiliary data when generating image appreciation image reference data by performing image processing optimized for appreciation image formation on an output medium with respect to the scene reference raw data;
Authentication information setting means for setting authentication information for authenticating the use of the generated reproduction auxiliary data;
Recording control means for recording on a recording medium with the reproduction auxiliary data and the authentication information attached to the scene reference raw data;
An imaging apparatus comprising: Encoding means for encoding the generated reproduction auxiliary data by a predetermined encoding method;
The imaging apparatus according to claim 1, wherein the recording control unit attaches the encoded reproduction assistance data and the authentication information to the scene reference raw data. Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;
Reproduction auxiliary data generating means for generating reproduction auxiliary data when generating image appreciation image reference data by performing image processing optimized for appreciation image formation on an output medium with respect to the scene reference raw data;
Shooting information data generating means for generating shooting information data indicating shooting condition settings at the time of shooting;
Authentication information setting means for setting authentication information for authenticating the use of the generated reproduction auxiliary data;
Recording control means for recording the recording auxiliary data, the shooting information data, and the authentication information on a recording medium attached to the scene reference raw data;
An imaging apparatus comprising: Encoding means for encoding the reproduction auxiliary data by a predetermined encoding method;
The imaging apparatus according to claim 3, wherein the recording control unit attaches the encoded reproduction auxiliary data, the shooting information data, and the authentication information to the scene reference raw data.   5. The reproduction auxiliary data includes correction information necessary for imaging apparatus characteristic correction processing for generating standardized scene reference image data from scene reference raw data. The imaging device according to one item.   6. The reproduction auxiliary data includes restoration information for restoring appreciation image reference data in an imaging device when generating appreciation image reference data on an output medium. The imaging device according to any one of the above.   2. The reproduction auxiliary data includes reproduction information for reproducing a process of generating appreciation image reference data in an imaging device when generating appreciation image reference data on an output medium. The imaging device according to any one of?   The imaging apparatus according to claim 7, wherein the reproduction information includes an index value for determining validity of the imaging condition.   9. The imaging apparatus according to claim 8, wherein the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions at the time of shooting.   The imaging apparatus according to claim 9, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting. Scene reference raw data that depends on the characteristics of the imaging device, and reproduction when the scene reference raw data is subjected to image processing that is optimized for the formation of an appreciation image on the output medium to generate appreciation image reference data Input means for inputting auxiliary data and authentication information for authenticating the use of the reproduction auxiliary data;
When the use of the reproduction auxiliary data is authenticated based on the input authentication information, the input scene reference raw data is subjected to an optimization process based on the input reproduction auxiliary data to be viewed image Viewing image reference data generating means for generating reference data;
An image processing apparatus comprising: The input means inputs auxiliary reproduction data encoded by a predetermined encoding method,
When use of the reproduction auxiliary data is authenticated based on the input authentication information, the decoding auxiliary means includes decoding means for decoding the encoded reproduction auxiliary data,
2. The appreciation image reference data generating unit generates appreciation image reference data by performing an optimization process on the input scene reference raw data based on the decoded reproduction auxiliary data. The image processing apparatus according to 11. Scene reference raw data that depends on the characteristics of the imaging device, and reproduction when the scene reference raw data is subjected to image processing that is optimized for the formation of an appreciation image on the output medium to generate appreciation image reference data Input means for inputting auxiliary data, shooting information data indicating shooting condition settings at the time of shooting, and authentication information for authenticating the use of the reproduction auxiliary data;
When the use of the reproduction assistance data is authorized based on the inputted authentication information, the optimization processing is performed on the inputted scene reference raw data based on the inputted reproduction assistance data and the photographing information data. Appreciation image reference data generating means for generating appreciation image reference data by applying
An image processing apparatus comprising: The input means inputs auxiliary reproduction data encoded by a predetermined encoding method,
When use of the reproduction auxiliary data is authenticated based on the input authentication information, the decoding auxiliary means includes decoding means for decoding the encoded reproduction auxiliary data,
The appreciation image reference data generating means generates an appreciation image reference data by performing an optimization process on the input scene reference raw data based on the decoded reproduction auxiliary data and the shooting information data. The image processing apparatus according to claim 13, characterized in that:   15. The reproduction auxiliary data includes correction information necessary for imaging apparatus characteristic correction processing for generating standardized scene reference image data from scene reference raw data. The image processing apparatus according to one item.   16. The reproduction auxiliary data includes restoration information for restoring appreciation image reference data in an imaging device when generating appreciation image reference data on an output medium. The image processing apparatus according to any one of the above.   12. The reproduction auxiliary data includes reproduction information for reproducing an appreciation image reference data generation process in an imaging device when generating appreciation image reference data on an output medium. The image processing device according to any one of -16.   The image processing apparatus according to claim 17, wherein the reproduction information includes an index value for determining validity of an imaging condition in the imaging apparatus.   The image processing apparatus according to claim 18, wherein the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions at the time of shooting by the imaging apparatus.   The image processing apparatus according to claim 19, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting. The image processing apparatus according to any one of claims 11 to 20,
Image forming means for forming an appreciation image on an output medium using appreciation image reference data generated by the appreciation image reference data generation means of the image processing apparatus;
An image recording apparatus comprising:

Images