WO2006077703A1

WO2006077703A1 - Imaging device, image processing device, and image recording device

Info

Publication number: WO2006077703A1
Application number: PCT/JP2005/023007
Authority: WO
Inventors: Hiroaki Takano; Takeshi Nakajima; Daisuke Sato; Tsukasa Ito
Original assignee: Konica Minolta Photo Imaging, Inc.
Priority date: 2005-01-20
Filing date: 2005-12-15
Publication date: 2006-07-27
Also published as: JP2006203573A

Abstract

It is possible to suppress information loss of acquired image data and create image-to-be-enjoyed reference data having a higher quality than the image data obtained by an imaging device at an image processing device or at an image recording device. The imaging device (100) performs imaging to generates scene reference raw data dependent on the characteristic of the imaging device. A re-creation auxiliary data generation unit (12) subjects the scene reference raw data to image processing for optimizing formation of an image-to-be-enjoyed on an output medium so as to generate re-creation auxiliary data used for generating the image-to-be-enjoyed reference data. A header information processing unit (8)attaches the re-creation auxiliary data to the file header (header area) of the scene reference raw data so as to prepare an attached data file, which is recorded in a recording medium of a storage device (9).

Description

Specification

Imaging apparatus, image processing apparatus, and image recording apparatus

Technical field

The present invention relates to an imaging apparatus such as a digital camera, and an image processing apparatus and an image recording apparatus that perform an optimization process for forming an appreciation image on an output medium on captured image data obtained by the imaging apparatus.

Background art

[0002] In recent years, captured image data obtained by an imaging device such as a digital camera has been recorded on a recording medium such as a CD-R (Compact Disc Recordable), a floppy (registered trademark) disk, a memory card, or the like. Distributed via communication networks such as CRT (Cathode Ray Tube), liquid crystal, plasma, etc., and display on display devices of small liquid crystal monitors of mobile phones, digital printers, inkjet printers, thermal printers, etc. There are various output methods such as printing as a hard copy image using an output device such as the above.

[0003] When outputting captured image data for viewing purposes, in general, gradation adjustment is performed on the captured image data so that a desired image quality can be obtained on a display motor or hard copy used for viewing. Various image processing such as brightness adjustment, color balance adjustment, and sharpness enhancement are performed. In Patent Document 1, scene reference raw data obtained by an imaging device (image processing is performed! /, Raw data that faithfully represents subject information) force standardized scene reference image Auxiliary data necessary for correction processing to generate data is attached to the scene reference raw data and recorded on a recording medium, and the auxiliary data attached to the scene reference raw data on the image processing device or the image recording device side. Disclosed is a technology that enables the creation of optimized viewing images without image information loss by performing image processing using the! Speak.

Patent Document 1: Japanese Patent Application Laid-Open No. 2004-96500

Disclosure of the invention

Problems to be solved by the invention [0004] However, in the conventional image processing described above, information loss of captured image data is suppressed and the degree of freedom of image quality adjustment is improved. However, the image processing apparatus or the image recording apparatus has a problem that it cannot be stably reproduced. In addition, although the various sensors and calculation process history information provided in the imaging device up to the actual shooting with the imaging device are useful information at the time of image quality adjustment, the image processing device or the image recording device Therefore, there is a problem in that it is impossible to generate appreciation image reference data with higher quality than image data obtained by the imaging device.

[0005] An object of the present invention is to suppress information loss of photographed image information and to view a high-quality appreciation image that is higher than image data obtained by an imaging device on the image processing apparatus or image recording apparatus side. It is possible to generate reference data.

Means for solving the problem

[0006] The problems of the present invention can be solved by the following means.

1. Scene reference raw data generation means for generating scene reference raw data depending on the characteristics of the imaging device by imaging, and optimizing the scene reference raw data for forming an appreciation image on an output medium Reproduction auxiliary data generating means for generating reproduction auxiliary data when performing image processing to generate appreciation image reference data, and recording control means for attaching the reproduction auxiliary data to the scene reference raw data and recording it on a recording medium; An imaging apparatus comprising:

2. Image processing means for performing image processing on the scene reference raw data using the reproduction auxiliary data to create the viewing image reference data, and based on the viewing image reference data created by the image processing means. 2. The imaging apparatus according to 1, comprising an image forming means for forming an appreciation image on an output medium.

3. The image processing means standardizes the scene reference raw data to create scene reference data, and performs image processing on the scene reference data using the viewing image reference data restoration information to perform the viewing image 3. The imaging apparatus according to 2, wherein reference data is generated, and the reproduction assistance data includes the viewing image reference data restoration information.

4.Preliminary shooting is performed before actual shooting, and shooting conditions are adjusted according to the result of preliminary shooting. 3. The imaging apparatus according to 2, further comprising a shadow condition adjusting unit, wherein the reproduction assistance data includes processing process reproduction information indicating a history of shooting conditions adjusted before the main shooting by the shooting condition adjusting unit. .

5. The imaging apparatus according to 4, wherein the processing process reproduction information includes an index value for determining the validity of the imaging condition.

6. The imaging apparatus according to 5, wherein the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions during shooting.

7. The imaging apparatus according to 6, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

8. It is provided with shooting information data generating means for generating shooting information data indicating shooting condition setting at the time of shooting, and the recording control means records the shooting information data attached to the scene reference raw data on a recording medium. The imaging apparatus according to any one of 1 to 7, characterized in that:

9. Generate scene appreciation image reference data by applying scene processing raw data that depends on the characteristics of the imaging device and image processing that is optimized for the appreciation image formation on the output medium. An input unit for inputting reproduction auxiliary data at the time, and an appreciation image that generates an appreciation image reference data by performing an optimization process on the input scene reference raw data based on the input reproduction auxiliary data An image processing apparatus comprising: reference data generation means.

10. The reproduction auxiliary data includes appreciation image reference data restoration information for restoring appreciation image reference data in the imaging device when generating appreciation image reference data on the output medium. The image processing apparatus according to 9.

11. The reproduction auxiliary data includes processing process reproduction information for reproducing the generation process of the appreciation image reference data in the imaging device when generating the appreciation image reference data on the output medium. The image processing apparatus according to 9.

12. The image processing apparatus according to 11, wherein the processing process reproduction information includes an index value for determining validity of imaging conditions in the imaging apparatus.

13. The index values include user characteristics, light source conditions, and exposure conditions when shooting with the imaging device. 13. The image processing device according to 12, wherein a value specifying at least one of the values is included.

14. The image processing apparatus according to 13, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

15. The input means inputs photographing condition data indicating photographing condition settings at the time of photographing, and the appreciation image reference data generating means inputs the reproduction auxiliary data inputted with respect to the inputted scene reference raw data. And the image processing apparatus according to any one of 9 to 14, wherein the image processing apparatus generates an appreciation image reference data by performing an optimization process based on the shooting condition data.

16. Forming an appreciation image on the output medium using the image processing device according to any one of 9 to 15 and appreciation image reference data generated by the appreciation image reference data generation means of the image processing device An image recording apparatus comprising: an image forming unit. <Explanation of terms>

Hereinafter, terms used in this specification will be described.

[0007] In the description of this specification, the term "generation" means that a program and a processing circuit that operate in the imaging apparatus, image processing apparatus, and image recording apparatus according to the present invention renew image signals and data. To produce. “Create” may be used as a synonym.

[0008] In the description of this specification, the term "imaging device" refers to a device including an imaging element (image sensor) having a photoelectric conversion function, and includes a so-called digital camera and scanner. . Examples of image sensors include CCD (Charge Coupled Device), a charge transfer mechanism, and a CCD image sensor that provides color sensitivity by combining a pine pattern color filter, CMOS (Complementary Metal). -Oxide Semiconductor) type image sensor. The output current of these image sensors is digitized by A / D change. The contents of each color channel at this stage are signal intensities based on the spectral sensitivity unique to the image sensor.

In addition, “scene reference raw data depending on the characteristics of the imaging device” is a direct raw output signal of the imaging device that records information faithful to the subject, and is digitized by an A / D converter. Noise correction such as fixed pattern noise and dark current noise Means the data performed. This scene reference raw data is used for image processing that modifies the data content to improve the effect of image viewing such as tone conversion 'sharpness enhancement' and saturation enhancement. The process of mapping the signal strength of the channel to a standardized color space such as RIMM R GB or sRGB is omitted. The information amount (for example, the number of gradations) of the scene reference raw data is preferably equal to or greater than the information amount (for example, the number of gradations) required for the viewing image reference data, according to the performance of the A / D converter. Good. For example, if the number of gradations of the viewing image reference data (described later) is 8 bits per channel, the gradation number of the scene reference raw data is preferably 12 bits or more, more preferably 14 bits or more, and more than 16 bits. preferable.

[0010] The "output medium" is, for example, a display device such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), plasma display, silver salt photographic paper, inkjet paper, thermal printer paper, etc. This is a paper for generating a hard copy image.

[0011] “Viewing image reference data” is used for display devices such as CRTs, LCDs, and plasma displays, and generates hard copy images on output media such as silver halide photographic paper, inkjet paper, and thermal printer paper. This means digital image data used for. Appreciation Image reference data is subjected to an “optimization process” so that an optimal image can be obtained on display devices such as CRT, LCD, and plasma display, and output media such as silver halide photographic paper, inkjet paper, and thermal printer paper. This is different from the scene reference raw data.

[0012] The "recording medium" is a storage medium used for storing "scene reference raw data" and "reproduction assistance data" output from the imaging apparatus, and is a compact flash (registered trademark), memory stick (registered). Trademark), SmartMedia (registered trademark), multimedia card, hard disk, floppy (registered trademark) disk, magnetic storage medium (MO), CD-R, etc. In addition, the unit that writes to the recording media can be installed in an independent or remote location connected wirelessly via a communication unit such as the Internet or a communication unit such as the Internet, even if it is integrated with the camera. Any unit such as a unit may be used. The file format when recording to a recording medium is not a format specific to the imaging device. TIFF (Tagged Image File Format), JPEG (Joint Photograp hie Coding Experts Group), Exif (Exchangeable Image File Format), etc.

[0013] The "shooting information data" is a record of shooting condition settings at the time of shooting, and may include the same tag information written in the header part of the Exif7 aisle. Specifically, exposure time, shutter speed, aperture value (F-number), ISO sensitivity, brightness value, subject distance range, light source, flash on / off, subject area, white balance, zoom magnification, shooting scene, strobe light source Tag (code) indicating information on the amount of reflected light, shooting saturation, type of subject, subject configuration, and the like.

[0014] In addition, "shooting information data" is a value obtained at the time of shooting by a sensor provided in the camera, data processed by the sensor, It is classified into the shooting conditions of the camera set based on the sensor value, but in addition to this, the shooting mode dial (for example, portrait, sport, macro shooting mode, etc.) It also includes information that the photographer has manually set the setting switch for forced flash.

[0015] In addition, "standardized scene reference image data" means that the signal intensity of each color channel based on at least the spectral sensitivity of the image sensor itself has been mapped to the standard color space such as RIMM RGB, ERIMM RGB, or scRGB. Yes, it means image data in which the image processing that modifies the data contents is omitted in order to improve the effect at the time of image viewing such as gradation conversion, sharpness enhancement, and saturation enhancement. The scene reference image data is the photoelectric conversion characteristics of the imaging device (opto-electronic conversion fonction defined by ISO 1452, for example, Corona “Fine Imaging and Digital Photography” (published by the Japan Photographic Society Publishing Committee, page 449). It is preferable that the correction is made (see). The amount of information (for example, the number of gradations) of the standardized scene reference image data conforms to the performance of the A / D converter and is equal to or greater than the amount of information (for example, the number of gradations) required for the viewing image reference data. Preferably there is. For example, if the number of gradations of the viewing image reference data is 8 bits per channel, the gradation number of the scene reference image data is preferably 12 bits or more, more preferably 14 bits or more, and even more preferably 16 bits or more.

[0016] In addition, "Imaging device characteristic correction for generating standardized scene reference image data" The “process” means a process of converting “scene reference raw data depending on characteristics of the imaging device” to “standardized scene reference image data”. The content of this process depends on the state of the “scene reference raw data that depends on the characteristics of the imaging device”, but at least the signal intensity of each color channel based on the spectral sensitivity unique to the imaging device is set to RIMM RGB, ERIMM RGB, sc RGB, etc. Mapping to the standard color space. For example, when the “scene reference raw data depending on the characteristics of the imaging device” is not subjected to the interpolation processing based on the color filter array, this processing needs to be additionally performed. (Details of the interpolation processing based on the color filter array are described in, for example, Corona, “Fine Imaging and Digital Photography”, page 51 of the Japan Photographic Society Publishing Committee.) “Standardized scene reference image data” in which the difference in signal value between different “imaging devices” is corrected while having almost the same amount of information as that of “data”.

[0017] In addition, "optimization processing" means obtaining an optimal image on a display device such as a CRT, LCD, plasma display, or an output medium such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. For example, if it is assumed that the image is displayed on a CRT display monitor compliant with the sRGB standard, the process is performed so that the optimum color reproduction is obtained within the color gamut of the sRGB standard. If output to silver salt photographic paper is assumed, processing is performed to obtain optimal color reproduction within the color gamut of silver salt photographic paper. In addition to color gamut compression, it also includes gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, and handling of output device output characteristics (LUT). Furthermore, it goes without saying that image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment or masking, and baking is performed.

[0018] An example of optimizing appreciation image reference data using "shooting information data" is shown below.

[0019] Based on the information of "subject configuration", for example, it is possible to perform a saturation enhancement process partially, or to perform a cover / burn process in a scene with a wide dynamic range.

[0020] According to the determination result of the shooting scene, for example, in night scene shooting, the degree of white balance adjustment can be relaxed and the color balance can be adjusted specially.

[0021] The distance between the photographer and the subject is estimated based on the "amount of reflected light from the strobe light source" information, and can be reflected in, for example, setting of image processing conditions that suppress over-exposed skin.

[0022] Based on the "subject type" information, for example, in person photography, the degree of sharpness is relaxed, By strengthening the smooth wrinkle treatment, the wrinkles of the skin can be made inconspicuous.

[0023] The image recording apparatus according to the present invention includes a color negative film, a color reversal film, a black and white, in addition to a mechanism for performing image processing according to the present invention on digital image data acquired from the imaging apparatus according to the present invention. Film scanners that input frame image information of photographic photosensitive materials recorded by analog cameras, such as negative film and black and white reversal film, and flatbed scanners that input image information reproduced on color paper that is silver salt photographic paper You may have. Also obtained by a digital camera other than the imaging device of the present invention, a compact flash (registered trademark), memory stick (registered trademark), smart media (registered trademark), multimedia card (registered trademark), floppy (registered trademark) disk, Digital image data can be obtained from a remote location via a means for reading digital image data stored on any known portable "recording media" such as magneto-optical storage media (MO) or CD-R, or via communication means. And processing means for forming an appreciation image on any known output medium such as display devices such as CRT, LCD, plasma display, hard copy image generation paper such as silver salt photographic paper, inkjet paper, thermal printer paper, etc. And may be provided.

The invention's effect

[0024] According to the present invention, reproduction auxiliary data for generating appreciation image reference data is generated on the imaging apparatus side, and the reproduction auxiliary data is attached to scene reference raw data depending on the characteristics of the imaging apparatus. When the data recorded on the recording medium is output to the output medium, the viewing image reference data is generated using the reproduction auxiliary data, and the information loss of the captured image information is reduced. In addition to the suppression, it is possible to obtain appreciation image reference data with higher quality than the image data obtained by the imaging device.

[0025] Further, in addition to the reproduction assistance data, the image processing apparatus or the image recording apparatus side reproduces the reproduction assistance data by attaching the shooting information data, which is the shooting condition setting at the time of shooting, to the scene reference raw data. Also, viewing image reference data is generated using the shooting information data, and it is possible to obtain viewing image reference data of higher quality.

Brief Description of Drawings

FIG. 1 is a block diagram showing a main part configuration of an imaging apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing an internal configuration of a reproduction auxiliary data generation unit.

FIG. 3 is a diagram showing the data structure of a data file recorded on the recording medium of the storage device in FIG.

圆 4] A diagram for explaining the contents of the imaging device characteristic correction information.

圆 5] A diagram showing the tone conversion characteristics of scene reference image data (a), and a diagram showing the tone conversion characteristics of scene reference image data and appreciation image reference data (b).

圆 6] A diagram showing the conversion characteristics of scene reference image data into appreciation image reference data (a), and a diagram showing gradation conversion characteristics of appreciation image reference data for each light source condition and exposure condition (b).

FIG. 7 is a diagram showing an example of the data structure of processing process reproduction information.

8] A diagram showing a discrimination map for discriminating a shooting scene.

FIG. 9 is a flowchart showing image data recording processing executed in the imaging apparatus of the embodiment.

FIG. 10 is a flowchart showing a shooting scene determination process executed in the imaging apparatus. [11] A flowchart showing a first occupancy ratio calculation process for calculating a first occupancy ratio for each area of brightness and hue.

FIG. 12 is a diagram showing an example of a program for converting RGB power into the HSV color system.

FIG. 13 is a diagram showing the lightness (V) —hue (H) plane and the region rl and region r2 on the V—H plane.

FIG. 14 is a diagram showing the brightness (V) —hue (H) plane and the region r3 and the region r4 on the V—H plane. 15] A diagram showing a curve representing a first coefficient for multiplying the first occupancy ratio for calculating the index α.

[16] A diagram showing a curve representing a second coefficient for multiplying the first occupancy rate for calculating the index j8.

FIG. 17 is a flowchart showing a second occupancy ratio calculation process for calculating a second occupancy ratio based on the composition of captured image data.

FIG. 18 is a diagram showing regions nl to n4 determined according to the distance from the outer edge of the screen of captured image data.

[19] A diagram showing, by region (nl to n4), curves representing the third coefficient for multiplying the second occupancy rate for calculating the index γ. FIG. 20 is a flowchart showing a bias amount calculation process.

FIG. 21 is a block diagram showing a main part configuration of an imaging apparatus in a modification of the present embodiment.

FIG. 22 is a diagram showing a data structure of a data file recorded on a recording medium of the storage device of FIG.

FIG. 23 is a flowchart showing image data recording processing executed in an imaging apparatus according to a modification of the present embodiment.

FIG. 24 is an external view of an image recording apparatus according to an embodiment of the present invention.

FIG. 25 is a block diagram showing the internal configuration of the image recording apparatus.

FIG. 26 is a block diagram showing the internal configuration of the image processing apparatus and output unit.

FIG. 27 is a flowchart showing image processing executed in the image recording apparatus.

Explanation of symbols

5 AZD strange ^^

8 Header information processing section

9 Storage device

12 Reproduction auxiliary data generator

121 Imaging device characteristic correction information generator

122 Appreciation image reference data restoration information generator

123 Processing process reproduction information generator

13 Shooting information data generator

15 Display

100, 101 Imaging device

201 Image recording device

301 Image processing device

303 Input section

306 Appreciation image reference data restoration condition generator

307 Processing process reproduction section

313 Image data processing unit

314 Appreciation image reference data generator BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, the configuration in the present embodiment will be described.

FIG. 1 shows a main part configuration of an imaging apparatus 100 according to an embodiment of the present invention. As shown in FIG. 1, the imaging device 100 includes a lens 1, an aperture 2, a CCD (Charge Coupled Device) 3, an analog processing circuit 4, an AZD converter 5, a temporary storage memory 6, an image processing unit 7, a header. Information processing section 8, storage device 9, CCD drive circuit 10, control section 11, reproduction auxiliary data generation section 12, operation section 14, display section 15, strobe drive circuit 16, strobe 17, focal length adjustment circuit 18, autofocus A drive circuit 19 and a motor 20 are provided.

[0030] The optical system of the imaging apparatus 100 includes a lens 1, a diaphragm 2, and a CCD 3.

The lens 1 adjusts the focus and forms an optical image of the subject. Aperture 2 adjusts the amount of light beam that has passed through lens 1. The CCD 3 photoelectrically converts the subject light imaged on the light receiving surface by the lens 1 into an electrical signal (imaging signal) of an amount corresponding to the amount of incident light for each sensor in the CCD 3. Then, the CCD 3 is sequentially controlled by the timing pulse input from the CCD drive circuit 10, and sequentially outputs this imaging signal to the analog processing circuit 4.

The analog processing circuit 4 performs RGB signal amplification, noise reduction processing, and the like on the image pickup signal input from the CCD 3. The processing in the analog processing circuit 4 is switched ONZOFF via the control unit 11 in accordance with the operation signal of the operation unit 14 force.

[0033] The AZD converter 5 converts the imaging signal input from the analog processing circuit 4 into a digital signal and outputs the digital signal. In the following, the digital signal obtained by AZD modification 5 will be described as scene reference raw data.

The temporary storage memory 6 is a notch memory or the like, and temporarily stores the image data output from the AZD modification 5.

[0035] The image processing unit 7 performs tone correction, spectral sensitivity crosstalk correction, dark current noise suppression, sharpening, for display on the display unit 15 with respect to the image data stored in the temporary storage memory 6. In addition to image quality improvement processing such as white balance adjustment and saturation adjustment, processing such as image size change, trimming, and aspect conversion is performed. The processing in the image processing unit 7 is switched ONZOFF via the control unit 11 in accordance with an operation signal from the operation unit 14.

[0036] The header information processing unit 8 adds the reproduction auxiliary data (details will be described later) generated by the reproduction auxiliary data generation unit 12 to the file header (header area) of the scene reference raw data stored in the temporary storage memory 6. Attach and create the attached data file (see Figure 3).

The storage device 9 is configured by a nonvolatile semiconductor memory or the like, and stores a control program for the imaging apparatus 100. In addition, the storage device 9 includes a mounting unit for mounting a recording medium such as a memory card, and in accordance with a control signal input from the control unit 11, reading of recorded data of a recording medium mounted on the mounting unit, Write data to the recording media.

[0038] The CCD drive circuit 10 outputs a timing pulse in accordance with a control signal input from the control unit 11, and controls the drive of the CCD 3.

[0039] The control unit 11 is configured by a CPU (Central Processing Unit) or the like, and is stored in the storage device 9, reads out a control program of the imaging device 100, and controls the entire imaging device 100 in accordance with the read control program. I do.

[0040] For example, the control unit 11 includes an automatic focus driving circuit 19 that controls a motor 20 that adjusts the focal length and focus (focus) of the lens 1 in accordance with an operation signal from the operation unit 14, and a focal length adjustment. By controlling the circuit 18, the CCD drive circuit 10, the analog processing circuit 4, the temporary storage memory 6, the image processing unit 7, the display unit 15, and the strobe driving circuit 16, the photographing operation is executed. In addition, the control unit 11 instructs the reproduction auxiliary data generation unit 12 to generate reproduction auxiliary data (details will be described later) to be attached to the file header of the scene reference raw data obtained by shooting. Instructs the recording medium to record a data file with supplementary reproduction data attached to the scene reference raw data.

[0041] The reproduction auxiliary data generation unit 12 generates reproduction auxiliary data necessary for generating appreciation image reference data by performing an optimization process for appreciation image formation on the output medium. The reproduction assistance data is output to the header information processing unit 8. Reproduction auxiliary data generation The internal configuration of the unit 12 will be described in detail later with reference to FIG.

The operation unit 14 includes various function buttons such as a shutter button, a power ON / OFF button, and a zoom button, a cursor key, and the like, and outputs an operation signal corresponding to each button and key to the control unit 11. It consists of a play etc. and performs the required display processing according to the display control signal input by the control unit 11. For example, the display unit 15 displays information for the user of the image capturing apparatus 100 to confirm the conditions for shooting, or displays the viewing image reference data generated for display on the display unit 15. To do. The display unit 15 also has a function as a finder that continuously displays images captured by the CCD 3 in the shooting mode.

The strobe driving circuit 16 controls the strobe 17 to emit light when the subject brightness is low, based on a control signal input from the control unit 11.

The strobe 17 boosts the battery voltage to a predetermined high voltage and stores it as a charge in a capacitor. The strobe 17 is driven by the strobe driving circuit 16 to emit light from the X tube with the electric charge stored in the capacitor, and irradiates the subject with auxiliary light.

The focal length adjustment circuit 18 controls the motor 20 for adjusting the focal length by moving the lens 1 by the control signal input from the control unit 11.

The automatic focus driving circuit 19 controls the motor 20 for moving the lens 1 and adjusting the focus (focus) by the control signal input from the control unit 11.

FIG. 2 shows an internal configuration of the reproduction assistance data generation unit 12. As shown in FIG. 2, the reproduction auxiliary data generation unit 12 includes an imaging device characteristic correction information generation unit 121, an appreciation image reference data restoration information generation unit 122, and a processing process reproduction information generation unit 123.

[0049] The imaging device characteristic correction information generation unit 121 generates information (imaging device characteristic correction information) necessary for imaging device characteristic correction processing for generating scene reference image data in which the scene reference raw data force is also standardized. To do. As shown in FIG. 4, the imaging device characteristic correction processing includes filter interpolation calculation (a), matrix calculation (b), photoelectric conversion characteristic and gain correction (c) for scene reference raw data.

[0050] The filter interpolation calculation shown in FIG. 4 (a) is the image data having a filter arrangement of one pixel and one color. Force Interpolation to linear image data of 1 pixel 3 colors (RGB). Here, the filter array is an array pattern of color filters for CCD color discrimination, and an array of RGB primary colors is generally used. As the filter interpolation method, it is possible to apply the neare st neighbor method, the bi-linear interpolation method, the noi ¹ ~~ Higg (bi-cubic convolution) method, etc. Is possible.

[0051] In the two-arrest neighbor method, a pixel nearest to the target pixel is selected, and the pixel is enlarged and reduced as it is in accordance with the enlargement / reduction size, and the pixel is interpolated only at a necessary portion. In the non-linear method, linear density interpolation is performed from the density values of the four pixels around the target pixel according to the coordinates (real values). In the bicubic method, interpolation is performed using a cubic function from the density values of 16 pixels around the pixel of interest in order to perform interpolation with higher accuracy than the bilinear method. The formula used for interpolation is 3 (兀) / 兀, which is the most complete density interpolation formula in theory (sampling theorem). This is approximated by the third-order term of X in the Tiller expansion and used as an interpolation formula.

[0052] The matrix operation shown in Fig. 4 (b) is an operation for correcting the difference in which the same subject color is recorded as a different signal due to the difference in the spectral characteristics of the color filter and the spectral sensitivity characteristics of the image sensor. It is a process and is converted from tristimulus values of RGB to XYZ color system by this matrix operation.

[0053] Fig. 4 (c) shows processing for correcting the difference due to photoelectric conversion characteristics (linear conversion characteristics) and gain correction (translation) of the image sensor. This processing is shown in Fig. 4 (c As shown in (), the characteristic is that UogY (Y: stimulus value of XYZ color system) is linearly converted to 1 ogE (E: exposure amount).

[0054] By the imaging device characteristic correction processing as shown in Figs. 4 (a) to (c), the scene reference raw data power depending on the characteristics of the imaging device XYZ in a standardized color space such as scene reference image data Converted to a value. Note that the color filter array pattern applied in the imaging device characteristic correction process, matrix coefficients a to i for matrix calculation, and the coefficient values for correcting differences associated with photoelectric conversion characteristics and gain correction are all different for each imaging device. is there.

[0055] The appreciation image reference data restoration information generation unit 122 restores the appreciation image reference data in the imaging device 100 when generating the appreciation image reference data from the scene reference image data. Encoding information of the restoration information (viewing image reference data restoration information) is generated.

FIG. 5 (a) shows the tone conversion characteristics of scene reference image data. As shown in FIG. 5 (a), this tone conversion characteristic is a characteristic that logY (Y: stimulus value of XYZ color system) is converted to linear with respect to logE (E: exposure amount).

[0057] Figure 5 (b) shows the L * a * b * color system L * and logE (same color space) as the gradation conversion characteristics in the sRGB color space of scene reference image data and appreciation image reference data. E: Exposure amount). In the gradation conversion characteristics shown in Fig. 5 (b), the appreciation image reference data now has a characteristic that L * changes linearly with respect to log E.

[0058] Fig. 6 (a) shows the conversion characteristics (curved line in the figure) of the viewing image reference data from the scene reference image data. The conversion characteristics shown in Fig. 6 (a) can be corrected with the shooting scene at the time of shooting (front light, backlight, under, flash proximity, etc.). Figure 6 (b) shows the gradation conversion characteristics of the appreciation image reference data for correcting the light conditions for backlighting, flash close-up photography, and under / over exposure conditions. The combination of the conversion characteristics shown in Fig. 6 (a) and the conversion characteristics shown in Fig. 6 (b) is the viewing image reference data restoration information.

[0059] The processing process reproduction information generation unit 123 reproduces the process of generating appreciation image reference data in the imaging apparatus 100 when the scene reference image data power and the appreciation image reference data are generated (processing process reproduction). Information).

FIG. 7 shows an example of processing process reproduction information. The processing process reproduction information indicates history information of shooting conditions from the actual shooting to a predetermined time before (for example, 2500 ms before). Specifically, as shown in FIG. Indicators 1 to 3 indicating the processing result of the discrimination process of shooting scenes (front light, backlight, under, flash proximity, etc.), total indicator 4 calculated from indicators 1 to 3, and evaluation of the calculated indicator Show history information for value 1, value for camera shake level evaluation 2, and correctness of shooting conditions.

In the processing process reproduction information, the camera shake level indicates the degree of camera shake in, for example, 10 levels (an integer of 1 to 10). The higher the camera shake level, the higher the value of the camera shake level.

[0062] Indices 1 to 3 are numerical values for specifying a shooting scene (forward light, backlight, under, flash proximity, etc.) at the time of shooting. Fig. 8 shows the discrimination map for discriminating the shooting scene. In this embodiment, as shown in FIG. 8, the index 1 is the strobe degree (strobe degree), the index 2 is the backlight intensity (backlight degree), and the index 3 is the under degree (under degree). As shown in Fig. 8 (a), the area force determined by index 1 (strobe degree) and index 2 (backlight intensity) is also distinguished from backlight and direct light, and index 1 (strobe degree) and index 3 (under degree). Under and flash proximity are discriminated from the area determined by. The shooting scene discrimination process will be described in detail later with reference to FIGS.

[0063] The definition of total index 4 is shown in equation (1). Total index 4 = 10 — [{(index 1 + index 2 + index 3

+ 18) / 36} X 10] (1)

Evaluation value 1 and evaluation value 2 indicate the light source condition 'exposure condition evaluation value and camera shake evaluation value, respectively, and are defined as in equations (2) and (3). Evaluation value 1 = Total index 4 X 3.4— 5 (2) Evaluation value 2 = — 0.54 X Camera shake level +8.5 (3)

Here, in Equations (2) and (3), among indicators 1 to 3, a value of 6 or less is 6 and a value of +6 or more is +6.

[0064] The correctness of shooting conditions is a numerical value indicating the degree of shooting that has been performed correctly, such as the shooting environment and the technical level of the photographer, and is expressed by equation (4) using evaluation value 1 and evaluation value 2. Defined in Shooting condition correctness = evaluation value I X 0.8 + evaluation value 2 X 0.2 (4)

The correctness of the shooting conditions increases as the shooting is performed correctly. As shown in the processing process reproduction information in FIG. 7, the photographing conditions (light source conditions, exposure conditions, etc.) are adjusted so that the numerical value of the photographing condition validity increases as time passes.

[0065] In FIGS. 8 (a) and 8 (b), the history of the indices 1 to 3 indicated by the processing process reproduction information shown in FIG. 7 is indicated by diamonds. The history information on the discrimination map shown in FIG. 8 may be displayed on the display unit 15.

[0066] In the above description, the camera shake level is an integer value of 1 to 10, and the numerical range of the indicators 1 to 3 is 6 or more + 6 or less, but these numerical ranges are not particularly limited. The definition of total index 4, evaluation value 1, and evaluation value 2 will change as the numerical range changes. Various user mode settings may be used as user characteristics other than camera shake. Also, include the shutter speed and aperture value as history information recorded in the process process reproduction information. [0067] The recording of history information as processing process reproduction information starts after the shooting mode is designated by the operation unit 14, and the time when the shutter button is pressed may be the time of actual shooting. Recording of history information starts after the shutter button of Part 14 is pressed halfway, and the time when the shutter button is pressed hard can be used for actual shooting!

FIG. 3 shows the data structure of a data file recorded on the recording medium of the storage device 9. The imaging device characteristic correction information, the appreciation image reference data restoration information, and the processing process reproduction information generated by the reproduction auxiliary data generation unit 12 are processed by the header information processing unit 8 as shown in FIG. The attached data file is created by attaching to the file header as reproduction auxiliary data, and is recorded on the recording medium of the attached data file storage device 9.

[0069] It should be noted that the scene reference image data that is simply used for display on the display unit 15 with the appreciation image reference data generated in the imaging apparatus 100 and the appreciation image reference data are associated with each other on a recording medium. You can record it or record the thumbnail image of the viewing image reference data on the recording medium as metadata of the scene reference raw data.

Next, the operation in the imaging apparatus 100 will be described.

With reference to the flowchart of FIG. 9, the image data recording process executed in the imaging apparatus 100 will be described.

[0072] When the shooting mode is designated by the operation unit 14, preliminary imaging is performed (step S1), and an image obtained by preliminary imaging (hereinafter referred to as preliminary imaging image) is displayed on the display unit 15. The viewing image reference data is formed (step S2), and the formed viewing image reference data is displayed on the display unit 15 (step S3). In step S1, preliminary imaging may be performed when the shutter button is pressed halfway by the operation unit 14.

Next, in the processing process reproduction information generation unit 123, a shooting scene discrimination process (see FIGS. 10 to 20) is performed, and the camera shake level and the indices 1 to ₃ as a result of the shooting scene discrimination process are captured. A value indicating the correctness of the condition is calculated, and the correctness of the calculated imaging force is determined (step S4). Then, the shooting condition is adjusted so that the value of the shooting condition validity is increased (step S5). The processing in steps S1 to S5 is performed by the shutter of the operation unit 14. The process is repeated until the actual shooting is instructed after the button is pressed, and history information of shooting conditions up to a predetermined time before the actual shooting is generated as processing process reproduction information.

[0074] When the shutter button of the operation unit 14 is pressed and a real shooting is instructed (step S6; YES), the imaging signal obtained from the CCD 3 is converted into a digital signal by the AZD change 5 to refer to the scene. Simultaneously with the generation of raw data (step S7), the reproduction auxiliary data generation unit 12 generates imaging device characteristic correction information, appreciation image reference data restoration information, and processing process reproduction information as reproduction auxiliary data ( Step S8). The processing process reproduction information generated in step S8 is obtained by adding information at the time of actual photographing to the history information before actual photographing generated by the processing of steps S1 to S5. Note that a step of creating appreciation image reference data to be recorded on the recording media may be added after the main photographing.

[0075] Next, the header information processing unit 8 attaches the reproduction assistance data generated in step S8 as tag information to the file header of the scene reference raw data generated in step S7. (Step S9), an attached data file (see FIG. 3) is created (Step S10). The attached data file is recorded and stored in the recording medium of the storage device 9 (step S11), and the image data recording process is completed.

Next, with reference to the flowchart of FIG. 10, the shooting scene determination process necessary for determining the validity of the shooting conditions will be described.

First, captured image data (for example, scene reference raw data) is divided into predetermined image areas, and an occupation ratio (a first occupation ratio, a second occupation ratio) indicating a ratio of each divided area to the entire captured image data. Occupancy ratio calculation processing is performed (step T1). Details of the occupation rate calculation process will be described later with reference to FIGS.

[0077] Next, a bias amount calculation process for calculating a bias amount indicating a bias of the gradation distribution of the photographed image data is performed (step T2). The bias amount calculation process in step 2 will be described in detail later with reference to FIG.

[0078] Next, indices 1 to 3 for specifying the shooting scene are calculated based on the occupation ratio calculated in step T1 and a coefficient set in advance according to the shooting conditions (step Τ3), and the main shooting is performed. The scene discrimination process ends. The calculation method of the index in step Τ3 will be detailed later. Explain in detail.

Next, the first occupancy rate calculation process will be described in detail with reference to the flowchart of FIG.

[0080] First, the RGB values of the captured image data are converted into the HSV color system (step T10). Figure 12 shows an example of a conversion program (HSV conversion program) that obtains hue values, saturation values, and brightness values by converting from RGB to the HSV color system in program code (c language). is there. In the HSV conversion program shown in Fig. 12, the digital image data values that are input image data are defined as InR, InG, and InB, the calculated hue value is defined as OutH, the scale is defined as 0 to 360, and the saturation The value is OutS, the brightness value is OutV, and the unit is defined as 0 to 255

[0081] Next, the captured image data is divided into regions having a combination of predetermined brightness and hue, and a two-dimensional histogram is created by calculating the cumulative number of pixels for each divided region (step Tl l). . Hereinafter, the area division of the captured image data will be described in detail.

[0082] Lightness (V) is lightness value power -25 (vl), 26-50 (v2), 51-84 (v3), 85-169 (v4), 170-199 (v5), 200-224 ( v6), divided into 7 regions from 225 to 255 (v7). Hue (H) is a flesh color range (HI and H2) with a hue value of 0 to 39, 330 to 359, a green hue range (H3) with a hue value of 40 to 160, and a blue hue range with a hue value of 61 to 250. It is divided into four areas (H4) and red hue area (H5). Note that the red hue region (H5) is not used in the following calculations because of the fact that it contributes little to the determination of imaging conditions. The flesh color hue area is further divided into a flesh color area (HI) and another area (H2). Hereinafter, of the flesh-colored hue areas (H = 0 to 39, 330 to 359), the hue '(H) that satisfies the following formula (5) is defined as the flesh-colored area (HI), and the area that does not satisfy formula (5) (H2).

[0083] 10 <Saturation (S) <175,

Hue '(H) = Hue) + 60 (0≤Hue) (when 300)),

Hue '(H) = Hue (H)-300 (when 300 ≤ Hue (H) <360),

Luminance (Y) = InR X 0.30 + InG X 0.59 + InB X 0.11 (A)

Hue, (H) Z Luminance (Y) <3.0 X (Saturation (S) Z255) + 0.7 (5)

Therefore, the number of divided areas of the captured image data is 4 × 7 = 28. In equation (5) V and brightness (V) can also be used.

[0084] When the two-dimensional histogram is created, a first occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step T12). The first occupancy rate calculation process ends. Table 1 shows the first occupancy ratio in each divided area, where Rij is the first occupancy ratio calculated in the divided area that also has the combined power of the lightness area vi and the hue area Hj.

[0085] [Table 1]

[1st occupancy]

Next, an index calculation method will be described.

[0087] Table 2 shows the first coefficient necessary for calculating the accuracy (X for each divided area) that quantitatively indicates the accuracy of strobe shooting, that is, the brightness state of the face area during strobe shooting. The coefficient of each divided area shown in Table 2 is a weighting coefficient by which the first occupancy Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the photographing conditions.

[0088] [Table 2]

[First factor]

FIG. 13 shows the brightness (v) —hue (H) plane. According to Table 2, a positive (+) coefficient is used for the first occupancy calculated from the region (rl) distributed in the high-brightness skin color hue region in Fig. 13, and other hues are used. A negative (-) coefficient is used for the first occupancy calculated from a certain blue hue region (r2). Figure 15 shows the first coefficient in the flesh-color area (HI) and the first coefficient in the other areas (green hue area (H3)) continuously changing over the entire brightness. It is shown as a curve (coefficient curve). According to Table 2 and Figure 15, in the high brightness (V = 17 0 224) region, the sign of the first coefficient in the skin color region (HI) is positive (+), and the other regions (e.g., green hue) In region (H3)), the sign of the first coefficient is negative (-), and the sign of both is different.

[0090] If the first coefficient in the lightness region vi and the hue region Hj is Cij, the sum of the Hk regions for calculating the index α is defined as in equation (6).

[0091] [Equation 1]

Sum of / Λ regions: Rik χ Cik (6)

Accordingly, the sum of the H1 and H4 regions is expressed by the following formulas (6-1) to (6-4).

[0093] Sum of HI regions = R11X (-44.0) + R21X (-16.0) + (omitted) ... + R71X (-11.3) (6-1)

Sum of H2 region = R12X0.0 + R22X8.6 + (omitted) ... + R72X (-11.1) (6-2) Sum of H3 region = R13X0.0 + R23X (-6.3) + (omitted) ... + R73X ( -10.0) (6-3) Sum of H4 region = R14 X 0.0 + R24 X (-1.8) + (Omitted) ... + R74 X (-14.6) (6-4) The index α is the formula (6-1) Using the sum of Η1 Η4 regions shown in ~ (6-4), it is defined as equation (7).

[0094] Index α = Sum of Η1 area + Sum of Η2 area + Sum of Η3 area + Sum of Η4 area +4.424 (7) Table 3 shows the accuracy of backlighting, that is, brightness of face area during backlighting The second coefficient necessary to calculate the index β that quantitatively indicates the state is shown for each divided region. The coefficient of each divided area shown in Table 3 is a weighting coefficient by which the first occupancy ratio Rij of each divided area shown in Table 1 is multiplied, and is set in advance according to the shooting conditions.

[0095] [Table 3]

[Second factor]

FIG. 14 shows the brightness (v) —hue (H) plane. According to Table 3, in Fig. 14, the skin color hue area Area (r4) force distributed in the middle brightness of the area The negative (-) coefficient is used for the calculated occupancy, and the occupancy calculated from the low brightness (shadow) area (r3) of the flesh hue area Is a positive (+) coefficient. Fig. 16 shows the second coefficient in the flesh color region (HI) as a curve (coefficient curve) that changes continuously over the entire brightness. According to Table 3 and Fig. 16, the sign of the second coefficient in the lightness value range of 85 to 169 (v4) in the flesh tone hue region is negative (-) and the lightness value is 26 to 84 (v2, The sign of the second coefficient in the low lightness (shadow) region of v3) is positive (+), which indicates that the sign of the coefficient in both regions is different.

[0097] If the second coefficient in the lightness region vi and the hue region Hj is Dij, the sum of the Hk regions for calculating the index β is defined as in equation (8).

[0098] [Equation 2]

Hk region sum = ^ Rik x Dik (8)

[0099] Therefore, the sum of the H1 to H4 regions is expressed by the following equations (8-1) to (8-4).

[0100] Sum of HI area = R11 X (-27.0) + R21 X 4.5 + (omitted)… + R71 X (-24.0) (8-1)

H2 region sum = R12 X 0.0 + R22 X 4.7 + (omitted)… + R72 X (-8.5) (8-2) H3 region sum = R13 X 0.0 + R23 X 0.0 + (omitted) ... + R73 X 0.0 (8-3)

Sum of H4 region = R14 X 0.0 + R24 X (-5.1) + (omitted)… + R74 X 7.2 (8-4) The index j8 is H1 expressed by the equations (8-1) to (8-4). Using the sum of the ~ H4 region, it is defined as in equation (9).

[0101] Indicator j8 = Sum of H1 region + Sum of H2 region + Sum of H3 region + Sum of H4 region + 1.554 (9)

Since the index α and the index β are calculated based on the brightness and hue distribution amount of the captured image data, they are effective for determining a captured scene when the captured image data is a color image.

Next, the second occupancy rate calculation process executed to calculate the index γ will be described in detail with reference to the flowchart of FIG.

[0103] First, the RGB values of the photographed image data are converted into the HSV color system (step T20). Next, the captured image data is divided into regions where the combined power of the distance from the outer edge of the captured image screen and the brightness is determined, and the cumulative number of pixels is calculated for each divided region to obtain a two-dimensional histogram. Is created (step T21). Hereinafter, the area division of the captured image data will be described in detail.

18A to 18D show four areas nl to n4 divided according to the distance from the outer edge of the screen of the captured image data. The area nl shown in FIG. 18 (a) is the outer frame, the area n2 shown in FIG. 18 (b) is the area inside the outer frame, and the area n3 shown in FIG. 18 (c) is the area n2. A further inner area, an area n4 shown in FIG. 18 (d) is an area at the center of the captured image screen. In addition, the lightness is divided into seven regions from vl to v7 as described above. Therefore, when the captured image data is divided into regions that are a combination of the distance from the outer edge of the captured image screen and the brightness, the number of divided regions is 4 × 7 = 28.

[0105] When the two-dimensional histogram is created, a second occupancy ratio indicating the ratio of the cumulative number of pixels calculated for each divided region to the total number of pixels (the entire captured image) is calculated (step T22). The second occupancy rate calculation process ends. Assuming that Qij is the second occupancy calculated in the divided area that also has the combined power of the brightness area vi and the screen area nj, the second occupancy ratio in each divided area is expressed as shown in Table 4.

[0106] [Table 4]

[Second occupancy]

Next, a method for calculating the index γ calculated using the second occupation ratio will be described.

Table 5 shows the third coefficient necessary for calculating the index γ for each divided region. The coefficient of each divided area shown in Table 5 is a weighting coefficient by which the second occupancy Qij of each divided area shown in Table 4 is multiplied, and is set in advance according to the photographing conditions.

[0109] [Table 5] [Third coefficient]

[0110] Fig. 19 shows the third coefficient in the screen areas nl to n4 as a curve (coefficient curve) that continuously changes over the entire brightness.

[0111] If the third coefficient in the brightness area vi and screen area nj is Eij, the sum of the nk area (screen area nk) for calculating the index γ is defined as in equation (10).

[0112] [Equation 3]

Sum of nk regions = Y Qik x Eik (1 0)

[0113] Accordingly, the sum of the nl to n4 regions is expressed by the following equations (10-1) to (10-4).

[0114] Sum of nl region = Q11 X 40.1 + Q21 X 37.0+ (omitted) ... + Q71 X 22.0 (10-1)

n2 region sum = Q12 X (-14.8) + Q22 X (-10.5) + (omitted)… + Q72 X 0.0 (10-2) n3 region sum = Q13 X 24.6 + Q23 X 12.1 + (omitted)… + Q73 X 10.1 (10-3) Sum of n4 region = Q 14 X 1.5 + Q24 X (-32.9) + (Omitted) ... + Q 74 X (-52.2) (10-4) Using the sum of the nl to n4 regions shown in 10-1) to (; 10-4), it is defined as in equation (11).

[0115] Index γ = Sum of nl regions + Sum of η2 regions + Sum of η3 regions + Sum of η4 regions 12.6201 (11)

The index γ is calculated based on the compositional characteristics (distance from the outer edge of the screen of the captured image data) based on the brightness distribution position of the captured image data. It is also effective for discrimination.

Next, with reference to the flowchart in FIG. 20, the bias amount calculation process (step Τ2 in FIG. 10) will be described.

[0117] First, the luminance Y (brightness) of each pixel is calculated from the RGB (Red, Green, Blue) values of the captured image data using Equation (A), and the standard deviation (xl) of the luminance is calculated. (Step T23). The standard deviation (xl) of brightness is defined as in equation (12). [0118] [Equation 4]

: Pixel luminance value-average luminance

Number of body pixels

[0119] In equation (12), the pixel luminance value is the luminance of each pixel of the captured image data, and the average luminance value is the average value of the luminance of the captured image data. The total number of pixels is the number of pixels of the entire captured image data.

Next, as shown in Expression (13), a luminance difference value (x2) is calculated (step T24).

[0121] Luminance difference value (x2) = (Maximum luminance value, Average luminance value) Z255 (13)

In equation (13), the maximum luminance value is the maximum luminance value of the captured image data.

[0122] Next, the average luminance value (x3) of the flesh color region in the center of the screen of the captured image data is calculated (step T25), and further the average luminance value (χ4) in the center of the screen is calculated (step S25). (Τ26). Here, the center of the screen is, for example, an area composed of an area η3 and an area η4 in FIG.

[0123] Next, the flesh color luminance distribution value (χ5) is calculated (step Τ27), and this deviation amount calculation processing ends. When the maximum brightness value of the skin color area of the captured image data is Yskinjnax, the minimum brightness value of the skin color area is Yskin_min, and the average brightness value of the skin color area is Yskin_ave, the skin color brightness distribution value (x5) is defined as in equation (14) Is done.

[0124] x5 = (Yskin— max― Yskin— mm) / 2― Yskin—ave (14)

Let x6 be the average luminance value of the skin color area in the center of the screen of the captured image data. Here, the center of the screen is, for example, an area composed of the area n2, the area n3, and the area n4 in FIG. At this time, index 1 shown in FIGS. 7 and 8 is defined as in equation (15) using index a, index γ, χ6, and index 2 is defined using index j8, index γ, χ6. It is defined as (16).

[0125] Indicator 1 = 0.46 X indicator α +0.61 X indicator γ +0.01 Χ χ6— 0.79 (15)

Index 2 = 0.58 Index | 8 +0.18 Index 7 + (-0.03) 6 + 3.34 (16) Here, the weighting factor multiplied by each index in Equation (15) and Equation (16) depends on the shooting conditions. Set in advance.

[0126] The index 3 shown in FIGS. 7 and 8 includes the deviation amounts (xl) to (x5) calculated in the deviation amount calculation process. It is obtained by multiplying a preset fourth coefficient according to the shooting conditions. Table 6 shows the fourth coefficient, which is a weighting coefficient by which each deviation is multiplied.

[0127] [Table 6]

[Fourth coefficient]

[0128] Index 3 is defined as shown in equation (17).

[0129] Indicator 3 = 1 0.02+ 2 1.13+ 3 0.06+ 4 (-0.01) + 5 0.03— 6.49 (17) This indicator 3 is a luminance histogram distribution information that consists only of the compositional features of the screen of the captured image data. In particular, it is effective for discriminating between strobe (flash proximity) shooting scenes and under shooting scenes.

[0130] As described above, according to the imaging apparatus 100 of the present embodiment, reproduction auxiliary data for generating viewing image reference data is generated on the imaging apparatus side, and the reproduction auxiliary data is used as characteristics of the imaging apparatus. By attaching to the dependent scene reference raw data and recording on the recording medium, when the data recorded on the recording medium is output on the output medium, the viewing image reference data is generated using the reproduction auxiliary data. As a result, information loss of captured image information can be suppressed, and high-quality appreciation image reference data higher than image data obtained by the imaging device can be obtained.

Next, in order to obtain a more preferable image at the output destination of the image data recorded on the recording medium of the imaging device, the imaging information data generation unit 13 is added to the configuration of the imaging device 100 in FIG. The imaging device 101 will be described. FIG. 21 shows a configuration of an imaging apparatus 101 as a modification of the imaging apparatus 100. In the imaging device 101 of FIG. 21, the same components as those of the imaging device 100 of FIG.

[0131] The photographing information data generation unit 13 generates photographing information data that is a photographing condition setting at the time of photographing. This shooting information data includes, for example, information directly related to the camera type (model) such as camera name and code number, exposure time, shutter speed, aperture value (F number), ISO sensitivity, brightness value, subject distance range, Light source, strobe flash, subject area, Information about the type of the subject, the image balance, the zoom magnification, the subject composition, the shooting scene, the amount of reflected light from the strobe light source, the shooting saturation, and the like.

FIG. 22 shows the data structure of a data file recorded on the recording medium of the storage device 9 of the imaging apparatus 101. The imaging device characteristic correction information, the appreciation image reference data restoration information and the processing process reproduction information generated by the reproduction auxiliary data generation unit 12 and the shooting information data generated by the shooting information data generation unit 13 are the header information processing unit 8 As a result, the attached data file is created by being attached to the file header of the scene reference raw data as auxiliary reproduction data, and is recorded in the recording medium of the attached data file storage device 9.

Next, the operation in the imaging apparatus 101 will be described.

With reference to the flowchart of FIG. 23, the image data recording process executed in the imaging apparatus 101 will be described.

[0135] When the shooting mode is designated by the operation unit 14, preliminary imaging is performed (step S20), and an image obtained by preliminary imaging (hereinafter referred to as preliminary imaging image) is displayed on the display unit 15. The viewing image reference data is formed (step S21), and the formed viewing image reference data is displayed on the display unit 15 (step S22). In step S20, preliminary imaging may be performed when the shutter button is pressed halfway by the operation unit 14.

[0136] Next, in the processing process reproduction information generating unit 123, the camera shake level and the indices 1 to 3 as a result of the scene discrimination process are calculated. The photographing condition validity value is calculated from the calculation result, and the validity of the photographing condition is calculated. Is determined (step S23). Then, the shooting condition is adjusted so that the value of the shooting condition validity is increased (step S 24). The processing of steps S20 to S24 is repeated until the actual shooting is instructed by pressing the shutter button of the operation unit 14, and history information of shooting conditions up to a predetermined time before the main shooting is generated as processing process reproduction information. It is.

[0137] When the shutter button on the operation unit 14 is pressed and an actual shooting is instructed (step S25; YES), the image pickup signal obtained from the CCD3 is converted into a digital signal by the AZD change 5 to generate a scene. At the same time as the reference raw data is generated (step S26), the reproduction assistance data The generation unit 12 generates imaging device characteristic correction information, appreciation image reference data restoration information, and processing process reproduction information as reproduction auxiliary data (step S27), and the shooting information data generation unit 13 generates shooting information data. (Step S28). The processing process reproduction information generated in step S27 is obtained by adding the information at the time of main photographing to the history information before main photographing generated by the processing of steps S20 to S24. Note that a step of creating appreciation image reference data for recording on a recording medium may be added after the main photographing.

Next, in the header information processing unit 8, the reproduction auxiliary data generated in step S 27 and the reproduction auxiliary data generated in step S 28 are added to the file header of the scene reference raw data generated in step S 26. The captured information data is attached as tag information (step S29), and an attached data file (see FIG. 22) is created (step S30). The attached data file is recorded and saved on the recording medium of the storage device 9 (step S31), and the image data recording process is completed.

As described above, according to the imaging device 101 shown in FIG. 21, in addition to the reproduction assistance data, the shooting information data that is the shooting condition setting at the time of shooting is attached to the file header of the scene reference raw data, and the recording medium By recording the image, it is possible to generate appreciation image reference data according to the shooting situation when the data recorded on the recording medium is output on the output medium.

[0140] The recording medium on which the attached data file is recorded is taken out from the imaging apparatus main body and mounted on an external apparatus such as an image processing apparatus or an image recording apparatus. In the external apparatus, viewing image formation on the output medium is performed. Therefore, the image processing to be optimized is performed to generate the viewing image reference data.

[0141] Next, an image recording apparatus that performs image processing that optimizes image data recorded on a recording medium for the formation of a viewing image on an output medium will be described.

FIG. 24 shows an external configuration of the image recording apparatus 201 according to the embodiment of the present invention.

[0142] The image recording apparatus 201 is provided with a magazine loading unit 203 on one side surface of a main body 202. The main body 202 is exposed to an exposure processing unit 204 that exposes silver salt photographic paper as an output medium. The print creation unit 205 is provided to create a print by developing and drying the silver halide photographic paper. It has been. The print created by the print creation unit 205 is discharged to a tray 206 provided on the other side of the main body 202. Further, inside the main body 202, a control unit 207 that controls each unit constituting the image recording apparatus 201 is provided.

In addition, on the upper part of the main body 202, a display unit 208, a film scanner unit 209 that is a transparent document reading device, a reflective document input device 210, and an operation unit 211 are arranged. Further, the main body 202 is provided with an image reading unit 214 that can read image data recorded on various recording media, and an image writing unit 215 that writes image data on various recording media.

A photographic photosensitive material is used as an original read from the film scanner unit 209 or the reflective original input device 210. Examples of the photographic material include a color negative film, a color reversal film, a black and white negative film, and a black and white reversal film. Frame image information captured by an analog camera is recorded. The film scanner unit 209 converts the frame image information recorded on the photographic photosensitive material into digital image data to obtain frame image data. When the photographic photosensitive material is color paper that is silver salt photographic paper, the reflective original input device 210 converts the frame image information recorded on the silver salt photographic paper into frame image data by a flatbed scanner.

[0145] The image reading unit 214 includes a PC card adapter 214a and a floppy (registered trademark) disk adapter 214b, into which a PC card 213a and a floppy (registered trademark) disk 213b can be respectively inserted. The PC card 213a has, for example, a memory in which a plurality of frame image data is stored after being captured by a digital camera. In the floppy (registered trademark) disk 213b, for example, a plurality of frame image data captured by a digital camera is recorded. Recording media on which frame image data is recorded in addition to PC card 213a and floppy disk 213b include, for example, multimedia card (registered trademark), memory stick (registered trademark), MD data, CD-ROM, and the like. Can be mentioned.

[0146] The image writing unit 215 includes a floppy (registered trademark) disk adapter 215a, an MO adapter 215b, and an optical disk adapter 215c. The floppy disk (registered trademark) disks 216a, M0216b, and the optical disk 216c, respectively. Can be inserted. Examples of the optical disk 216c include CD-R and DVD-R.

[0147] In Fig. 24, the operation unit 211, the display unit 208, the film scanner unit 209, and the reflection original input The force device 210 and the image reading unit 214 are integrally provided in the main body 202. Force One or more of these may be provided separately.

[0148] In the image recording apparatus 201 shown in Fig. 24, an example is shown in which a photosensitive material is exposed and developed to create a print, but the print creation method is not limited to this, and for example, an inkjet A method such as a method, an electrophotographic method, a heat sensitive method, or a sublimation method may be used.

FIG. 25 shows the internal configuration of the image recording apparatus 201. As shown in FIG. 25, the image recording apparatus 201 includes a control unit 207, an exposure processing unit 204, a print creation unit 205, a film scanner unit 209, a reflection original input device 210, an image reading unit 214, and an image writing unit 215. , Data storage means 271, template storage means 272, operation section 211, and display section 208.

[0149] The control unit 207 is constituted by a microcomputer, and various control programs stored in a storage unit (not shown) such as a ROM (Read Only Memory) and a CPU (Central Process Unit) (not shown) The operation of each unit constituting the image recording apparatus 201 is controlled in cooperation with the.

In addition, the control unit 207 has an image processing unit 270 and is read from the film scanner unit 209 or the reflection original input device 210 based on an input signal from the information input unit 12 of the operation unit 211. Image data read from the image reading unit 214 or image data input from an external device via a communication means (not shown) to form image information for exposure to form exposure image information. Output to 204. The image processing unit 270 performs a conversion process according to the output form on the image processed image data, and outputs it to a designated output destination. The output destination of the image processing unit 270 includes a display unit 208, an image writing unit 215, a communication unit, and the like.

The exposure processing unit 204 exposes an image on the photosensitive material, and outputs the photosensitive material to the print creating unit 205. The print creating unit 205 develops the exposed photosensitive material and dries it to create prints Pl, P2, and P3. The print P1 is a service size, a no-vision size, a panorama size, etc., the print P2 is an A4 size print, and the print P3 is a business card size print. [0152] The film scanner unit 209 reads a frame image recorded on a transparent original such as a developed negative film or reversal film imaged by an analog camera, and obtains a digital image signal of the frame image. The reflection original input device 210 reads an image on the print P (photo print, document, various printed materials) by a flat bed scanner, and acquires a digital image signal.

The operation unit 211 is provided with information input means 212. For example, the information input unit 212 includes a touch panel or the like, and outputs an operation signal of the information input unit 212 to the control unit 207. The operation unit 211 may be configured to include a keyboard and a mouse.

The image reading unit 214 reads the frame image information recorded on the PC card 213a or the floppy (registered trademark) disk 213b and transfers the frame image information to the control unit 207. The image reading unit 214 includes, as the image transfer means 230, a PC card adapter 214a, a floppy (registered trademark) disk adapter 214b, and the like. The image reading unit 14 reads the frame image information recorded on the PC card 213a inserted into the PC card adapter 214a and the floppy disk 213b inserted into the floppy disk adapter 214b. Transfer to the control unit 207. For example, a PC card reader, a PC card slot, or the like is used as the PC card adapter 214a.

The image writing unit 215 includes a floppy (registered trademark) disk adapter 215a, an MO adapter 215b, and an optical disk adapter 215c as the image transport unit 231. In accordance with a write signal input from the control unit 207, the image writing unit 215 includes a floppy disk 216a inserted into the floppy disk adapter 215a and an MO 216b inserted into the MO adapter 215b. The image data generated by the image processing method according to the present invention is written to the optical disk 216c inserted into the optical disk adapter 215c.

[0156] The data storage means 271 stores and sequentially stores image information and corresponding order information (information about how many prints are to be created from which frame images, print size information, etc.)

[0157] The template storage means 272 corresponds to the sample identification information Dl, D2, D3. Image data (data indicating a background image, an illustration image, etc.) and at least one template data for setting a synthesis area with the sample image data. Here, when a predetermined template is selected from a plurality of templates stored in advance in the template storage means 272 by an operator's operation (the operator's operation is based on an instruction from the client), the control unit 207 When the image identification information Dl, D2, and D3 are specified by the operator's operation (this operator's operation is based on the client's instruction), the image information and the selected template are combined. The sample image data is selected based on the sample identification information Dl, D2, and D3, and the selected sample image data is combined with the image data and Z or character data ordered by the client. Creates a print based on the desired sample image data. This template synthesis is performed by the well-known Chromaki method.

Note that the sample identification information is not limited to the three types of sample identification information Dl, D2, and D3, but may be more or less than three types.

[0159] In addition, sample identification information Dl, D2, and D3 for specifying a print sample is a force that is configured to be input by the operation unit 21 1 force Sample identification information Dl, D2, and D3 are printed samples or Since it is recorded on the order sheet, it can be read by reading means such as OCR (Optical Character Reader). Or, the operator can input from the keyboard.

[0160] In this way, sample image data is recorded corresponding to sample identification information D1 for specifying a print sample, sample identification information D1 for specifying a print sample is input, and this sample identification information is input. Select sample image data based on D1, and combine the selected sample image data with the image data and Z or character data based on the order to create prints based on the specified samples. Users can actually order samples for printing and can meet the diverse requirements of a wide range of users.

[0161] Further, the first sample identification information D2 designating the first sample and the image data of the first sample are stored, and the second sample identification information D3 designating the second sample and the first sample identification data D3 are stored. The image data of two samples is stored, the sample image data selected based on the designated first and second sample identification information D2, D3, the image data based on the order, and the Z or character data Since a print based on the specified sample is created, a wider variety of images can be synthesized, and a print that meets a wider variety of user requirements can be created.

[0162] The display unit 208 includes a display such as a CRT or LCD, and performs display processing in accordance with a display control signal input from the control unit 207.

[0163] The image processing unit 270 of the control unit 207 uses a communication means (not shown) via another computer in the facility where the image recording apparatus 201 is installed or a communication network such as the Internet. It is also possible to receive image data representing captured images and work instructions such as printing from a distant computer, and to perform image processing and create prints remotely.

[0164] Also, the image processing unit 270 uses communication means (not shown) to send image data representing the captured image after the image processing of the present invention and the accompanying order information to another combination in the facility. Utah can also be sent to distant computers via the Internet.

As described above, the image recording apparatus 201 inputs the image information of the various recording media and the image information obtained by dividing and metering the image original, and the image information of the input image taken from the input means into “ An image that is processed to obtain an image that gives a favorable impression when observing the image on the output medium by acquiring or estimating information such as “the size of the output image” and “the size of the main subject in the output image” Image processing means, image output means for displaying processed images, printing output, writing to recording media, and another computer in the facility via a communication line, image data to a remote computer via the Internet, etc. And communication means for transmitting the attached order information.

FIG. 26 shows an internal configuration when the image recording apparatus 201 is divided into an image processing apparatus 301 and an output unit 302 that outputs image data processed by the image processing apparatus 301.

As shown in FIG. 26, the image processing device 301 includes an input unit 303, a header information analysis unit 304, an imaging device characteristic correction processing unit 305, an appreciation image reference data restoration condition generation unit 306, a processing program. The process reproduction unit 307 and the optimization processing unit 308 are configured.

[0168] The input unit 303 includes the image reading unit 214 of Fig. 25 and includes a mounting unit for mounting a recording medium. When the recording medium is loaded in the loading unit, the input unit 303 reads the data file recorded in the recording medium and outputs the data file to the header information analysis unit 304. In the present embodiment, the input unit 303 is described as reading a data file from the attached recording medium. However, the input unit 303 includes a wired or wireless communication unit, and inputs the data file via the communication unit. Anyway!

[0169] The header information analysis unit 304 analyzes the data file input from the input unit 303, and converts the data file into scene reference raw data, reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration). Information, processing process reproduction information) and shooting information data, and output the scene reference raw data to the scene reference image data generation unit 311 in the imaging device characteristic correction processing unit 305, and correct the imaging device characteristic correction information to the device characteristic correction. Output to the processing unit 309, output the viewing image reference data restoration information to the viewing image reference data restoration condition generation unit 306, and output the processing process reproduction information to the processing process reproduction unit 307 to optimize the shooting information data The information is output to the photographing information data processing unit 313 in the unit 308.

The imaging device characteristic correction processing unit 305 includes a device characteristic correction processing unit 309, a processing condition table 310, a scene reference image data generation unit 311, and a temporary storage memory 312.

The device characteristic correction processing unit 309 determines scene generation image data generation conditions based on the imaging device characteristic correction information input from the header information analysis unit 304 and the processing condition table 310. The processing condition table 310 is a table that stores processing conditions for generating scene reference image data for each characteristic of the imaging apparatus.

[0172] The scene reference image data generation unit 311 performs imaging device characteristic correction processing on the scene reference raw data input from the header information analysis unit 304 in accordance with the generation conditions determined by the device characteristic correction processing unit 309. Depending on the characteristics of the imaging device, generate standardized scene reference image data and output it to the temporary storage memory 312. Specifically, in the imaging device characteristic correction process, the signal intensity of each color channel based on at least the spectral sensitivity unique to the imaging device of the imaging device that generated the scene reference raw data, for example, RIMM RGB, E RIMM RGB, scRGB, etc. Includes mapping to standard color space. The temporary storage memory 312 temporarily stores the scene reference image data generated by the scene reference image data generation unit 311.

[0174] The appreciation image reference data restoration condition generation unit 306 is based on the appreciation image reference data restoration information input from the header information analysis unit 304 !, and is restored to restore the appreciation image reference data in the imaging device. Determine the conditions.

[0175] Based on the processing process reproduction information input from the header information analysis unit 304, the processing process reproduction unit 307 sets reproduction conditions for reproducing the generation process of the appreciation image reference data in the imaging device. decide.

[0176] The optimization processing unit 308 includes the photographing information data processing unit 313, the appreciation image reference data generation unit 3

14, composed of a temporary storage memory 315 and a setting input unit 316.

The shooting information data processing unit 313 determines a generation condition for generating appreciation image reference data corresponding to the shooting condition based on the shooting information data input from the header information analysis unit 304.

[0178] The appreciation image reference data generation unit 314 reads the scene reference image data from the temporary storage memory 312 and generates the appreciation image reference data determined by the shooting information data processing unit 313 and restores the appreciation image reference data. The restoration conditions determined by the condition generation unit 306, the reproduction conditions determined by the processing process reproduction unit 307, and the output destination specified by the setting input unit 316 (storage device 318, output device 317, display unit 208) Based on this information, the scene reference image data is subjected to an optimization process for obtaining an optimal image at the output destination to generate viewing image reference data, which is temporarily stored together with the operation information of the setting input unit 316. Output to memory 315. Optimization processing includes, for example, compression to the color gamut of the output destination, gradation compression from 16 bits to 8 bits, reduction of the number of output pixels, processing to support output characteristics (LUT) of output devices and display devices, etc. Is included. Furthermore, image processing such as noise suppression, sharpening, color balance adjustment, saturation adjustment, and dodging processing is included.

[0179] The temporary storage memory 315 temporarily stores the viewing image reference data generated by the viewing image reference data generation unit 314, and the output destination (storage device 318, output device) set by the setting input unit 316. 317 and display 208).

[0180] The setting input unit 316 refers to the appreciation image generated by the appreciation image reference data generation unit 314. This is an input device for designating the data output destination, and corresponds to the operation unit 211 in FIG.

Note that, among the components of the image processing apparatus 301, components other than the input unit 303 and the setting input unit 316 are included in the image processing unit 270 shown in FIG.

[0182] The output unit 302 includes a display unit 208, an output device 317 corresponding to the exposure processing unit 204 and the print creation unit 205 in FIG. 25, and a storage device 318 corresponding to the image writing unit 215 in FIG. C

Next, the operation of the image recording apparatus 201 will be described.

[0184] With reference to FIG. 27, the image processing executed in the image recording apparatus 201 will be described.

[0185] When data is input to the input unit 303 (that is, when a recording medium is mounted to the mounting unit) (step S40), the data file recorded on the recording medium is read and the header information analysis unit In 304, the contents of the data file are analyzed (step S41), and are divided into scene reference raw data, reproduction auxiliary data (imaging device characteristic correction information, appreciation image reference data restoration information, processing process reproduction information), and shooting information data. .

[0186] The shooting information data processing unit 313 determines generation conditions for generating viewing image reference data according to shooting conditions based on the shooting information data (step S42), and generates viewing image reference data restoration condition generation. Based on the appreciation image reference data restoration information, the unit 306 determines restoration conditions for restoring the appreciation image reference data in the imaging device (step S44). The processing process reproduction unit 307 reproduces the processing process reproduction data. Based on the information, the reproduction condition for reproducing the generation process of the viewing image reference data in the imaging device is determined (step S45).

[0187] In the device characteristic correction processing unit 309, the generation condition of the scene reference image data is determined by referring to the imaging apparatus characteristic correction information and the processing condition table 310, and the scene reference raw data is determined according to the determined generation condition. Imaging device characteristic correction processing is performed (step S43), and scene reference image data is generated (step S46).

[0188] Next, in step S46, based on the various image processing conditions (appearance image reference data generation conditions, restoration conditions, reproduction conditions) determined in steps S42, S44, and S45. The optimized scene reference image data is subjected to optimization processing (step S47), and appreciation image reference data is generated (step S48).

[0189] The viewing image reference data is subjected to processing specific to the output destination set in the setting input unit 316 (operation unit 211) (step S49), and the output destination device is checked. The image is output from the chair (step S50), and this image processing ends.

In the flowchart of FIG. 27, the image processing for the data file including the shooting information data as shown in FIG. 22 has been described. However, for the data file not including the shooting information data as shown in FIG. In the image processing, step S42 in FIG. 27 is not performed.

[0191] As described above, according to the image recording device 201 of the present embodiment, the optimum is based on the reproduction information data and the shooting information data attached to the file header of the scene reference raw data output from the imaging device. By generating converted viewing image reference data, it is possible to obtain high-quality recognition image reference data that is higher than image data obtained by an imaging device that does not involve information loss of captured image information.

[0192] It should be noted that even if the reproduction assistance data does not contain processing process reproduction information, it is possible to obtain sufficiently high quality viewing image reference data. It is possible to obtain appreciation image reference data.

Claims

The scope of the claims

[1] Scene reference raw data generation means for generating scene reference raw data depending on characteristics of the imaging device by imaging;

Reproduction auxiliary data generating means for generating reproduction auxiliary data when generating image reference data by performing image processing optimized for image formation on an output medium with respect to the scene reference raw data;

Recording control means for attaching the reproduction auxiliary data to the scene reference raw data and recording it on a recording medium;

An imaging apparatus comprising:

[2] Image processing means for applying image processing to the scene reference raw data using the auxiliary reproduction data and creating the award image reference data;

Based on appreciation image reference data created by the image processing means! / Image forming means for forming a recognition image on the output medium;

The imaging apparatus according to claim 1, further comprising:

[3] The image processing means creates scene reference data by standardizing the scene reference raw data, performs image processing on the scene reference data using the viewing image reference data restoration information, and performs the viewing. Generate image reference data,

The imaging apparatus according to claim 2, wherein the reproduction assistance data includes the appreciation image reference data restoration information.

[4] Preliminary shooting is performed before the actual shooting! 撮影, equipped with shooting condition adjustment means to adjust shooting conditions according to the result of preliminary shooting,

3. The imaging apparatus according to claim 2, wherein the reproduction auxiliary data includes processing process reproduction information indicating a history of imaging conditions adjusted by the imaging condition adjusting unit before actual imaging.

[5] The imaging apparatus according to claim 4, wherein the processing process reproduction information includes an index value for judging the validity of the imaging condition.

6. The imaging device according to claim 5, wherein the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions at the time of shooting.

7. The imaging apparatus according to claim 6, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

[8] A shooting information data generation means for generating shooting information data indicating shooting condition settings at the time of shooting is provided.

The imaging apparatus according to any one of claims 1 to 7, wherein the recording control unit records the shooting information data on a recording medium with the scene reference raw data attached thereto. .

[9] Scene reference raw data depending on the characteristics of the imaging device, and image processing that optimizes the scene reference raw data to form an appreciation image on the output medium, generates appreciation image reference data Input means for inputting reproduction auxiliary data when performing,

Appreciation image reference data generation means for generating appreciation image reference data by performing an optimization process on the input scene reference raw data based on the input reproduction auxiliary data;

An image processing apparatus comprising:

[10] The reproduction assistance data includes appreciation image reference data restoration information for restoring appreciation image reference data on the imaging device when generating appreciation image reference data on the output medium. The image processing device according to claim 9.

[11] The reproduction assistance data includes processing process reproduction information for reproducing the generation process of the appreciation image reference data on the imaging device when generating the appreciation image reference data on the output medium. The image processing apparatus according to claim 9, wherein:

12. The image processing apparatus according to claim 11, wherein the processing process reproduction information includes an index value for determining the validity of the imaging condition in the imaging apparatus.

13. The index value according to claim 12, wherein the index value includes a value that specifies at least one of user characteristics, light source conditions, and exposure conditions at the time of shooting by the imaging device. Image processing device.

14. The image processing apparatus according to claim 13, wherein the light source condition and the exposure condition are determination results of a shooting scene determination process at the time of shooting.

[15] The input means inputs shooting condition data indicating shooting condition settings at the time of shooting, The appreciation image reference data generating means generates an appreciation image reference data by performing an optimization process on the input scene reference raw data based on the input reproduction assistance data and the shooting condition data. 15. The image processing device according to any one of claims 9 to 14, wherein:

The image processing device according to any one of claims 9 to 15,

Image forming means for forming an appreciation image on an output medium using appreciation image reference data generated by the appreciation image reference data generation means of the image processing device;

An image recording apparatus comprising: