JP2008113445A - Automatic adjustment of image quality in accordance with brightness of subject - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for appropriately performing image quality adjustment processing in accordance with brightness of a subject. <P>SOLUTION: Image quality adjustment processing is performed using a degree of brightness of an object obtained from image generation history information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image adjustment technique for adjusting the image quality of image data.

  The image quality of image data generated by a digital still camera (DSC), a digital video camera (DVC), or the like can be arbitrarily adjusted by using an image retouching application on a personal computer. An image retouching application generally has an image adjustment function that automatically adjusts the image quality of image data. By using this image adjustment function, the image quality of an image output from an output device can be improved. Can do. As an image output device, for example, a CRT, LCD, printer, projector, television receiver, and the like are known.

  Also, a printer driver that controls the operation of a printer, which is one of the output devices, has a function of automatically adjusting the image quality. Image quality can be improved (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 11-8768

  However, in the automatic image quality adjustment function provided by the image retouching application or the printer driver, image quality correction is executed based on image data having general image quality characteristics. On the other hand, since the image data to be subjected to image processing can be generated under various conditions, the image quality may not be improved even if the image quality automatic adjustment function is executed uniformly.

  By the way, various light sources such as natural light sources such as sunlight, artificial light sources such as fluorescent lamps, incandescent lamps, candles, etc. are used depending on the shooting location, shooting time, and user preference. It is done. The spectral distribution of light varies depending on the type of light source. Therefore, even with the same subject, an image with a color shift (color cast) may be obtained depending on the type of light source. For example, when a light source with strong red light such as a candle is used, a reddish image may be obtained. Such color deviation is called color balance. Further, since the adjustment of the color deviation is often performed using a white area close to an achromatic color, it is also called white balance.

  In addition, the operation of the image generation apparatus is also set variously according to the user's preference and the type of subject. For example, when shooting a person in fine weather, there are cases where shooting is performed with a flash even when the subject is bright in order to prevent shadows from appearing in the face.

  As described above, even if image quality correction based on image data having general image quality characteristics is performed on an image in which the operation settings of the image generation apparatus are different or color cast occurs, the image quality of the entire screen is reduced. In some cases, it could not be improved. Such a problem is not limited to images generated by DSC, but is common to images generated by other image generation devices such as DVC.

  The present invention has been made to solve the above problems, and an object thereof is to appropriately and automatically adjust the image quality corresponding to individual image data.

  In order to solve at least a part of the above problems, an output device according to the present invention includes at least image data generated by an image generation device and operation information of the image generation device at the time of image data generation, and the image data Output image using the image generation history information associated with the image generation history information, and the image generation history information includes subject luminance information related to the brightness of the subject at the time of image data generation. An image quality adjusting unit capable of adjusting the image quality of the image data using the brightness level of the subject obtained from the subject brightness information, and an image output for outputting an image according to the image data with the adjusted image quality A section.

  According to the output device of the present invention, the image quality adjustment process using the brightness level of the subject is executed, so that the image quality can be automatically and appropriately adjusted according to the individual image data.

  In the output device, it is preferable that the image quality adjustment unit executes a color balance adjustment process of the image data using a degree of brightness of the subject.

  In this way, color balance adjustment processing using the degree of brightness of the subject is executed, so that appropriate color balance adjustment processing corresponding to individual image data can be executed.

  In each of the above output devices, the image quality adjustment unit may set the intensity of the color balance adjustment process to the brightness in at least a part of the range of the brightness level of the subject in which the brightness level is small. It is preferable to adjust so that the smaller the degree is, the stronger.

  By doing so, strong color balance adjustment can be executed in a range where the brightness level of the subject is small and the brightness level is small.

  In each of the above output devices, the adjustment of the image quality includes (i) a process of determining a color fog amount indicating a degree of color deviation in the image data by analyzing the image data; and (ii) A process for determining a processing amount of the color balance adjustment process based on the size of the color cast amount; and (iii) a process for executing the color balance adjustment process according to the determined processing amount, The adjustment of the intensity of the color balance adjustment process is preferably realized by changing a process parameter that affects the result of at least one of the processes (i) and (ii).

  By doing so, it is possible to execute a color balance adjustment process according to the magnitude of the color deviation. In addition, the intensity of the color balance adjustment process can be easily adjusted.

  In each of the output devices, it is preferable that the image quality adjustment unit determines the amount of color fogging using a pixel value in a region close to an achromatic color in the image data.

  By doing so, the color cast amount is determined based on the region close to the achromatic color, so that it is possible to prevent the dark region and the region far from the achromatic color from affecting the color balance adjustment processing.

  In each of the output devices, it is preferable that the image quality adjustment unit determines the magnitude of the color fogging amount using a pixel value of a region excluding a region having a predetermined hue among regions close to the achromatic color. .

  By doing this, the color cast amount is determined without using the region having the predetermined hue, and therefore it is possible to suppress the region having the predetermined hue from affecting the color balance adjustment process.

  In each of the above output devices, the adjustment of the intensity of the color balance adjustment process is realized by changing a processing parameter indicating a ratio of the amount of the color balance adjustment process to the magnitude of the color fog amount. preferable.

  In this way, the ratio of the amount of color balance adjustment processing to the amount of color cast is determined according to the strength of the color balance adjustment processing. Color balance adjustment processing can be executed with a corresponding processing amount.

  In each of the output devices, it is preferable that the intensity adjustment of the color balance adjustment process is realized by changing a processing parameter indicating a range of a region close to the achromatic color.

  In this way, when the intensity of the color balance adjustment process is strong, the color fog amount is determined using a wider range of data, and therefore data of pixels farther from the achromatic color is used for determining the color fog amount. be able to. Therefore, when the color fog is generated, the color balance adjustment process can be executed with a larger processing amount as the strength is higher.

  In each of the output devices, the image quality adjustment unit uses the light emitted from the auxiliary light source when generating the image data when the image generation history information includes light emission information of the auxiliary light source when the image data is generated. It is preferable to execute the color balance adjustment process using the degree of brightness of the subject when it is determined whether or not the irradiation of light is performed and it is determined that the light irradiation is not performed.

  In this way, since the color balance adjustment process can be executed when the auxiliary light source is not illuminated, the brightness level of the subject when the auxiliary light source is not illuminated. Based on the above, the color balance adjustment process can be appropriately executed.

  In each of the output devices, the image generation history information further includes light emission information of an auxiliary light source at the time of generating the image data, and information on a distance between the subject of the image data and the image generating device at the time of generating the image data. The image quality adjustment unit determines that the light emission by the auxiliary light source has been performed using the light emission information, and the distance from the subject is equal to or less than a predetermined distance threshold value. In addition, it is preferable that an image quality adjustment process suitable for a person image can be executed when the brightness level of the subject is equal to or greater than a predetermined luminance threshold value.

  In this way, whether or not to perform image quality adjustment processing suitable for a human image is automatically determined based on whether or not the auxiliary light source emits light, the distance to the subject, and the brightness of the subject. Since the determination is made, appropriate image quality adjustment can be automatically performed on individual image data.

  In each of the output devices, the image quality adjustment unit uses the photometric luminance information when the image generation history information includes photometric luminance information related to a result of measuring the brightness of the subject when generating the image data. It is preferable to be able to calculate the degree of brightness.

  In this way, the degree of brightness of the subject can be appropriately calculated based on the result of measuring the brightness of the subject.

  In each of the output devices, the image quality adjustment unit, when the image generation history information includes information about an aperture value of the image generation device and information about a shutter speed when the image data is generated, It is preferable that the degree of brightness of the subject can be calculated using the shutter speed.

  In this way, the brightness level of the subject is calculated using the aperture value and the shutter speed, so even if there is no information about the result of measuring the brightness of the subject, the brightness level of the subject can be adjusted. Thus, it is possible to appropriately calculate corresponding to individual image data.

  In each of the output devices, the image quality adjustment unit includes the image generation history information including information regarding an aperture value of the image generation device, information regarding a shutter speed, and information regarding sensitivity of an optical circuit when the image data is generated. In this case, it is preferable that the brightness level of the subject can be calculated using the aperture value, the shutter speed, and the sensitivity.

  In this way, the brightness level of the subject is calculated using the aperture value, the shutter speed, and the sensitivity of the optical circuit, so even if there is no information about the result of measuring the brightness of the subject, The degree of brightness can be calculated appropriately corresponding to each image data.

  The present invention can be realized in various forms, for example, an image output method and an image output device, an image data processing method and an image data processing device, and a function for implementing these methods or devices. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Image output system configuration:
B. Image file structure:
C. Image output device configuration:
D. Image processing in digital still cameras:
E. Image processing in the printer:
F. Example of automatic image quality adjustment processing:
G. Example of color balance adjustment processing:
H. Example of color balance adjustment processing according to the brightness of the subject:
I. Another example of automatic image quality adjustment processing:
J. et al. Configuration of an image output system using an image data processing device:
K. Variation:

A. Image output system configuration:
FIG. 1 is an explanatory diagram showing an example of an image output system to which an output device as an embodiment of the present invention can be applied. The image output system 10 includes a digital still camera 12 as an image generation apparatus that generates an image file, and a printer 20 as an image output apparatus. The image file generated in the digital still camera 12 is sent to the printer 20 through the cable CV or by directly inserting the memory card MC storing the image file into the printer 20. The printer 20 executes image quality adjustment processing of image data based on the read image file and outputs an image. As the output device, in addition to the printer 20, a monitor 14 such as a CRT display or LCD display, a projector, or the like can be used. Hereinafter, description will be given based on the case where the printer 20 including the image quality adjustment unit and the image output unit is used as an output device and the memory card MC is directly inserted into the printer 20.

  FIG. 2 is a block diagram showing a schematic configuration of the digital still camera 12. The digital still camera 12 of this embodiment includes an optical circuit 121 for collecting optical information, an image acquisition circuit 122 for controlling the optical circuit to acquire an image, and a process for processing the acquired digital image. An image processing circuit 123, a flash 130 as an auxiliary light source, a control circuit 124 for controlling each circuit, and a photometry unit 131 for measuring the brightness of the subject are provided. The control circuit 124 includes a memory (not shown). The optical circuit 121 includes a lens 125 that collects optical information, a diaphragm 129 that adjusts the amount of light, and a CCD 128 that converts optical information that has passed through the lens into image data.

  The digital still camera 12 stores the acquired image in the memory card MC. As a storage format of image data in the digital still camera 12, a JPEG format is generally used, but other storage formats such as a TIFF format, a GIF format, a BMP format, and a RAW data format can be used. .

  The digital still camera 12 also includes a selection / determination button 126 for setting various shooting conditions (described later) and a liquid crystal display 127. The liquid crystal display 127 is used when a captured image is previewed or an aperture value or the like is set using the selection / determination button 126.

  When shooting is performed with the digital still camera 12, the image data and the image generation history information are stored in the memory card MC as an image file. The image generation history information can include setting values of parameters such as an aperture value at the time of shooting (at the time of image data generation) (details will be described later).

B. Image file structure:
FIG. 3 is an explanatory diagram conceptually showing an example of the internal configuration of an image file that can be used in this embodiment. The image file GF includes an image data storage area 101 for storing image data GD and an image generation history information storage area 102 for storing image generation history information GI. The image data GD is stored, for example, in the JPEG format, and the image generation history information GI is stored, for example, in the TIFF format (a format in which data and data areas are specified using tags). Note that the terms “file structure” and “data structure” in the present embodiment mean a file or data structure in a state where the file or data is stored in the storage device.

The image generation history information GI is information related to an image when image data is generated (taken) in an image generation apparatus such as the digital still camera 12, and includes the following setting values.
-Subject brightness value.
-Flash (with or without light emission).
-Subject distance.
-Subject distance range.
-Flash intensity.
・ Aperture value.
ISO speed rate (ISO sensitivity).
·White balance.
・ Shooting mode.
·Manufacturer's name.
·Model name.
-Gamma value.

  The image file GF of the present embodiment basically only needs to include the image data storage area 101 and the image generation history information storage area 102 described above, and has a file structure in accordance with an already standardized file format. Can take. The case where the image file GF according to the present embodiment is adapted to the Exif file format will be specifically described below.

  The Exif file has a file structure according to the digital still camera image file format standard (Exif), and its specifications are determined by the Japan Electronics and Information Technology Industries Association (JEITA). The Exif file format includes a JPEG image data storage area for storing image data in the JPEG format and an attached information storage area for storing various information related to the stored JPEG image data, as in the conceptual diagram shown in FIG. And. The JPEG image data storage area corresponds to the image data storage area 101 in FIG. 3, and the attached information storage area corresponds to the image generation history information storage area 102. The attached information storage area stores image generation history information related to JPEG images such as shooting date and time, aperture value, and subject distance.

  FIG. 4 is an explanatory diagram for explaining an example of the data structure of the attached information storage area 103. In the Exif file format, hierarchical tags are used to specify data areas. Each data area can include therein a plurality of lower data areas specified by lower tags. In FIG. 4, an area surrounded by a square represents one data area, and a tag name is written at the upper left. This embodiment includes three data areas whose tag names are APP0, APP1, and APP6. The APP1 data area includes two data areas whose tag names are IFD0 and IFD1. The IFD0 data area includes three data areas whose tag names are PM, Exif, and GPS. The data and data area are stored according to a prescribed address or offset value, and the address or offset value can be retrieved by tag name. On the output device side, data corresponding to desired information can be acquired by designating an address or offset value corresponding to the desired information.

  FIG. 5 is an explanatory diagram for explaining an example of the data structure (data tag name and parameter value) of the Exif data area that can be referred to by tracing the tag names in the order of APP1-IFD0-Exif in FIG. . The Exif data area can include a data area whose tag name is MakerNote, as shown in FIG. 4, and the MakerNote data area can include a larger number of data, but is not shown in FIG. .

  In the Exif data area, as shown in FIG. 5, the subject brightness value, flash, subject distance, subject distance range, flash intensity, aperture value, shutter speed, ISO speed rate, white balance, etc. Stores parameter values related to information. In this embodiment, the subject luminance value is photometric luminance information indicating the photometric result of the brightness of the subject at the time of image data generation, and is used as the subject luminance information of the image data. Also, the flash is the light emission information of the auxiliary light source, the subject distance and the subject distance range are information about the distance between the subject of the image data and the image generating device, the aperture value is information about the aperture value, and the shutter speed is information about the shutter speed. The ISO speed rate is used as information about the sensitivity of the optical circuit.

  The subject luminance value information is information relating to the brightness of the subject at the time of image data generation, and the result measured by the photometry unit provided in the image generation device or the like is stored as a parameter value. The unit of the parameter value is APEX, which can be used as the brightness level of the subject. The subject brightness value is used in combination with other shooting conditions (aperture value, shutter speed, optical circuit sensitivity (ISO speed rate)) expressed in the same APEX unit, so that appropriate values for these shooting conditions can be set and checked. Can be easily performed.

The flash information is information regarding the operation of the flash, and can include four types of information regarding the operation mode and the operation result. The operation mode is set from a plurality of values including the following three values, for example.
1: Forced flash mode.
2: Light emission prohibition mode.
3: Automatic flash mode.

  The operation result is set, for example, from two values, with or without light emission. Using this operation result, it is possible to determine whether or not light irradiation by the auxiliary light source has been performed at the time of image data generation.

  Some image generation apparatuses include a mechanism for detecting light reflected by an object of flash light. When an obstacle such as a flash cover blocks the flash light, or when the flash does not emit light even though the flash is operated, no light is irradiated. Such a case can be identified by the presence or absence of reflected light. The flash information can include information on the presence / absence of such a reflected light detection mechanism and the presence / absence of reflected light detection at the time of image data generation (at the time of photographing). When the reflected light detection mechanism is present and the reflected light detection is absent, it can be determined that the auxiliary light source has not been irradiated even if the above-described operation result is light emission.

  If the image generation apparatus does not include a flash, “no flash” can be set in the flash information. When “no flash” is set, it can be determined that light irradiation by the auxiliary light source has not been performed.

  The subject distance information is information relating to the distance between the image generation device and the subject at the time of image data generation. For example, it is set in units of meters based on distance information set for focusing when generating image data.

  The subject distance range information is information relating to the distance between the image generation device and the subject at the time of image data generation. The parameter value is selected and set from a plurality of distance ranges according to the distance to the subject. As the distance range, for example, macro (0 to 1 m), near view (1 to 3 m), distant view (3 m to) and the like are used.

  The flash intensity information is information relating to the amount of light emitted by the flash at the time of image data generation, and the unit of measurement thereof is, for example, BCPS (Beam Candle Power Seconds).

  The aperture value is information regarding the aperture value at the time of image data generation, and an F value is used as a parameter value. Therefore, the larger the aperture value, the smaller the aperture.

  The shutter speed information is information related to the shutter speed when image data is generated, and its unit is seconds.

  The ISO speed rate information is information relating to the sensitivity of the optical circuit at the time of image data generation, and a corresponding parameter value in the ISO sensitivity that is an index of the sensitivity of the silver salt film is set. The ISO sensitivity is used for setting appropriate image generation conditions (imaging conditions) in combination with parameters relating to image generation such as an aperture value. In an image generation apparatus such as a digital still camera, image generation conditions such as an aperture value can be easily set by using the corresponding ISO sensitivity as an index indicating the sensitivity of the optical circuit.

  The white balance information is information related to a setting mode for white balance (color balance) of image data at the time of image data generation. When the user generates white balance (color balance) when the image generation apparatus generates image data, “manual setting” is set as the parameter value of the white balance information. As a method for setting the white balance by the user, for example, the user designates the type of light source according to the shooting environment or preference, and the image generation device performs image color balance adjustment based on the designated information to generate image data. The method of doing is used. When the user does not set the white balance, “automatic setting” is set as the parameter value of the white balance information.

  These pieces of information are all operation information of the image generation apparatus. These pieces of operation information can be set by the user or automatically set by the image generation apparatus when the image data is generated. Further, in the image generation apparatus, the user sets a shooting mode, and the image generation apparatus automatically sets related parameters (aperture value, ISO sensitivity, etc.) according to the set shooting mode. Some are possible. The shooting mode can be selected from a plurality of predetermined modes, for example, a standard mode, a portrait mode, a landscape mode, a night view mode, and the like. When the standard mode is selected as the shooting mode, parameters related to image data generation are set to standard values.

  Information associated with the image data is appropriately stored in an area other than the Exif data area in FIG. For example, the manufacturer name and model name as information for specifying the image generation apparatus are stored in the data area whose tag name is IFD0.

C. Image output device configuration:
FIG. 6 is a block diagram illustrating a schematic configuration of the printer 20 of the present embodiment. The printer 20 is a printer that can output an image. For example, an ink jet that ejects four colors of ink of cyan C, magenta Mg, yellow Y, and black K onto a printing medium to form a dot pattern. Printer. Alternatively, an electrophotographic printer that forms an image by transferring and fixing toner onto a print medium may be used. In addition to the above four colors, light cyan LC having a lighter density than cyan C, light magenta LM having a lighter density than magenta Mg, and dark yellow DY having a darker density than yellow Y may be used for the ink. When performing monochrome printing, only black K may be used, or red or green may be used. The type of ink or toner to be used can be determined according to the characteristics of the output image.

  As shown in the figure, the printer 20 drives a print head 211 mounted on the carriage 21 to discharge ink and form dots, and a mechanism that causes the carriage 21 to reciprocate in the axial direction of the platen 23 by the carriage motor 22. And a mechanism for transporting the printing paper P by the paper feed motor 24, and a control circuit 30. With these mechanisms, the printer 20 functions as an image output unit. The mechanism for reciprocating the carriage 21 in the axial direction of the platen 23 is an endless drive between the carriage motor 22 and the sliding shaft 25 that holds the carriage 21 that is laid in parallel with the platen 23 axis. A pulley 27 that stretches the belt 26, a position detection sensor 28 that detects the origin position of the carriage 21, and the like. The mechanism for transporting the printing paper P includes a platen 23, a paper feed motor 24 that rotates the platen 23, a paper feed auxiliary roller (not shown), and a gear train that transmits the rotation of the paper feed motor 24 to the platen 23 and the paper feed auxiliary roller. (Not shown).

  The control circuit 30 appropriately controls the movement of the paper feed motor 24, the carriage motor 22, and the print head 211 while exchanging signals with the operation panel 29 of the printer. The printing paper P supplied to the printer 20 is set so as to be sandwiched between the platen 23 and the paper feed auxiliary roller, and is fed by a predetermined amount according to the rotation angle of the platen 23.

  The carriage 21 has a print head 211 and can be mounted with an ink cartridge of available ink. A nozzle for ejecting usable ink is provided on the lower surface of the print head 211 (not shown).

  FIG. 7 is a block diagram showing the configuration of the printer 20 with the control circuit 30 of the printer 20 as the center. Inside the control circuit 30, there are a CPU 31, a PROM 32, a RAM 33, a memory card slot 34 for acquiring data from the memory card MC, a peripheral device input / output for exchanging data with the paper feed motor 24, the carriage motor 22, and the like. A unit (PIO) 35, a drive buffer 37, and the like are provided. The drive buffer 37 is used as a buffer for supplying dot on / off signals to the print head 211. These are connected to each other by a bus 38 and can exchange data with each other. The control circuit 30 is also provided with a transmitter 39 that outputs a drive waveform at a predetermined frequency, and a distribution output device 40 that distributes the output from the transmitter 39 to the print head 211 at a predetermined timing.

  The control circuit 30 outputs dot data to the drive buffer 37 at a predetermined timing while synchronizing with the movement of the paper feed motor 24 and the carriage motor 22. Further, the control circuit 30 reads the image file from the memory card MC, analyzes the attached information, and performs image processing based on the obtained image generation history information. That is, the control circuit 30 functions as an image quality adjustment unit. A detailed flow of image processing executed by the control circuit 30 will be described later.

D. Image processing in digital still cameras:
FIG. 8 is a flowchart showing a flow of processing for generating the image file GF in the digital still camera 12.

  The control circuit 124 (FIG. 2) of the digital still camera 12 generates image data GD in response to a photographing request, for example, pressing of the shutter button (step S100). When parameter values such as an aperture value, ISO sensitivity, and shooting mode are set, image data GD is generated using the set parameter values.

  The control circuit 124 stores the generated image data GD and the image generation history information GI as an image file GF in the memory card MC (step S110), and ends this processing routine. The image generation history information GI is automatically set such as parameter values used at the time of image generation such as aperture value and ISO sensitivity, parameter values that can be arbitrarily set such as a shooting mode, manufacturer name, and model name. Contains parameter values. The image data GD is converted from the RGB color space to the YCbCr color space, JPEG compressed, and stored as an image file GF.

  Through the above processing executed in the digital still camera 12, image generation history information GI including each parameter value at the time of image data generation is set in the image file GF stored in the memory card MC together with the image data GD. The Rukoto.

E. Image processing in the printer:
FIG. 9 is a flowchart showing a processing routine of image processing in the printer 20 of this embodiment. The following description is based on the case where the memory card MC storing the image file GF is directly inserted into the printer 20. When the memory card MC is inserted into the memory card slot 34, the CPU 31 of the control circuit 30 (FIG. 7) of the printer 20 reads the image file GF (FIG. 3) from the memory card MC (step S200). Next, in step S210, the CPU 31 retrieves image generation history information GI indicating information at the time of image data generation from the attached information storage area of the image file GF. When the image generation history information GI can be found (step S220: Y), the CPU 31 acquires and analyzes the image generation history information GI (step S230). The CPU 31 executes image processing to be described later based on the analyzed image generation history information GI (step S240), outputs the processed image (step S250), and ends this processing routine.

  On the other hand, an image file generated using a drawing application or the like does not include image generation history information GI having information such as an aperture value. If the image generation history information GI cannot be found (step S220: N), the CPU 31 performs standard processing (step S260), outputs the processed image (step S250), and ends this processing routine. .

  FIG. 10 is a flowchart showing a processing routine of image processing (corresponding to step S240 in FIG. 9) based on image generation history information. The CPU 31 of the control circuit 30 (FIG. 7) of the printer 20 extracts image data GD from the read image file GF (step S300).

  As described above, the digital still camera 12 stores the image data GD as a JPEG format file, and the JPEG format file stores image data using the YCbCr color space. In step S310, the CPU 31 executes a calculation using a 3 × 3 matrix S in order to convert image data based on the YCbCr color space into image data based on the RGB color space. This matrix operation is expressed by, for example, the following arithmetic expression.

  When the color space of the image data generated by the digital still camera 12 is wider than a predetermined color space, for example, the sRGB color space, the image data based on the RGB color space obtained in step S310 is displayed in the RGB color space. Valid data may be included outside the definition area. If it is specified in the image generation history information GI that the data outside the definition area is treated as valid data, the data outside the definition area is held as it is, and the subsequent image processing is continued. If the data outside the definition area is not designated as valid data, the data outside the definition area is clipped into the definition area. For example, when the definition area is 0 to 255, negative data less than 0 is rounded to 0, and 256 or more data is rounded to 255. When the color space that can be expressed by the image output unit is not wider than a predetermined color space, for example, the sRGB color space, it is preferable to perform clipping in the definition area regardless of the designation in the image generation history information GI. As such a case, for example, there are cases where the color space that can be expressed is output to a CRT that is an sRGB color space.

  Next, in step S320, the CPU 31 executes gamma correction and calculation using the matrix M to convert image data based on the RGB color space into image data based on the XYZ color space. The image file GF can include a gamma value and color space information at the time of image generation. When the image generation history information GI includes such information, the CPU 31 acquires a gamma value of the image data from the image generation history information GI, and executes a gamma conversion process of the image data using the acquired gamma value. Further, the CPU 31 acquires the color space information of the image data from the image generation history information GI, and executes the matrix calculation of the image data using the matrix M corresponding to the color space. When the image generation history information GI does not include a gamma value, the gamma conversion process can be executed using a standard gamma value. In addition, when the image generation history information GI does not include color space information, a matrix operation can be performed using a standard matrix M. As these standard gamma values and matrix M, for example, gamma values and matrices for the sRGB color space can be used. This matrix operation is, for example, an arithmetic expression shown below.

  The color space of the image data obtained after execution of the matrix operation is an XYZ color space. The XYZ color space is an absolute color space, and is a device-independent color space that does not depend on a device such as a digital still camera or a printer. Therefore, by performing color space conversion via the XYZ color space, device-independent color matching can be performed.

Next, in step S330, the CPU 31 performs calculation using the matrix N- ¹ and inverse gamma correction, and converts the image data based on the XYZ color space into image data based on the wRGB color space. Here, as the wRGB color space, for example, a color space having a larger color gamut than the standard sRGB color space and appropriately determined in consideration of the reproduction color gamut of the printer can be used. When executing inverse gamma correction, the CPU 31 acquires the gamma value on the printer side from the PROM 32, and executes inverse gamma conversion processing of the image data using the inverse of the acquired gamma value. Further, the CPU 31 acquires a matrix N ^-1 corresponding to the conversion from the XYZ color space to the wRGB color space from the PROM 32, and executes a matrix operation of the image data using the matrix N ^-1 . This matrix operation is, for example, an arithmetic expression shown below.

  Next, in step S340, the CPU 31 executes automatic image quality adjustment processing. In the automatic image quality adjustment process in the present embodiment, the image quality automatic image quality adjustment process is executed using the image generation history information included in the image file GF. The automatic image quality adjustment process will be described later.

  Next, in step S350, the CPU 31 executes CMYK color conversion processing for printing and halftone processing. In the CMYK color conversion process, the CPU 31 refers to a look-up table (LUT) for conversion from the wRGB color space to the CMYK color space stored in the PROM 32, and changes the color space of the image data from the wRGB color space to the CMYK color space. Change to. That is, image data composed of RGB gradation values is used by the printer 20, for example, C (Cyan) Mg (Magenta) Y (Yellow) K (Black) LC (LightCyan) LM (LightMagenta). Convert to image data consisting of tone values.

  In the halftone process, the CPU 31 performs a so-called halftone process to generate halftone image data from the color-converted image data. The halftone image data is rearranged in the order to be sent to the drive buffer 37 (FIG. 7), becomes final print data, and this processing routine is terminated. The image data processed by this processing routine is output in step S250 of the image processing routine shown in FIG.

  The flow of processing in FIGS. 9 to 10 is the same in other embodiments described later.

F. Example of automatic image quality adjustment processing:
FIG. 11 is a flowchart showing a processing routine of automatic image quality adjustment processing (corresponding to step S340 in FIG. 10) in the present embodiment. The CPU 31 (FIG. 7) analyzes the image generation history information GI and obtains parameter values such as subject luminance value information (step S400). In step S410, the CPU 31 executes a color balance adjustment process based on the acquired brightness of the subject (details will be described later), and ends the automatic image quality adjustment process.

G. Example of color balance adjustment processing:
FIG. 12 is a flowchart showing a processing routine of color balance adjustment processing in the present embodiment. In step S500, the CPU 31 (FIG. 7) selects a region close to an achromatic color used for calculation of the color cast amount (described later). Next, in step S510, the color cast amount of each color of red R, green G, and blue B is calculated using the pixel value of the region selected in step S500. The color fog amount is an index indicating the degree of color shift in the image data. For example, as the red color cast amount, the average value of the red R tone values and the tone value obtained by combining all the colors The difference from the average value can be used (details will be described later). As the types of colors used for obtaining the color cast amount, various combinations such as cyan C, magenta Mg, and yellow Y can be used in addition to the combinations of red R, green G, and blue B which are basic colors. . Next, in step S520, the amount of color balance adjustment processing is set, and in step S530, the gradation value of each color is adjusted so that the color fog amount is reduced (described later).

  FIG. 13 is an explanatory diagram showing the color fog amount and gradation value adjustment. FIG. 13A shows an example of the gradation value distribution of each color of red R, green G, and blue B in a reddish image. These distributions indicate gradation value distributions in a color cast amount calculation area close to an achromatic color, which will be described later, and red R is biased toward a larger side than green G and blue B. Such an image showing a distribution in which the red R is biased toward the larger one is likely to be generated when a light source having a strong red color, for example, a candle or an incandescent lamp is used.

  Formula 4 shown below is an arithmetic expression for calculating the color fog amounts ΔR, ΔG, and ΔB in this embodiment.

  In the example shown in Formula 4, the luminance value calculated using the average gradation values Rave, Gave, and Bave of each RGB color and the average gradation value of each color as the color fog amounts ΔR, ΔG, and ΔB of each RGB color The difference from Lave is used. As an arithmetic expression for calculating the luminance value, for example, a conversion expression from the RGB color space to the YCbCr color space shown in the following Expression 5 can be used.

  The luminance value Lave obtained by using this arithmetic expression can be said to be an average gradation value calculated by assigning a difference in brightness according to color as a weight to each of red R, green G, and blue B. When the color misregistration is small, the average gradation values Rave, Gave, and Bave of each color of RGB are almost the same value. Therefore, the luminance value, that is, the average gradation value Lave using the brightness as a weight, The average gradation values Rave, Gave, and Bave are almost the same value. As a result, small values are obtained as the color fog amounts ΔR, ΔG, ΔB of the respective colors. When the color misregistration is large, the average gradation values Rave, Gave, and Bave of RGB colors are different from each other. In this case, the larger the amount of color fog, the greater the deviation from the luminance value Level as the reference value. As described above, by using the average gradation value (luminance value) calculated by using the brightness depending on the color as the weight as the reference value for calculating the color fog amount, the color fog amount closer to the sense of the human eye is calculated. be able to.

  FIG. 13B is an explanatory diagram showing the relationship between the red R input level Rin and the output level Rout in the gradation value adjustment processing of this embodiment. The graph G1A is configured such that the output level Rout is smaller than the input level Rin. When the gradation value of red R is adjusted using this graph G1A, the gradation value of red R is reduced in an image whose color is shifted to red, that is, an image where red R is biased toward the larger one. Deviation can be reduced.

  Such a graph G1A can be configured, for example, by adjusting the output level Rout at the adjusted input level Rref so as to be smaller than the original value by the adjustment amount RM. The output level Rout corresponding to the other input level Rin is interpolated by a spline function. The adjustment amount RM is a value determined based on the color fog amount ΔR (FIG. 13A, Formula 4). For example, a value obtained by multiplying the color fog amount ΔR by a predetermined coefficient k can be used. As the predetermined coefficient k, a value determined based on the sensitivity evaluation of the output result of the image can be used. The relationship between the color fog amount ΔR and the adjustment amount RM is not necessarily a proportional relationship, and may be a relationship in which the adjustment amount RM increases as the color fog amount increases. The predetermined coefficient k can be used as the intensity of the color balance adjustment process (details will be described later). A predetermined value can be used as the adjustment input level Rref. For example, when the range that red R can take is 0 to 255, the intermediate value 128 may be used.

  A graph G1B shows an input / output relationship used in the gradation value adjustment processing having a larger color balance adjustment processing amount than the graph G1A. Here, “the color balance adjustment processing amount is large” means that the amount of change in the color gradation value is large. When the color fog amount ΔR is large, the adjustment amount RM calculated using the predetermined coefficient k is large, so that the color balance adjustment processing amount is also large. Therefore, even when the color fog amount ΔR is large, the color deviation can be reduced. In this way, by configuring so that the color balance adjustment processing amount increases as the color cast amount increases, the color deviation can be appropriately reduced based on the size.

  When the predetermined coefficient k used for calculating the adjustment amount RM is increased, the adjustment amount RM for the same color fog amount ΔR can be increased. As a result, the amount of color balance adjustment processing for the same color fog amount can be increased. That is, by increasing the coefficient k, it is possible to increase the strength of the color balance adjustment process. Here, “the intensity of the color balance adjustment process is strong” means that the processing amount of the color balance adjustment process for the same image having a color shift is large. Therefore, the predetermined coefficient k can be referred to as a processing parameter that represents the ratio of the amount of color balance adjustment processing to the amount of color fog, and color balance adjustment based on the amount of color fog. It can be said that the processing parameter affects the processing result for determining the processing amount of the processing. Even when the intensity of the color balance adjustment process is strong, if the color shift is small, the processing amount of the color balance adjustment process is small.

  The graph G2A is configured such that the output level Rout is higher than the input level Rin, and shows the input / output relationship used when the red R is biased toward the smaller side. A graph G2B shows an input / output relationship used in the gradation value adjustment processing having a larger color balance adjustment processing amount than the graph G2A. When the color is biased toward the smaller side, that is, when the average gradation value Rave is smaller than the reference luminance value Lave, the adjustment amount is based on the color fog amount ΔR as in the case where the color is biased toward the larger side. RM is determined, and the color balance adjustment processing amount is determined.

  The relationship between the input level and the output level described above is similarly set for colors other than red R.

  FIG. 14 is an explanatory diagram showing a region used for calculation of the color cast amount. The image IMG14 in FIG. 14A is an image in which the white bird Bd14 is on the green carpet Cpt14. The calculation of the color fog amount is performed according to the pixel value in the area CBA.

  FIG. 14B is an explanatory diagram showing conditions for selecting a region CBA used for calculation of the color cast amount. The horizontal axis indicates the saturation St, and the vertical axis indicates the luminance value L. In this embodiment, a pixel close to an achromatic color that satisfies the following two conditions is selected as a pixel to be used for color cast amount calculation.

(A1) The saturation St is equal to or less than the saturation threshold Stth.
(B1) The luminance value L is greater than or equal to the luminance threshold value Lth.

  These two conditions a1 and b1 can be understood as follows.

  When a colorful subject that is not an achromatic color is captured, the saturation of the region in which the subject is captured increases. In such an area, the amount of color cast due to the color specific to the subject increases, but this color cast is not a color shift caused by the type of light source. However, if the color cast amount is calculated based on the color of this region and the gradation value adjustment is performed so as to reduce the obtained color cast amount, the vividness and hue of the color specific to the subject may be greatly changed. . Therefore, by calculating the color cast amount using an area that satisfies the above-described condition a1, that is, an area where the saturation St is equal to or less than the saturation threshold value Stth, the vividness and hue of the color unique to the subject can be greatly changed. The color balance adjustment process can be executed. As the saturation threshold Stth, a value determined based on the sensitivity evaluation of the image output result can be used. For example, when the range that saturation can take is 0 to 1, it may be set to 0.1. As the saturation threshold value is reduced, a region close to an achromatic color can be selected, and therefore the influence of the vividness and hue of the color unique to the subject on the color balance adjustment process can be further reduced.

  There is a high possibility that a region with a large luminance value is strongly receiving light from the light source. Therefore, a bright region having a large luminance value is more likely to reflect the color shift due to the spectral distribution of the light source more strongly. Therefore, the color fog amount according to the type of the light source is calculated more accurately by calculating the color fog amount using a region satisfying the above-described condition b1, that is, a region where the luminance value L is equal to or greater than the luminance threshold value Lth. be able to. As the brightness threshold value Lth, a value determined based on the sensitivity evaluation of the image output result can be used. For example, 180 may be used when the range of luminance values is 0 to 255.

  As described above, by using an area configured by pixels satisfying the two conditions a1 and b1, the color fog amount based on the type of the light source can be calculated more appropriately. In the example of FIG. 14A, the region CBA used for calculating the color fog amount is a pixel region that satisfies the two conditions a1 and b1, and does not include the region of the green carpet Cpt14. Therefore, when the green color of the carpet Cpt14 is bright green, it is possible to suppress an increase in the amount of green color fog. Note that the pixels used for calculating the color cast amount do not need to form one area as shown in FIG. 14A, and may be isolated. That is, all the pixels satisfying the above-described conditions a1 and b1 are selected as pixels used for calculating the color fog amount.

  When the image data is expressed in a color space that does not include the luminance value and saturation as parameters, for example, when the image data is expressed using the RGB color space, the color space that includes the luminance value and saturation as parameters For example, the luminance value and the saturation at each pixel position can be acquired by converting into an HLS color space, a HIS color space, or the like.

H. Example of color balance adjustment processing according to the brightness of the subject:
H1. First embodiment of color balance adjustment processing according to the brightness of the subject:
FIG. 15 is an explanatory diagram illustrating an example of color balance adjustment processing according to the brightness of the subject. FIG. 15A is an explanatory diagram showing the relationship between the subject luminance value and the color balance adjustment processing intensity. The subject luminance value is photometric luminance information included in the image generation history information GI shown in FIG. 5, and indicates the degree of brightness of the subject. As the color balance adjustment processing intensity, for example, the ratio of the amount of color fog and the amount of color balance adjustment processing can be used (in the example of FIG. 13B, the magnitude of a predetermined coefficient k is the intensity. Can be used as). In FIG. 15A, a graph SR shows a relationship used in color balance adjustment processing according to the brightness of the subject, and a graph STD shows a relationship used in standard color balance adjustment processing independent of the subject brightness. Show. The graph SR is configured such that the color balance adjustment processing intensity when the subject brightness value is equal to or less than a predetermined threshold is greater than the intensity when the subject brightness value is greater than the predetermined threshold. . The graph STD is configured such that the intensity is constant regardless of the subject luminance value. FIG. 15B is an explanatory diagram showing the relationship between the subject luminance value and the amount of color balance adjustment processing when the graph SR is used. As shown in FIG. 15 (a), the color balance adjustment processing intensity is configured to increase when the subject luminance value is equal to or less than a predetermined threshold value. As it decreases, the amount of processing increases. Further, the processing amount is a value adjusted based on the color fog amount, and the processing amount increases as the color fog amount increases even with the same subject luminance value.

  The reason for changing the color balance adjustment processing intensity in accordance with the subject luminance value is because the following phenomenon is taken into consideration. One of the factors that determine the subject brightness value, that is, the brightness of the subject, is the type of light source. Various light sources such as fluorescent lamps, incandescent lamps, candles, and natural light (sunlight) are used. Among these light sources, artificial light sources such as fluorescent lamps, incandescent lamps, and candles have a spectral distribution of light that is significantly different from that of natural light, particularly daylight. Therefore, when shooting under these artificial light sources, image data with a large color cast amount and a strong sense of incongruity is likely to be generated. Moreover, since these artificial light sources are weaker than daylight, the brightness of the subject tends to be small. Also, in the case of sunset or sunrise, the brightness of the subject is low because the same natural light, but the intensity of light is weaker than in daylight, and the spectrum distribution is different from that in daylight. Small and easy to generate image data with a large color cast. On the other hand, when photographing in natural light, particularly daylight, the type of light source, that is, the amount of color cast derived from the spectral distribution of the light source is small. Further, since the amount of daylight is large, the brightness of the subject tends to increase. That is, when the brightness of the subject is small, there is a high possibility of color shift based on the type of light source. Therefore, when the brightness of the subject is equal to or lower than the predetermined threshold, the color cast based on the type of light source can be further reduced by executing the color balance adjustment process with a stronger intensity. As such a threshold value of the brightness of the subject, a value determined based on the sensitivity evaluation of the output result of the image can be used. For example, 3 in the APEX unit may be used as the threshold value.

FIG. 16 is an explanatory diagram illustrating another example of the relationship between the subject luminance value and the color balance adjustment processing intensity. In the graph SR used in the color balance adjustment process according to the brightness of the subject, the color balance adjustment process strength increases as the subject brightness value decreases in a range where the subject brightness value is smaller than a predetermined threshold value. It is comprised so that it may become. In image data generated under natural light (daylight), there is a high possibility that the subject luminance value is large. Conversely, in the image data generated under an artificial light source, there is a high possibility that the subject luminance value is small. That is, the smaller the subject brightness value, the higher the possibility that image data has been generated under an artificial light source. Therefore, in the range where the subject brightness value is less than or equal to the threshold value, the intensity of the color balance adjustment processing is increased as the subject brightness value becomes smaller. To reduce the color cast.

  The color balance adjustment processing intensity may be changed stepwise in a plurality of stages without being continuously changed according to the subject luminance value. By doing so, the adjustment of the color balance adjustment processing intensity can be simplified.

  In both the case of the graph SR shown in FIG. 15A and the case of the graph SR shown in FIG. 16, “the intensity of the color balance adjustment process is adjusted so that it becomes stronger as the brightness degree is smaller”. In common. That is, in this specification, “adjusting the intensity of the color balance adjustment process so that it becomes stronger as the degree of brightness is smaller” means that “the intensity when the degree of brightness is a predetermined threshold value or less is This includes a case where “adjustment is made to be greater than the intensity when it is larger than a predetermined threshold value” and a case where “adjustment is made so that the intensity increases monotonously as the brightness level decreases”. Here, “the intensity increases monotonously with a decrease in the brightness level” means that “the intensity does not decrease with a decrease in the brightness level”.

  Further, it is preferable that such “adjustment of the color balance adjustment processing intensity so that it becomes stronger as the degree of brightness is smaller” is executed in a range where the degree of brightness is small. When shooting is performed using a light source that tends to cause a large color shift, the brightness of the subject tends to be dark. In other words, it can be said that there is a high possibility that a large color shift occurs when the brightness level of the subject is small. Therefore, by executing “adjustment of the color balance adjustment processing intensity so that the brightness level is small” in a range where the brightness of the subject is small, the vividness and hue of the subject-specific color can be greatly changed. In addition, color misregistration based on the type of light source can be reduced. As such a small range of the subject brightness, for example, a range in which the value obtained by converting the subject brightness in APEX units is 3 or less can be used.

H2. Second embodiment of color balance adjustment processing according to the brightness of the subject:
FIG. 17 is an explanatory diagram illustrating another example of the color balance adjustment process according to the brightness of the subject. The image IMG17 in FIG. 17A is the same as the image IMG14 in FIG. In the image IMG 17, areas CBA1 and CBA2 used for calculating the color fog amount are selected. A region CBA2 indicates a region used for calculating the color fog amount, which is used in the color balance adjustment process with a stronger intensity than the region CBA1.

  FIG. 17B is an explanatory diagram showing conditions for selecting the regions CBA1 and CBA2 used for calculating the color cast amount. The horizontal axis indicates the saturation St, and the vertical axis indicates the luminance value L. The area CBA1 is composed of pixels whose luminance value L is equal to or higher than the first luminance threshold value Lth1 and whose saturation St is equal to or lower than the first saturation threshold value Stth1. In the region CBA2, the luminance value L is equal to or greater than the second luminance threshold value Lth2 that is smaller than the first luminance threshold value Lth1, and the saturation St is larger than the first saturation threshold value Stth1. It is composed of pixels that are equal to or lower than the second saturation threshold value Stth2. Therefore, the region CBA2 can include pixels with higher saturation than the region CBA1. As a result, in an image in which the color balance is shifted, the color fog amount calculated using the region CBA2 is larger than the color fog amount calculated using the region CBA1, so that the region CBA2 is more preferable. The amount of color balance adjustment processing increases. That is, a stronger color balance adjustment process can be executed by adjusting the luminance threshold value and the saturation threshold value so as to widen the condition range in order to select the color cast amount calculation region. Therefore, the luminance threshold value and the saturation threshold value can be referred to as processing parameters indicating the range of the region close to the achromatic color, and the processing for determining the size of the color cast amount by analyzing the image data It can be said that the processing parameter affects the result. In this embodiment, it can be said that the color balance adjustment processing intensity is adjusted by adjusting the size of the color cast amount calculated from the same image.

  As the color balance adjustment processing intensity in the color balance adjustment processing example according to the brightness of the subject shown in FIGS. 15 and 16, the size of the condition range of the above-described color fog calculation region can be used. For example, in the example of the graph SR shown in FIG. 15A, when the subject brightness value is larger than a predetermined threshold value, the area CBA1 in FIG. 17 is used as the color cast amount calculation area, and the subject brightness value is predetermined. If the threshold value is equal to or less than the threshold value, the area CBA2 can be used. By doing so, the color cast based on the type of light source can be further reduced. The threshold values of the luminance value and the saturation for selecting the region CBA1 and the region CBA2 can be determined based on the sensitivity evaluation of the image output result. In addition, as in the example of the graph SR illustrated in FIG. 16, when a plurality of intensities can be obtained according to the subject luminance value, the condition range of the color fog calculation region is increased as the intensity increases. A luminance value and a saturation threshold are configured. By doing so, it is possible to reduce the color cast due to the type of light source corresponding to individual image data. The adjustment of the condition range may be configured such that one of the luminance value threshold and the saturation threshold is set to a fixed value and the other is adjusted. In either case, the condition range is widened as the strength increases. Further, it is also possible to adjust both the condition range of the color fog calculation area and the ratio of the color balance adjustment processing amount to the color fog amount described above in accordance with the intensity.

I. Another example of automatic image quality adjustment processing:
I1. Second embodiment of automatic image quality adjustment processing:
FIG. 18 is a flowchart showing a processing routine of automatic image quality adjustment processing in the present embodiment. The difference from the embodiment shown in FIG. 11 is that the type of color balance adjustment processing is switched based on the determination result of whether or not light irradiation by the auxiliary light source has been performed at the time of image data generation. In this embodiment, using the parameter value (FIG. 5) of the flash information obtained by analyzing the image generation history information GI, it is determined whether or not light irradiation by the auxiliary light source has been performed. When the operation result included in the flash information indicates light emission, the CPU 31 (FIG. 7) determines that light irradiation has been performed. In addition, as described above, when the reflected light detection mechanism is present and the reflected light detection is absent, it is determined that light irradiation has not been performed even when the operation result is light emission. Also, when “no flash” is set, it is determined that light irradiation has not been performed.

  When it is determined that light irradiation by the auxiliary light source has not been performed (step S610: Y), the CPU 31 performs a color balance adjustment process with different intensities depending on the brightness of the subject, as in the example of FIG. This is executed (step S620), and the automatic image quality adjustment process is terminated.

  If it is determined that light has been emitted from the auxiliary light source (step S610: N), a standard color balance adjustment process that does not increase the intensity even if the brightness level of the subject is small is executed. (Step S630). An auxiliary light source such as a flash has a spectral distribution close to natural light (daylight). Therefore, the amount of color fog derived from the spectral distribution of the light source can be reduced by irradiating light with the auxiliary light source. Therefore, when light is emitted from the auxiliary light source, the standard color balance adjustment process that does not increase the intensity even when the brightness level of the subject is smaller than the predetermined threshold is executed. It is possible to suppress a significant change in the vividness and hue of the color.

I2. Third embodiment of automatic image quality adjustment processing:
FIG. 19 is a flowchart showing a processing routine of automatic image quality adjustment processing in the present embodiment. A difference from the embodiment shown in FIG. 18 is that a determination is made as to whether or not the subject distance is equal to or greater than the irradiation threshold (step S730). In this embodiment, the parameter value (FIG. 5) of the subject distance information acquired from the image generation history information GI is used as the distance between the subject and the image generating device. If the subject distance is equal to or greater than the predetermined irradiation threshold (step S730: Y), color balance adjustment processing based on the brightness of the subject is executed (step S720). The farther the subject distance, that is, the distance between the image generating device and the subject, the smaller the amount of light received by the subject from the auxiliary light source. For this reason, when the subject distance is large (far), the effect of suppressing color misregistration derived from the spectral distribution of the light source due to light irradiation from the auxiliary light source is not sufficiently exhibited. Therefore, when the subject distance is equal to or greater than the irradiation threshold, it is determined that the light irradiation by the auxiliary light source has not been executed, and the color balance adjustment process based on the brightness of the subject is executed, thereby It is possible to reduce the color deviation of an image in which the amount of color cast is increased without sufficiently reaching the subject. The irradiation threshold value can be determined based on the sensitivity evaluation of the image output result. For example, it may be 2 m.

  When the subject distance is smaller than the irradiation threshold value (step S730: N), a standard color balance adjustment process that does not increase the intensity even when the subject brightness level is smaller than the predetermined threshold value is executed (step S730: N). Step S740). By doing so, it is possible to suppress a significant change in the vividness and hue of the color unique to the subject.

  The irradiation threshold is a reference value for determining the amount of light received by the subject. Therefore, by adjusting the irradiation threshold based on other parameter values that change the amount of light received by the subject, a more appropriate determination can be performed. For example, the irradiation threshold value may be configured to increase as the flash intensity included in the image generation history information GI increases. In this way, the subject distance magnitude determination can be appropriately executed based on the flash intensity, that is, the amount of light that the auxiliary light source irradiates the subject.

I3. Fourth embodiment of automatic image quality adjustment processing:
FIG. 20 is a flowchart showing a processing routine of automatic image quality adjustment processing in the present embodiment. A difference from the embodiment shown in FIG. 19 is that it is determined whether or not the image is a photograph of a person, and if it is a person image, an image quality adjustment process suitable for the person image is performed. is there. In this embodiment, light is emitted from the auxiliary light source (step S810: N, step S830: N), and the subject distance is less than or equal to a predetermined person distance threshold and the subject luminance value is If it is greater than or equal to the person brightness threshold (step S850: Y), it is determined that the image is a photograph of a person. A parameter value (FIG. 5) of subject distance information acquired from the image generation history information GI is used as the subject distance, and a parameter value (FIG. 5) of subject luminance value information is used as the subject brightness value.

  When shooting a person in bright sunny weather, make the subject distance small (near) so that the person appears sufficiently large, and then illuminate with an auxiliary light source such as a flash so that no shadows appear on the face. There are many. Therefore, when light from the auxiliary light source is irradiated, it is further determined that the image is a person image when the subject distance is equal to or smaller than the person distance threshold value and the subject luminance value is equal to or greater than the person luminance threshold value. can do. The person distance threshold and the person luminance threshold can be determined based on the sensitivity evaluation of the image output result. For example, the person distance threshold value may be 2 m, and the person brightness threshold value may be 6 (unit is APEX).

  If it is determined that the image is a person image (step S850: Y), a process suitable for the person scene (described later) is executed (step S860), and the automatic image quality adjustment process is terminated.

  On the other hand, if it is not determined that the image is a person image (step S810: Y, or step S830: Y, or step S850: N), processing (described later) suitable for the standard scene is executed. (Step S870), the automatic image quality adjustment processing is terminated.

  In this embodiment, processing suitable for each scene is executed as shown in FIG. For example, when it is determined that the image is a human image, the contrast is set to be slightly soft, the brightness is set to be slightly bright, the saturation is set to be slightly low, and the sharpness is set to be slightly weak. Since the skin color is designated as the memory color, the skin color correction is executed using the skin color data stored in advance. Furthermore, noise processing is turned off. By executing the image quality adjustment process in this way, the atmosphere of the image is softened and the skin color of the person is adjusted to a preferable skin color. However, the image quality adjustment process does not necessarily follow the setting shown in FIG. 21, and another setting may be used.

  As described above, in this embodiment, it is possible to automatically select a shooting scene for each piece of image data using the degree of brightness of the subject and perform appropriate image quality processing. As a result, there is an advantage that it is not necessary for the user to manually perform complicated operations during shooting or printing.

J. et al. Configuration of an image output system using an image data processing device:
FIG. 22 is an explanatory diagram showing an example of an image output system to which the image data processing apparatus as an embodiment of the present invention can be applied. The image output system 10B includes a digital still camera 12 as an image generation apparatus that generates an image file, a computer PC that executes image quality adjustment processing based on the image file, and a printer 20B as an image output apparatus that outputs an image. I have. The computer PC is a commonly used type of computer and functions as an image data processing device. As the image output device, in addition to the printer 20B, a monitor 14B such as a CRT display or LCD display, a projector, or the like can be used. In the following description, it is assumed that the printer 20B is used as an image output device. The present embodiment is different from the above-described image output system embodiment (FIG. 1) in that an image data processing apparatus including an image quality adjustment unit and an image output apparatus including an image output unit are configured independently. Note that a computer PC as an image data processing apparatus and a printer including an image output unit can be called an “output apparatus” in a broad sense.

  The image file generated in the digital still camera 12 is sent to the computer PC via the cable CV or by directly inserting the memory card MC storing the image file into the computer PC. The computer PC executes image quality adjustment processing of image data based on the read image file. The image data generated by the image quality adjustment process is sent to the printer 20B via the cable CV and output by the printer 20B.

  The computer PC includes a CPU 150 that executes a program that realizes the above-described image quality adjustment processing, a RAM 151 that temporarily stores calculation results and image data of the CPU 150, an image quality adjustment processing program, a lookup table, and an aperture value table. For example, a hard disk drive (HDD) 152 that stores data necessary for image quality adjustment processing is provided. The CPU 150, the RAM 151, and the HDD 152 function as an image quality adjustment unit. Furthermore, the computer PC includes a memory card slot 153 for mounting a memory card MC and an input / output terminal 154 for connecting a connection cable from the digital still camera 12 or the like.

  The image file GF generated by the digital still camera 12 is provided to the computer PC via a cable or a memory card MC. When an image retouching application or an image data processing application program such as a printer driver is activated by a user operation, the CPU 150 executes an image processing routine (FIG. 9) for processing the read image file GF. Further, the image data processing application program is automatically started by detecting the insertion of the memory card MC into the memory card slot 153 or the connection of the digital still camera 12 via the cable to the input / output terminal 154. Also good.

  The image data processed by the CPU 150 is sent to an image output device, for example, the printer 20B, instead of being output in step S250 of the image processing routine (FIG. 9), and the image output device receiving the image data receives the image data. Run the output.

  In this embodiment, since image processing is performed using the image quality adjustment unit provided in the computer PC, it is possible to use an image output device that does not include the image quality adjustment unit. When the image output device includes an image quality adjustment unit, the computer PC sends image data to the image output device without performing image processing, and the image quality adjustment unit of the image output device performs image processing. It is good.

  As described above, in each of the above-described embodiments, an appropriate image quality adjustment process is automatically performed on an image or a person image whose color is shifted according to the type of light source using the brightness of the subject. Therefore, high quality output results can be obtained easily.

  The present invention is not limited to the above-described embodiments, and can be implemented in various forms without departing from the gist thereof. For example, the following modifications are possible.

K. Variation:
K1. Modification 1:
In each of the embodiments described above, the subject brightness information related to the brightness of the subject is a combination of the aperture value and shutter speed instead of the subject brightness value, and the sensitivity of the optical circuit (ISO speed rate) is added to these two pieces of information. Combinations can be used. Two values L1 and L2 shown in Equation 6 below are degrees of the brightness of the subject calculated based on the combination information.

  In Equation 6, Av is a value obtained by converting the aperture value (F value) in APEX units, Tv is a value obtained by converting the shutter speed in APEX units, and Sv is the sensitivity of the optical circuit (ISO speed rate) in APEX units. It is a converted value. These three values Av, Tv, and Sv are values that are adjusted according to the brightness of the subject so that the amount of light received by the image generation apparatus when generating the image data becomes an appropriate value. When the brightness level of the subject is large, the aperture value (F value) is large, that is, the Av is adjusted to be large. Further, the shutter speed is adjusted to be small, that is, Tv is increased, and the ISO speed rate is adjusted to be small, that is, Sv is decreased. L1 (= Av + Tv) indicates the amount of light received by the image generation apparatus, and Av and Tv are adjusted so that the size of L1 becomes an appropriate value according to the brightness of the subject. L2 (= Av + Tv−Sv) indicates the brightness of the subject. When Sv (sensitivity of the optical circuit) can be adjusted, Av, Tv, and Sv are adjusted so that the magnitude of L2 becomes an appropriate value according to the brightness of the subject. When the brightness level of the subject is large, L1 and L2 are adjusted to be large, and when the brightness level of the subject is small, L1 and L2 are adjusted to be small. Therefore, in each of the above-described embodiments, L1 or L2, that is, “Av + Tv” or “Av + Tv−Sv” can be used as the brightness level of the subject. In particular, when Av, Tv, and Sv are adjusted so that the amount of light received by the image generating apparatus is an appropriate value, L2 is equal to a value obtained by converting the degree of brightness of the subject into APEX units. Therefore, it can be used as it is in place of the above-described subject luminance value. If the image generation history information GI includes these values Av, Tv, and Sv, the brightness level of the subject can be calculated using the parameter values.

K2. Modification 2:
In each of the above-described embodiments, an area having a hue specific to a predetermined subject may be excluded from the color fog calculation area. FIG. 23 is an explanatory diagram showing a hue range specific to a subject. The hue H of this example has a possible range of 0 degrees to 360 degrees, 0 degrees indicates red, 120 degrees indicates green, and 240 degrees indicates blue. In this example, the range of hue H from 0 degrees to 30 degrees is defined as a skin color range SR indicating the skin color of a person, the range of hue H of 100 degrees to 130 degrees is defined as a plant range GR indicating plant or mountain green, and the hue H Is the sky range BR indicating the sky blue. By excluding the region where the hue H is within the skin color range SR from the color cast amount calculation region, it is possible to suppress a significant change in the vividness and hue of the skin color specific to the subject in the person image. Similarly, by excluding the plant range GR, it is possible to suppress a large change in the vividness and hue of green in a landscape image in which a plant image, a mountain, or the like is captured. Further, by excluding the sky range BR, it is possible to suppress a large change in the vividness and hue of blue in a landscape image in which the sky is captured. The hue range is not necessarily the above-described range, and a different range may be set.

K3. Modification 3:
In each of the above-described embodiments, instead of using all the pixels in the color fog calculation area, only the remaining pixels obtained by thinning out the pixels equally may be used for calculating the color fog. By doing so, it is possible to execute the color fog calculation processing at high speed. Further, the entire area in the image may be used as the color fog calculation area. By doing so, the calculation process of the color cast amount becomes simple.

K4. Modification 4:
If the color balance (white balance) of the image data has been set by the user, execute either “Reduce the intensity of the color balance adjustment process” or “Do not execute the color balance adjustment process” It is preferable to do this. By doing so, it is possible to suppress a large change in the color balance set by the user. The white balance information (FIG. 5) included in the image generation history information GI can be used to determine whether the color balance (white balance) setting of the image data is set by the user.

K5. Modification 5:
In the determination of the distance between the subject and the image generation apparatus in each of the above-described embodiments, distance information capable of setting a distance range as a parameter value can be used in addition to the subject distance information. For example, subject distance range information (FIG. 5) selected and set from three distance ranges of macro (0 to 1 m), near view (1 to 3 m), and distant view (3 m to) can also be used. In this case, a representative distance can be set in advance for each distance range, and the size determination can be performed by comparing the representative distance with a threshold value. As a typical distance, for example, for the distance range in which the upper limit value and the lower limit value of the distance are set, the intermediate value is used, and for the distance range in which only the upper limit value or the lower limit value is set, The upper limit value or the lower limit value can be used.

K6. Modification 6:
When the auxiliary light source and the image generation device are installed at different positions to generate image data, instead of determining the distance between the subject and the image generation device in each of the above embodiments, the subject and It is preferable to perform the determination of the distance from the auxiliary light source. Thus, an image in which the subject is not sufficiently irradiated with light because the distance between the auxiliary light source and the subject is long can be a target of color balance adjustment processing according to the brightness of the subject.

K7. Modification 7:
When the image file GF does not include the gamma value and the color space information of the image data, the color space conversion process (Step S320 and Step S330) in the image processing routine shown in FIG. 10 can be omitted. FIG. 24 is a flowchart illustrating an image processing routine when the color space conversion process is omitted. The image data extracted in step S900 is converted from image data based on the YCbCr color space to image data based on the RGB color space in step S902. Next, in step S904, automatic image quality adjustment processing using the image data obtained in step S902 is executed. Next, in S906, CMYK color conversion processing for printing and halftone processing are executed.

K8. Modification 8:
In each of the above-described embodiments, the automatic image quality adjustment process is executed after the color space conversion is executed. However, the color space conversion may be executed after the automatic image quality adjustment process is executed. For example, the image processing may be executed according to the flowchart shown in FIG.

K9. Modification 9:
In each of the above embodiments, a printer is used as the image output unit, but an image output unit other than the printer can be used. FIG. 26 is a flowchart illustrating a processing routine of image processing based on image generation history information when a CRT is used as an image output unit. Unlike the flowchart in which the printer shown in FIG. 10 is an image output unit, CMYK color conversion processing and halftone processing are omitted. Further, since the CRT can express the RGB color space of the image data obtained by executing the matrix operation (S), the color space conversion process is also omitted. If the image data based on the RGB color space obtained in step S952 includes data outside the definition area of the RGB color space, step S954 is executed after the data outside the definition area is clipped. When the color space that can be used by the image output unit is different from the RGB color space, the color conversion process to the color space that can be used by the image output unit is performed in the same manner as when the CMYK color conversion process is executed when using a printer And the resulting image is output from the image output unit.

K10. Modification 10:
In the above embodiment, an Exif format file has been described as a specific example of the image file GF, but the format of the image file according to the present invention is not limited to this. In other words, any image file including image data generated by the image generation apparatus and image generation history information GI describing conditions (information) at the time of image data generation may be used. With such a file, the image quality of the image data generated by the image generation device can be automatically adjusted appropriately and output from the output device.

K11. Modification 11:
The values of the matrices S, N ⁻¹ , and M in each formula are merely examples, and can be appropriately changed according to the color space based on the image file, the color space that can be used by the image output unit, and the like.

K12. Modification 12:
In the above embodiment, the digital still camera 12 has been described as the image generating apparatus. However, an image file can be generated using an image generating apparatus such as a scanner or a digital video camera.

K13. Modification 13:
In the above embodiment, the case where the image data GD and the image generation history information GI are included in the same image file GF has been described as an example. However, the image data GD and the image generation history information GI are not necessarily included in the same file. It need not be stored. That is, it is only necessary that the image data GD and the image generation history information GI are associated. For example, association data that associates the image data GD and the image generation history information GI is generated, and one or a plurality of image data and the image generation history are generated. The information GI may be stored in independent files, and the image generation history information GI associated with processing the image data GD may be referred to. In such a case, the image data GD and the image generation history information GI are stored in separate files, but at the time of image processing using the image generation history information GI, the image data GD and the image generation history information GI are This is because they are inseparable and function in the same manner as when stored in the same file. That is, an aspect in which the image data GD and the image generation history information GI are associated at least at the time of image processing is included in the image file GF in the present embodiment. Furthermore, a moving image file stored in an optical disc medium such as a CD-ROM, a CD-R, a DVD-ROM, a DVD-RAM is also included.

1 is a block diagram showing a configuration of an image output system as an embodiment of the present invention. 1 is a block diagram showing a schematic configuration of a digital still camera 12. FIG. Explanatory drawing which shows notionally an example of the internal structure of the image file which can be used in a present Example. Explanatory drawing explaining the example of a data structure of the attached information storage area 103. FIG. Explanatory drawing explaining an example of the data structure of an Exif data area. FIG. 2 is a block diagram illustrating a schematic configuration of a printer 20. FIG. 3 is a block diagram showing a configuration of the printer 20 with a control circuit 30 of the printer 20 as a center. 7 is a flowchart showing a flow of processing for generating an image file GF in the digital still camera 12. The flowchart which shows the process routine of an image process. 6 is a flowchart showing a processing routine of image processing based on image generation history information. 5 is a flowchart showing a processing routine of automatic image quality adjustment processing. 5 is a flowchart showing a processing routine for color balance adjustment processing. Explanatory drawing which shows color fog amount and gradation value adjustment. Explanatory drawing which shows the area | region used for calculation of a color fog amount. Explanatory drawing which shows the example of the color balance adjustment process according to the brightness of a to-be-photographed object. Explanatory drawing which shows another example of the relationship between a subject luminance value and color balance adjustment processing intensity. Explanatory drawing which shows another example of the color balance adjustment process according to the brightness of a to-be-photographed object. 5 is a flowchart showing a processing routine of automatic image quality adjustment processing. 5 is a flowchart showing a processing routine of automatic image quality adjustment processing. 5 is a flowchart showing a processing routine of automatic image quality adjustment processing. Explanatory drawing which shows the content of the process suitable for an imaging | photography scene. Explanatory drawing which shows an example of the image output system which can apply an image data processing apparatus. Explanatory drawing which shows the hue range peculiar to a photographic subject. 6 is a flowchart illustrating an image processing routine when color space conversion processing is omitted. 12 is a flowchart illustrating another example of a processing routine for image processing based on image generation history information. 12 is a flowchart illustrating another example of a processing routine for image processing based on image generation history information.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Image output system 10B ... Image output system 12 ... Digital still camera 14 ... Monitor 14B ... Monitor 20 ... Printer 20B ... Printer 21 ... Carriage 22 ... Carriage motor 23 ... Platen 24 ... Motor 25 ... Peristaltic shaft 26 ... Drive belt 27 ... Pulley 28 ... Position detection sensor 29 ... Operation panel 30 ... Control circuit 31 ... CPU
32 ... PROM
33 ... RAM
34 ... Memory card slot 35 ... Peripheral device input / output unit 37 ... Drive buffer 38 ... Bus 39 ... Transmitter 40 ... Distribution output device 101 ... Image data storage area 102 ... Image generation history information storage area 103 ... Attached information storage area 121 ... Optical circuit 122 ... Image acquisition circuit 123 ... Image processing circuit 124 ... Control circuit 125 ... Lens 126 ... Selection / determination button 127 ... Liquid crystal display 128 ... CCD
129 ... Aperture 130 ... Flash 150 ... CPU
151 ... RAM
152 ... HDD
153 ... Memory card slot 154 ... I / O terminal 211 ... Print head CV ... Cable GD ... Image data GF ... Image file GI ... Image generation history information MC ... Memory card P ... Printing paper PC ... Computer IMG14 ... Image IMG17 ... Image Cpt14 ... Carpet Cpt17 ... Carpet Bd14 ... Bird Bd17 ... Bird CBA ... Color fog calculation area CBA1 ... Color fog calculation area CBA2 ... Color fog calculation area

Claims (17)

An output device that outputs an image using image data generated by the image generation device and image generation history information associated with the image data at least including operation information of the image generation device at the time of generating the image data Because
When the image generation history information includes subject brightness information related to the brightness of the subject at the time of generating the image data, the image quality of the image data is adjusted using the degree of brightness of the subject obtained from the subject brightness information. An image quality adjustment unit capable of
An image output unit that outputs an image according to the image data in which the image quality is adjusted;
An output device comprising: The output device according to claim 1,
The output device, wherein the image quality adjustment unit executes a color balance adjustment process of the image data using a degree of brightness of the subject. The output device according to claim 2,
The image quality adjustment unit
In at least a part of the range of the brightness level of the subject in which the brightness level is small,
An output device that adjusts the intensity of the color balance adjustment processing so that the intensity decreases as the brightness level decreases. The output device according to claim 3,
The image quality adjustment is
(I) processing for determining a color fog amount indicating a degree of color deviation in the image data by analyzing the image data;
(Ii) processing for determining a processing amount of the color balance adjustment processing based on the size of the color fogging amount;
(Iii) a process of executing the color balance adjustment process according to the determined processing amount;
Including
Adjusting the intensity of the color balance adjustment process is realized by changing a process parameter that affects the result of at least one of the processes (i) and (ii). The output device according to claim 4,
The image quality adjustment unit
An output device that determines the size of the color fog amount using a pixel value in a region close to an achromatic color in the image data. The output device according to claim 5,
The image quality adjustment unit
An output device that determines the magnitude of the color cast amount using pixel values of a region excluding a region having a predetermined hue among regions close to the achromatic color. The output device according to any one of claims 4 to 6,
The output device wherein the adjustment of the intensity of the color balance adjustment processing is realized by changing a processing parameter representing a ratio of the processing amount of the color balance adjustment processing to the size of the color fog amount. The output device according to any one of claims 5 to 7,
Adjusting the intensity of the color balance adjustment process is realized by changing a processing parameter indicating a range of an area close to the achromatic color. The output device according to any one of claims 2 to 8,
When the image generation history information includes light emission information of the auxiliary light source at the time of generating the image data, the image quality adjustment unit uses the light emission information to irradiate light by the auxiliary light source at the time of image data generation. An output device that performs color balance adjustment processing using the degree of brightness of the subject when it is determined whether or not light irradiation has not been performed. The output device according to any one of claims 1 to 9,
When the image generation history information further includes light emission information of an auxiliary light source at the time of generating the image data and information on a distance between the subject of the image data and the image generating device at the time of generating the image data.
The image quality adjustment unit makes a determination using the light emission information that light irradiation by the auxiliary light source has been performed, and further, a distance from the subject is equal to or less than a predetermined distance threshold, An output device capable of executing image quality adjustment processing suitable for a human image when the degree of brightness is equal to or greater than a predetermined luminance threshold. The output device according to any one of claims 1 to 10,
The image quality adjustment unit
When the image generation history information includes photometric luminance information related to the result of measuring the brightness of the subject at the time of generating the image data, it is possible to calculate the degree of brightness of the subject using the photometric luminance information. There is an output device. The output device according to any one of claims 1 to 10,
The image quality adjustment unit
When the image generation history information includes information about the aperture value of the image generation apparatus and information about the shutter speed at the time of generating the image data, the brightness value of the subject is determined using the aperture value and the shutter speed. An output device capable of calculating the degree. The output device according to any one of claims 1 to 10,
The image quality adjustment unit
When the image generation history information includes information on the aperture value of the image generation device, information on the shutter speed, and information on the sensitivity of the optical circuit when the image data is generated, the aperture value, the shutter speed, and the An output device capable of calculating the degree of brightness of the subject using sensitivity. An image that processes image data using image data generated by the image generation device and image generation history information that includes at least operation information of the image generation device at the time of the image data generation and is associated with the image data A data processing device,
When the image generation history information includes subject brightness information related to the brightness of the subject at the time of generating the image data, the image quality of the image data is adjusted using the degree of brightness of the subject obtained from the subject brightness information. An image data processing apparatus comprising an image quality adjustment unit capable of performing the above. An image that processes image data using image data generated by the image generation device and image generation history information that includes at least operation information of the image generation device at the time of the image data generation and is associated with the image data A data processing method,
When the image generation history information includes subject brightness information related to the brightness of the subject at the time of generating the image data, the image quality of the image data is adjusted using the degree of brightness of the subject obtained from the subject brightness information. An image data processing method including the step of: Computer processing image data using image data generated by the image generation device and image generation history information associated with the image data at least including operation information of the image generation device at the time of image data generation A computer program for executing the program,
When the image generation history information includes subject brightness information related to the brightness of the subject at the time of generating the image data, the image quality of the image data is adjusted using the degree of brightness of the subject obtained from the subject brightness information. A computer program for causing a computer to realize a function to perform the function.   The computer-readable recording medium which recorded the computer program of Claim 16.

Images