JP4042803B2 - Output image adjustment of image data - Google Patents

Description

  The present invention relates to an image adjustment technique for adjusting the image quality of image data.

  Conventionally, a user selects a shooting scene such as a person or a landscape with a camera or a printer driver, and image quality adjustment suitable for each image is performed based on the selection (for example, Japanese Patent Laid-Open No. 11-146219, Japanese Patent Laid-Open Nos. 2001-169135 and 2001-298631).

  However, according to the conventional technique, when a shooting scene such as a person or a landscape is selected, the user needs to perform a complicated operation during shooting or printing.

  For example, when a shooting scene is selected by a camera, it is necessary to switch the shooting scene each time a person image and a landscape image are shot continuously. In addition, when a shooting scene is selected by a printer driver, it is necessary to select a shooting scene for each image. Therefore, when a person image and a landscape image are mixed, complicated operations are involved.

  The present invention has been made to solve the above problem, and an object thereof is to automatically perform appropriate image quality adjustment on individual image data.

An apparatus according to an aspect of the present invention uses image data generated by an image generation apparatus and image generation history information associated with the image data and including at least subject area information indicating a subject area in an image. An output device for outputting
  An image quality adjustment unit that calculates a hue of each pixel in the subject area and automatically executes an image quality adjustment process suitable for the image based on a ratio of pixels having a hue of a specific color gamut;
  An image output unit that outputs an image according to the image data in which the image quality is adjusted;
With
  The image generation history information includes shooting mode information indicating a shooting mode used when the image data is generated by shooting in the image generation device,
  The image quality adjustment processing includes a plurality of types of processing including two or more of contrast correction, brightness correction, color balance correction, saturation correction, sharpness correction, memory color correction, and noise removal processing. Including
  When the shooting mode information included in the image generation history information indicates a shooting mode other than a standard scene, the image quality adjustment unit does not execute all of the plurality of types of processes included in the image quality adjustment process. It is configured.

  The output device according to the present invention automatically determines whether or not to perform image quality adjustment suitable for an image including an object specified by the specific color gamut based on subject area information indicating a subject area in the image. Since the image quality adjustment processing is performed in accordance with the determination, appropriate image quality adjustment can be automatically performed on individual image data.

  In the output device, it is preferable that the specific color gamut is a skin color gamut, and the image quality adjustment process is a process suitable for a human image.

  Thus, it is possible to automatically determine whether or not to perform image quality adjustment suitable for a person image based on subject area information indicating the subject area in the image, and perform image quality adjustment processing according to the determination.

  The image quality adjustment process suitable for the person image may be executed only for a part of the area including the subject area.

  In addition, the image quality adjustment is performed only on a part of the processing target area in the image that includes a pixel that exists in the subject area and has a color of the specific color gamut. It may be a thing. Further, the processing target area is a first type of pixel that exists in the subject area and has a color of the specific color gamut, and is located outside the subject area and is continuous to the first type of pixel and the specific area. And a second type of pixel having a color in the color gamut.

  In the output device, when the image quality adjustment process suitable for the human image is not executed, the image quality adjustment unit is configured such that the proportion of pixels having a hue in the green region is greater than a second predetermined threshold value, or When the ratio of the pixels having the hue of the sky gamut is larger than the third predetermined threshold value, it is preferable to execute the image quality adjustment process suitable for the landscape image.

  Thus, when the image quality adjustment suitable for the person image is not executed, whether or not the image quality adjustment suitable for the landscape image is executed is automatically determined based on the subject area information indicating the subject area in the image. Image quality adjustment processing can be performed according to the determination.

  The present invention can be realized in various forms, for example, an image output method and an image output device, an image data processing method and an image data processing device, and a function for implementing these methods or devices. The present invention can be realized in the form of a computer program, a recording medium recording the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Hereinafter, output image adjustment of an image file according to the present invention will be described based on several embodiments with reference to the drawings in the following order.
A. Image output system configuration:
B. Image file structure:
C. Configuration of image output system that can use image files:
D. Image processing in digital still cameras:
E. Image processing in the printer:
F. Example of automatic image quality adjustment processing:
G. Configuration of an image output system using an image data processing device:
H. Variation:

A. Image output system configuration:
FIG. 1 is an explanatory diagram showing an example of an image output system to which an output device as an embodiment of the present invention can be applied. The image output system 10 includes a digital still camera 12 as an image generation apparatus that generates an image file, and a printer 20 as an image output apparatus. The image file generated in the digital still camera 12 is sent to the printer 20 through the cable CV or by directly inserting the memory card MC storing the image file into the printer 20. The printer 20 executes image quality adjustment processing of image data based on the read image file and outputs an image. As the output device, in addition to the printer 20, a monitor 14 such as a CRT display or LCD display, a projector, or the like can be used. Hereinafter, description will be made based on a case where the printer 20 including the image quality adjustment unit and the image output unit is used as an output device and the memory card MC is directly inserted into the printer 20.

  FIG. 2 is a block diagram showing a schematic configuration of the digital still camera 12. The digital still camera 12 of this embodiment includes an optical circuit 121 for collecting optical information, an image acquisition circuit 122 for controlling the optical circuit to acquire an image, and a process for processing the acquired digital image. An image processing circuit 123, a flash 130 as an auxiliary light source, and a control circuit 124 for controlling each circuit are provided. The control circuit 124 includes a memory (not shown). The optical circuit 121 includes a lens 125 that collects optical information, a diaphragm 129 that adjusts the amount of light, and a CCD 128 that converts optical information that has passed through the lens into image data.

  The digital still camera 12 stores the acquired image in the memory card MC. As a storage format of image data in the digital still camera 12, a JPEG format is generally used, but other storage formats such as a TIFF format, a GIF format, a BMP format, and a RAW data format can be used. .

  The digital still camera 12 also includes a selection / determination button 126 for setting various shooting conditions and a liquid crystal display 127. The liquid crystal display 127 is used when a captured image is previewed or an aperture value or the like is set using the selection / determination button 126.

  When shooting is performed by the digital still camera 12, the image data and the image generation history information are stored in the memory card MC as an image file. The image generation history information can include setting values of parameters such as an aperture value at the time of shooting (at the time of image data generation) (details will be described later).

B. Image file structure:
FIG. 3 is an explanatory diagram conceptually showing an example of the internal configuration of an image file that can be used in this embodiment. The image file GF includes an image data storage area 101 for storing image data GD and an image generation history information storage area 102 for storing image generation history information GI. The image data GD is stored, for example, in the JPEG format, and the image generation history information GI is stored, for example, in the TIFF format (a format in which data and data areas are specified using tags). Note that the terms “file structure” and “data structure” in the present embodiment mean a file or data structure in a state where the file or data is stored in the storage device.

The image generation history information GI is information related to an image when image data is generated (taken) in an image generation apparatus such as the digital still camera 12, and includes the following setting values.
-Subject distance.
-Subject distance range.
-Subject area.
・ Exposure time.
・ Aperture value.
ISO speed rate (ISO sensitivity).
・ Shooting mode.
·Manufacturer's name.
·Model name.
-Gamma value.

  The image file GF of the present embodiment basically only needs to include the image data storage area 101 and the image generation history information storage area 102 described above, and has a file structure in accordance with an already standardized file format. Can take. The case where the image file GF according to the present embodiment is adapted to the Exif file format will be specifically described below.

  An Exif file has a file structure in accordance with an image file format standard (Exif) for digital still cameras, and its specifications are determined by the Japan Electronics and Information Technology Industries Association (JEITA). The Exif file format includes a JPEG image data storage area for storing image data in the JPEG format and an attached information storage area for storing various information related to the stored JPEG image data, as in the conceptual diagram shown in FIG. And. The JPEG image data storage area corresponds to the image data storage area 101 in FIG. 3, and the attached information storage area corresponds to the image generation history information storage area 102. The attached information storage area stores image generation history information related to JPEG images such as shooting date and time, aperture value, and subject distance.

  FIG. 4 is an explanatory diagram for explaining an example of the data structure of the attached information storage area 103. In the Exif file format, hierarchical tags are used to specify data areas. Each data area can include therein a plurality of lower data areas specified by lower tags. In FIG. 4, an area surrounded by a square represents one data area, and a tag name is written at the upper left. This embodiment includes three data areas whose tag names are APP0, APP1, and APP6. The APP1 data area includes two data areas whose tag names are IFD0 and IFD1. The IFD0 data area includes three data areas whose tag names are PM, Exif, and GPS. The data and data area are stored according to a prescribed address or offset value, and the address or offset value can be retrieved by tag name. On the output device side, data corresponding to desired information can be acquired by designating an address or offset value corresponding to the desired information.

  FIG. 5 is an explanatory diagram for explaining an example of the data structure (data tag name and parameter value) of the Exif data area that can be referred to by tracing the tag names in the order of APP1-IFD0-Exif in FIG. . The Exif data area can include a data area whose tag name is MakerNote, as shown in FIG. 4, and the MakerNote data area can include a larger number of data, but is not shown in FIG. .

  As shown in FIG. 5, the Exif data area stores parameter values relating to information such as a subject area, subject distance, exposure time, aperture value, and ISO speed rate. In this embodiment, the subject area can be used as information indicating the subject area.

  FIG. 6 is a diagram showing a subject area 510 in the image 500. As shown in the figure, the subject area is represented by the center coordinates with the upper left corner of the image as the origin and the diameter of the area. The subject area may be a rectangular area. In this case, the area range is expressed by a height and a width.

  Information associated with the image data is appropriately stored in an area other than the Exif data area in FIG. For example, the manufacturer name and model name as information for specifying the image generation apparatus are stored in the data area whose tag name is IFD0.

C. Image output device configuration:
FIG. 7 is a block diagram illustrating a schematic configuration of the printer 20 of the present embodiment. The printer 20 is a printer that can output an image. For example, an ink jet that ejects four colors of ink of cyan C, magenta Mg, yellow Y, and black K onto a printing medium to form a dot pattern. Printer. In addition, an electrophotographic printer that forms an image by transferring and fixing toner onto a printing medium can also be used. In addition to the above four colors, light cyan LC having a lighter density than cyan C, light magenta LM having a lighter density than magenta Mg, and dark yellow DY having a darker density than yellow Y may be used for the ink. When performing monochrome printing, only black K may be used, or red R or green G may be used. The type of ink or toner to be used can be determined according to the characteristics of the output image.

  As shown in the figure, the printer 20 drives a print head 211 mounted on the carriage 21 to discharge ink and form dots, and a mechanism that causes the carriage 21 to reciprocate in the axial direction of the platen 23 by the carriage motor 22. And a mechanism for transporting the printing paper P by the paper feed motor 24, and a control circuit 30. With these mechanisms, the printer 20 functions as an image output unit. The mechanism for reciprocating the carriage 21 in the axial direction of the platen 23 is an endless drive between the carriage motor 22 and the sliding shaft 25 that holds the carriage 21 that is laid in parallel with the platen 23 axis. A pulley 27 that stretches the belt 26, a position detection sensor 28 that detects the origin position of the carriage 21, and the like. The mechanism for transporting the printing paper P includes a platen 23, a paper feed motor 24 that rotates the platen 23, a paper feed auxiliary roller (not shown), and a gear train that transmits the rotation of the paper feed motor 24 to the platen 23 and the paper feed auxiliary roller. (Not shown).

  The control circuit 30 appropriately controls the movement of the paper feed motor 24, the carriage motor 22, and the print head 211 while exchanging signals with the operation panel 29 of the printer. The printing paper P supplied to the printer 20 is set so as to be sandwiched between the platen 23 and the paper feed auxiliary roller, and is fed by a predetermined amount according to the rotation angle of the platen 23.

  The carriage 21 has a print head 211 and can be mounted with an ink cartridge of available ink. A nozzle for ejecting usable ink is provided on the lower surface of the print head 211 (not shown).

  FIG. 8 is a block diagram showing the configuration of the printer 20 with the control circuit 30 of the printer 20 as the center. Inside the control circuit 30, there are a CPU 31, a PROM 32, a RAM 33, a memory card slot 34 for acquiring data from the memory card MC, a peripheral device input / output for exchanging data with the paper feed motor 24, the carriage motor 22, and the like. A unit (PIO) 35, a drive buffer 37, and the like are provided. The drive buffer 37 is used as a buffer for supplying dot on / off signals to the print head 211. These are connected to each other by a bus 38 and can exchange data with each other. The control circuit 30 is also provided with a transmitter 39 that outputs a drive waveform at a predetermined frequency, and a distribution output device 40 that distributes the output from the transmitter 39 to the print head 211 at a predetermined timing.

  The control circuit 30 outputs dot data to the drive buffer 37 at a predetermined timing while synchronizing with the movement of the paper feed motor 24 and the carriage motor 22. Further, the control circuit 30 reads the image file from the memory card MC, analyzes the attached information, and performs image processing based on the obtained image generation history information. That is, the control circuit 30 functions as an image quality adjustment unit. A detailed flow of image processing executed by the control circuit 30 will be described later.

D. Image processing in digital still cameras:
FIG. 9 is a flowchart showing a flow of processing for generating the image file GF in the digital still camera 12.

  The control circuit 124 (FIG. 2) of the digital still camera 12 generates the image data GD in response to a shooting request, for example, pressing of the shutter button (step S100). When parameter values such as an aperture value, ISO sensitivity, and shooting mode are set, image data GD is generated using the set parameter values.

  The control circuit 124 stores the generated image data GD and the image generation history information GI as an image file GF in the memory card MC (step S110), and ends this processing routine. The image generation history information GI is automatically set such as parameter values used at the time of image generation such as aperture value and ISO sensitivity, parameter values that can be arbitrarily set such as a shooting mode, manufacturer name, and model name. Contains parameter values. The image data GD is converted from the RGB color space to the YCbCr color space, JPEG compressed, and stored as an image file GF.

  Through the above processing executed in the digital still camera 12, image generation history information GI including each parameter value at the time of image data generation is set in the image file GF stored in the memory card MC together with the image data GD. The Rukoto.

E. Image processing in the printer:
FIG. 10 is a flowchart illustrating a processing routine of image processing in the printer 20 of the present embodiment. In the following description, the case where the memory card MC storing the image file GF is directly inserted into the printer 20 will be described. When the memory card MC is inserted into the memory card slot 34, the CPU 31 of the control circuit 30 (FIG. 8) of the printer 20 reads the image file GF (FIG. 3) from the memory card MC (step S200). Next, in step S210, the CPU 31 retrieves image generation history information GI indicating information at the time of image data generation from the attached information storage area of the image file GF. When the image generation history information GI can be found (step S220: Y), the CPU 31 acquires and analyzes the image generation history information GI (step S230). The CPU 31 executes image processing to be described later based on the analyzed image generation history information GI (step S240), outputs the processed image (step S250), and ends this processing routine.

  On the other hand, an image file generated using a drawing application or the like does not include image generation history information GI having information such as an aperture value. If the image generation history information GI cannot be found (step S220: N), the CPU 31 performs standard processing (step S260), outputs the processed image (step S250), and ends this processing routine. .

  FIG. 11 is a flowchart showing a processing routine of image processing (corresponding to step S240 in FIG. 10) based on image generation history information. The CPU 31 of the control circuit 30 (FIG. 8) of the printer 20 extracts image data GD from the read image file GF (step S300).

  The digital still camera 12 stores the image data GD as a JPEG format file as described above, and the JPEG format file stores image data using the YCbCr color space. In step S310, the CPU 31 executes a calculation using a 3 × 3 matrix S in order to convert image data based on the YCbCr color space into image data based on the RGB color space. This matrix operation is expressed by, for example, the following arithmetic expression.

  When the color space of the image data generated by the digital still camera 12 is wider than a predetermined color space, for example, the sRGB color space, the image data based on the RGB color space obtained in step S310 is displayed in the RGB color space. Valid data may be included outside the definition area. If it is specified in the image generation history information GI that the data outside the definition area is treated as valid data, the data outside the definition area is held as it is, and the subsequent image processing is continued. If the data outside the definition area is not designated as valid data, the data outside the definition area is clipped into the definition area. For example, when the definition area is 0 to 255, negative data less than 0 is rounded to 0, and 256 or more data is rounded to 255. When the color space that can be expressed by the image output unit is not wider than a predetermined color space, for example, the sRGB color space, it is preferable to perform clipping in the definition area regardless of the designation in the image generation history information GI. As such a case, for example, there are cases where the color space that can be expressed is output to a CRT that is an sRGB color space.

  Next, in step S320, the CPU 31 executes gamma correction and calculation using the matrix M to convert image data based on the RGB color space into image data based on the XYZ color space. The image file GF can include a gamma value and color space information at the time of image generation. When the image generation history information GI includes such information, the CPU 31 acquires a gamma value of the image data from the image generation history information GI, and executes a gamma conversion process of the image data using the acquired gamma value. Further, the CPU 31 acquires the color space information of the image data from the image generation history information GI, and executes the matrix calculation of the image data using the matrix M corresponding to the color space. When the image generation history information GI does not include a gamma value, the gamma conversion process can be executed using a standard gamma value. In addition, when the image generation history information GI does not include color space information, a matrix operation can be performed using a standard matrix M. As these standard gamma values and matrix M, for example, gamma values and matrices for the sRGB color space can be used. This matrix operation is, for example, an arithmetic expression shown below.

  The color space of the image data obtained after execution of the matrix operation is an XYZ color space. The XYZ color space is an absolute color space, and is a device-independent color space that does not depend on a device such as a digital still camera or a printer. Therefore, by performing color space conversion via the XYZ color space, device-independent color matching can be performed.

Next, in step S330, the CPU 31 performs calculation using the matrix N- ¹ and inverse gamma correction, and converts the image data based on the XYZ color space into image data based on the wRGB color space. When executing the inverse gamma correction, the CPU 31 acquires the gamma value on the printer side from the PROM 32, and executes the inverse gamma conversion processing of the image data using the inverse of the acquired gamma value. Further, the CPU 31 acquires a matrix N ^-1 corresponding to the conversion from the XYZ color space to the wRGB color space from the PROM 32, and executes a matrix operation of the image data using the matrix N ^-1 . This matrix operation is, for example, an arithmetic expression shown below.

  Next, in step S340, the CPU 31 executes automatic image quality adjustment processing. In the automatic image quality adjustment process in the present embodiment, the image quality automatic image quality adjustment process is executed using the image generation history information (subject area information) included in the image file GF. The automatic image quality adjustment process will be described later.

  Next, in step S350, the CPU 31 executes CMYK color conversion processing for printing and halftone processing. In the CMYK color conversion process, the CPU 31 refers to a look-up table (LUT) for conversion from the wRGB color space to the CMYK color space stored in the PROM 32, and changes the color space of the image data from the wRGB color space to the CMYK color space. Change to That is, image data composed of RGB gradation values is used in the printer 20, for example, C (Cyan), M (Magenta), Y (Yellow), K (Black), LC (LightCyan), LM (LightMagenta). Are converted into image data composed of gradation values of the six colors.

  In the halftone process, the CPU 31 performs a so-called halftone process to generate halftone image data from the color-converted image data. The halftone image data is rearranged in the order to be sent to the drive buffer 37 (FIG. 8), becomes final print data, and this processing routine ends. The image data processed by this processing routine is output in step S250 of the image processing routine shown in FIG.

F. Example of automatic image quality adjustment processing:
FIG. 12 is a flowchart showing the processing routine of the first embodiment of the automatic image quality adjustment processing (corresponding to step S340 in FIG. 11). The CPU 31 (FIG. 8) analyzes the image generation history information GI and acquires subject area information (step S400). Next, in step S410, the CPU 31 calculates the hue H of each pixel in the subject area. In this embodiment, in order to calculate the hue H, conversion from the RGB color system to the HSI (Hue, Saturation, Intensity) color system is performed. The HSI color system is preferable in that the hue is not affected by the brightness of the image because the hue H and the lightness I are independent. It is also possible to use other appropriate color systems such as an HSV (Hue, Saturation, Value) color system and an HSL (Hue, Saturation, Luminance) color system.

  An expression for calculating the hue H of the HSI color system will be described below. If I = max {R, G, B} and i = min {R, G, B}, the hue H is indefinite (achromatic) when I = 0, and is calculated by the following equation when I ≠ 0. it can.

  However, when the hue H <0, 2π is added to the calculated H value. The range of the hue H is 0 to 2π. In this embodiment, the hue H is expressed by setting the range to 0 ° to 360 °.

  Next, for all the pixels in the subject area, the frequencies of pixels having hues included in the skin color, green, and sky blue color gamut are respectively calculated (step S420). FIG. 13 shows a subject area in an image and a histogram of hues of all pixels included in the subject area. Specifically, (A) shows a person image, and (B) shows a histogram in the case of a person image. In this embodiment, the range where the hue is 0 ° to 30 ° is the skin color gamut SR, the range where the hue is 100 ° to 130 ° is the green region GR, and the range where the hue is 230 ° to 260 ° is the sky gamut BR. . The hue range is not necessarily the above-described range, and a different range may be set.

  Next, in step S430, the ratio r (Skin) of the flesh color pixels to the total number of pixels in the subject area is calculated, and it is determined whether or not this ratio r (Skin) is larger than the first threshold value Th1. If r (Skin)> Th1, that is, it is determined that the image is a person image, the process proceeds to step S440, a process suitable for the person scene is executed, and the automatic image quality adjustment process ends.

  On the other hand, if r (Skin)> Th1 does not hold, the process proceeds to step S450, and the ratio r (Green) of the green pixels to the total number of pixels in the subject area is larger than the second threshold Th2. It is determined whether or not. If r (Green)> Th2, that is, if it is determined that the image is a landscape image including trees and forests, the process proceeds to step S470, a process suitable for the landscape scene is executed, and the automatic image quality adjustment process is terminated.

  On the other hand, if r (Green)> Th2 does not hold, the process proceeds to step S460, and the ratio r (Sky) of the sky blue pixels to the total number of pixels in the subject area is larger than the third threshold Th3. It is determined whether or not. Here, when it is determined that r (Sky)> Th3, that is, it is a landscape image including the sky, the process proceeds to step S470, and processing suitable for the landscape scene is executed, and the automatic image quality adjustment processing ends. , R (Sky)> Th3 does not hold, the process proceeds to step S480, processing suitable for the standard scene is executed, and automatic image quality adjustment is terminated.

  The above-described threshold values Th1, Th2, and Th3 are preferably set so as to increase the accuracy of determination in steps S430, S450, and S460. Further, a specific value may be set as an initial value so that the user can change the value.

  In this embodiment, processing suitable for each scene is executed as shown in FIG. For example, if it is determined that the image is a human image, the contrast is slightly soft, the brightness is slightly bright, the color balance is standard, the saturation is slightly low, and the sharpness is slightly weak. Set to Since the skin color is designated as the memory color, the skin color correction is executed using the skin color data stored in advance. Furthermore, noise processing is turned off. By executing the image quality adjustment process in this way, the atmosphere of the image is softened and the skin color of the person is adjusted to a preferable skin color. However, the image quality adjustment process does not necessarily follow the setting shown in FIG. 14, and another setting may be used. In the first embodiment, the entire image is the processing target for automatic image quality adjustment. Instead, only the subject region may be the processing target. Alternatively, only a part of the area including the subject area may be the processing target.

  As described above, in the first embodiment of automatic image quality adjustment, it is possible to automatically select a shooting scene for each image data and perform appropriate image quality processing. As a result, there is an advantage that it is not necessary for the user to manually perform complicated operations during shooting or printing.

  FIG. 15 shows a processing routine in the second embodiment of the automatic image quality adjustment processing. FIG. 16 shows the processing contents of the second embodiment. In step S400, as in the first embodiment, information related to the subject area 510a (FIG. 16A) is acquired from the image generation history information. In the example of FIG. 16A, the subject area 510a is rectangular. In step S401, the color gamut of the subject area is determined by analyzing the color of the subject area. Here, “the color gamut of the subject area” means a color range of pixels occupying a large number in the subject area (that is, a representative color range of the subject area). As a method for calculating the color gamut of the subject area, for example, as in the first embodiment, the color of each pixel is converted into the HSI color system, a histogram relating to the hue H (FIG. 13B) is obtained, and the frequency A method in which a large color gamut is used as the color gamut of the subject area can be used. Alternatively, an average color of the subject area may be calculated, and a color included in a predetermined color range centered on the average color may be set as the color gamut of the subject area. In the example of FIG. 16A, since the subject area 510a is set in the face of a person, a skin color gamut SR as shown in FIG. 13B is determined as the color gamut of the subject area. Note that as the color gamut of the subject area, one selected from a plurality of predetermined color gamut candidates (for example, skin color gamut SR, green color area GR, sky color gamut BR) may be used, or an actual color gamut may be used. A representative color gamut calculated from the color of the pixel in the subject area may be used.

  Hereinafter, a color within the color gamut of the subject area is referred to as a “subject color”, and a pixel having the subject color is referred to as a “subject color pixel”. In step S402, a subject color pixel existing in the subject region and a subject color pixel outside the subject region that is continuous to the subject color pixel are searched, and a processing target region composed of the plurality of subject color pixels is determined. FIG. 16B shows a processing target area 512 (shown by hatching) obtained in this way. In this example, since the color gamut of the subject area 510a is a flesh color, pixels having a flesh color are searched from within the subject area 510a. Among the subject color pixels that exist outside the subject area 510 a, pixels that are continuous to the subject color pixels in the subject area 510 a are also extracted as pixels that constitute the processing target area 512. Here, “pixels are continuous” means that they are adjacent to either the top, bottom, left, or right. Here, it should be noted that when the subject area 510a is larger than the face, the processing target area 512 is smaller than the subject area 510a.

  It should be noted that the processing target area 512 may be composed only of subject color pixels, or all pixels included inside the outermost contour of the area composed of only subject color pixels may be the processing target area 512. In the former method, for example, the black eye area existing in the face is not included in the processing target area 512, but in the latter method, the black eye area is also included in the processing target area 512. If the processing target area 512 is determined by the former method, there is an advantage that it is possible to prevent the color of a pixel having a color other than the subject color from being changed excessively. On the other hand, if the processing target area 512 is determined by the latter method, there is an advantage that the image quality in the processing target area 512 can be kept harmonious even after the image quality adjustment.

  In step S403, image quality adjustment suitable for the color gamut of the subject is performed on the processing target area 512 determined in this way. Specifically, when the subject color gamut is flesh-colored, a process suitable for a person scene (the process in step S440 in FIG. 12) is executed. If the subject color gamut is green or sky blue, a process suitable for a landscape scene (the process in step S470 in FIG. 12) is executed. If the subject color gamut is a color gamut other than these, a process suitable for the standard scene (the process in step S480 in FIG. 12) is executed.

  As described above, in the second embodiment, the color gamut of the subject region is determined, and a region including a pixel in the subject region having a color (subject color) within the color gamut and a pixel having a subject color continuous thereto. Since the image quality adjustment process is executed for the processing object, image quality adjustment suitable for the color gamut of the subject area can be performed only for a part of the image. In the second embodiment, for example, when the second skin color area is included in a place away from the subject area, the color of the subject area is not affected by the color of the second skin color area. Accordingly, there is an advantage that image quality adjustment in the vicinity of the subject area can be performed.

G. Configuration of an image output system using an image data processing device:
FIG. 17 is an explanatory diagram showing an example of an image output system to which the image data processing apparatus as one embodiment of the present invention can be applied. The image output system 10B includes a digital still camera 12 as an image generation apparatus that generates an image file, a computer PC that executes image quality adjustment processing based on the image file, and a printer 20B as an image output apparatus that outputs an image. I have. The computer PC is a commonly used type of computer and functions as an image data processing device. As the image output device, in addition to the printer 20B, a monitor 14B such as a CRT display or LCD display, a projector, or the like can be used. In the following description, it is assumed that the printer 20B is used as an image output device. The present embodiment is different from the above-described image output system embodiment (FIG. 1) in that an image data processing apparatus including an image quality adjustment unit and an image output apparatus including an image output unit are configured independently. Note that a computer PC as an image data processing apparatus and a printer including an image output unit can be called an “output apparatus” in a broad sense.

  The image file generated in the digital still camera 12 is sent to the computer PC via the cable CV or by directly inserting the memory card MC storing the image file into the computer PC. The computer PC executes image quality adjustment processing of image data based on the read image file. The image data generated by the image quality adjustment process is sent to the printer 20B via the cable CV and output by the printer 20B.

  The computer PC includes a CPU 150 that executes a program that realizes the above-described image quality adjustment processing, a RAM 151 that temporarily stores calculation results and image data of the CPU 150, an image quality adjustment processing program, a lookup table, and an aperture value table. For example, a hard disk drive (HDD) 152 that stores data necessary for image quality adjustment processing is provided. The CPU 150, the RAM 151, and the HDD 152 function as an image quality adjustment unit. Furthermore, the computer PC includes a memory card slot 153 for mounting a memory card MC and an input / output terminal 154 for connecting a connection cable from the digital still camera 12 or the like.

  The image file GF generated by the digital still camera 12 is provided to the computer PC via a cable or a memory card MC. When an image retouching application or an image data processing application program such as a printer driver is activated by a user operation, the CPU 150 executes an image processing routine (FIG. 10) for processing the read image file GF. Further, the image data processing application program is automatically started by detecting the insertion of the memory card MC into the memory card slot 153 or the connection of the digital still camera 12 via the cable to the input / output terminal 154. Also good.

  The image data processed by the CPU 150 is sent to the image output device, for example, the printer 20B, instead of being output in step S250 of the image processing routine (FIG. 10), and the image output device receiving the image data receives the image data. Run the output.

  In this embodiment, since image processing is performed using the image quality adjustment unit provided in the computer PC, it is possible to use an image output device that does not include the image quality adjustment unit. When the image output device includes an image quality adjustment unit, the computer PC sends image data to the image output device without performing image processing, and the image quality adjustment unit of the image output device performs image processing. Also good.

  As described above, in each of the above-described embodiments, it is possible to automatically determine a shooting scene and execute an appropriate image quality adjustment process according to the determination, so that a high-quality output result can be easily obtained. .

H. Variation:
The present invention is not limited to the above-described embodiments, and can be implemented in various forms without departing from the gist thereof. For example, the following modifications are possible.

H1. Modification 1:
In the above embodiment, when the image file GF includes shooting mode information, the shooting mode is determined before proceeding to the automatic image quality processing routine, and the automatic image quality processing is not executed when a shooting mode other than the standard scene is selected. A configuration is preferable. By doing so, there is an advantage that it is possible to prevent the same image quality processing from being performed twice when the user manually sets the shooting mode during shooting.

H2. Modification 2:
In the above-described embodiment, the automatic image quality adjustment process is performed on all images including the subject area. However, the user may select whether to execute the automatic image quality adjustment process.

H3. Modification 3:
It is preferable to adjust the white balance before executing the automatic image quality adjustment processing described with reference to FIGS. Thereby, the hue shift due to the difference in the color temperature of the light source can be corrected, so that the accuracy of the determinations in steps S430, S450, and S460 in FIG. 12 can be increased.

H4. Modification 4:
When the image file GF does not include the gamma value of the image data and the color space information, the color space conversion process (step S320 and step S330) in the image processing routine shown in FIG. 11 can be omitted. FIG. 18 is a flowchart showing an image processing routine when the color space conversion process is omitted. The image data extracted in step S500 is converted from image data based on the YCbCr color space to image data based on the RGB color space in step S510. Next, in step S520, automatic image quality adjustment processing using the image data obtained in step S510 is executed. Next, in S530, CMYK color conversion processing for printing and halftone processing are executed.

H5. Modification 5:
In each of the embodiments described above, the automatic image quality adjustment processing is executed after the color space conversion is executed. However, as shown in FIG. 19, the color space conversion (step S340) is executed after the automatic image quality adjustment processing (step S340) is executed. S320, S330) may be executed.

H6. Modification 6:
In each of the above embodiments, a printer is used as the image output unit, but an image output unit other than the printer can be used. FIG. 18 is a flowchart illustrating a processing routine of image processing based on image generation history information when a CRT is used as an image output unit. Unlike the flowchart in which the printer shown in FIG. 11 is an image output unit, CMYK color conversion processing and halftone processing are omitted. Further, since the CRT can express the RGB color space of the image data obtained by executing the matrix operation (S), the color space conversion process is also omitted. If the image data based on the RGB color space obtained in step S610 includes data outside the definition area of the RGB color space, step S620 is executed after the data outside the definition area is clipped. When the color space that can be used by the image output unit is different from the RGB color space, the color conversion process to the color space that can be used by the image output unit is performed in the same manner as when the CMYK color conversion process is executed when using a printer And the resulting image is output from the image output unit.

H7. Modification 7:
In the above embodiment, an Exif format file has been described as a specific example of the image file GF. However, the format of the image file according to the present invention is not limited to this, and any other format can be used. In general, the image file only needs to include image data generated by the image generation apparatus and image generation history information GI describing conditions (information) when generating image data. With such a file, the image quality of the image data generated by the image generation device can be automatically adjusted appropriately and output from the output device. The subject area is not limited to the Exif format parameters described above, but can be expressed by various types of parameters and data. For example, parameters indicating the focus position or focus area of autofocus may be used. In addition, when the user can arbitrarily specify the position and shape of the subject area at the time of imaging, parameters indicating them may be used as parameters representing the subject area.

H8. Modification 8:
The values of the matrices S, N ⁻¹ , and M in each formula are merely examples, and can be appropriately changed according to the color space based on the image file, the color space that can be used by the image output unit, and the like.

H9. Modification 9:
In the above-described embodiment, the digital still camera 12 is used as the image generation apparatus. However, an image file can be generated using an image generation apparatus such as a scanner or a digital video camera.

H10. Modification 10:
In the above embodiment, the case where the image data GD and the image generation history information GI are included in the same image file GF has been described as an example. However, the image data GD and the image generation history information GI are not necessarily included in the same file. It need not be stored. That is, the image data GD and the image generation history information GI need only be associated with each other. For example, association data that associates the image data GD and the image generation history information GI is generated, and one or a plurality of image data and the image generation history The information GI may be stored in independent files, and the image generation history information GI associated with processing the image data GD may be referred to. In such a case, the image data GD and the image generation history information GI are stored in separate files, but at the time of image processing using the image generation history information GI, the image data GD and the image generation history information GI are This is because they are inseparable and function in the same manner as when stored in the same file. That is, an aspect in which the image data GD and the image generation history information GI are associated at least at the time of image processing is included in the image file GF in the present embodiment. Furthermore, a moving image file stored in an optical disc medium such as a CD-ROM, a CD-R, a DVD-ROM, a DVD-RAM is also included.

  The present invention can be applied to a printer, a digital camera, a computer having an image processing function, and the like.

1 is a block diagram showing a configuration of an image output system as an embodiment of the present invention. 1 is a block diagram showing a schematic configuration of a digital still camera 12. FIG. It is explanatory drawing which shows notionally an example of the internal structure of the image file which can be used in a present Example. 5 is an explanatory diagram illustrating an example of a data structure of an attached information storage area 103. FIG. It is explanatory drawing explaining an example of the data structure of an Exif data area. 5 is an explanatory diagram illustrating a subject area 510 in an image 500. FIG. 2 is a block diagram illustrating a schematic configuration of a printer. FIG. 2 is a block diagram illustrating a configuration of the printer 20 with a control circuit 30 of the printer 20 as a center. FIG. 4 is a flowchart showing a flow of processing for generating an image file GF in the digital still camera 12. 4 is a flowchart illustrating a processing routine of image processing in the printer 20. It is a flowchart which shows the process routine of the image process based on image generation history information. 3 is a flowchart illustrating a processing routine of a first embodiment of automatic image quality adjustment processing. It is a figure which shows the to-be-photographed object area | region in the image of 1st Example, and the histogram of the hue of the pixel contained in the to-be-photographed object area | region. It is explanatory drawing which shows the content of the process suitable for the imaging | photography scene in the Example of an automatic image quality adjustment process. It is a flowchart which shows the process routine of 2nd Example of an automatic image quality adjustment process. It is a figure which shows the processing content of 2nd Example. It is explanatory drawing which shows an example of the image output system which can apply an image data processing apparatus. It is a flowchart which shows the image processing routine when a color space conversion process is abbreviate | omitted. It is a flowchart which shows another example of the process routine of the image process based on image generation history information. It is a flowchart which shows another example of the process routine of the image process based on image generation history information.

Claims (1)

An output device that outputs an image using image data generated by an image generation device and image generation history information associated with the image data and including at least subject region information indicating a subject region in the image,
An image quality adjustment unit that calculates a hue of each pixel in the subject area and automatically executes an image quality adjustment process suitable for the image based on a ratio of pixels having a hue of a specific color gamut;
An image output unit that outputs an image according to the image data in which the image quality is adjusted;
Equipped with a,
The image generation history information includes shooting mode information indicating a shooting mode used when the image data is generated by shooting in the image generation device,
The image quality adjustment processing includes a plurality of types of processing including two or more of contrast correction, brightness correction, color balance correction, saturation correction, sharpness correction, memory color correction, and noise removal processing. Including
When the shooting mode information included in the image generation history information indicates a shooting mode other than a standard scene, the image quality adjustment unit does not execute all of the plurality of types of processes included in the image quality adjustment process. The output device is configured to .

Images