JP2014230128A - Computer program and digital camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform adjustment so as to keep color balance in all the areas of an image and to allow identified color to be converted into achromatic color concerning predetermined image conversion.SOLUTION: A program allows a computer to perform the steps for: inputting object image data; inputting achromatic color correspondence information indicating correspondence relation between achromatic output image data being achromatic color among the output image data of image conversion processing and the input image data of image conversion processing corresponding to the achromatic output image data; performing image conversion processing with respect to reference data to be achromatic reference in the object image data and acquiring post-conversion reference data; acquiring achromatic input image data corresponding to the achromatic image data corresponding to the post-conversion reference data on the basis of the achromatic correspondence information concerning the achromatic image data; calculating a data ratio between the reference data and the achromatic input image data for each channel; and performing image conversion processing by multiplying the object image data by each data ratio.

Description

  The present invention relates to a computer program and a digital camera.

  At present, digital cameras that perform image recording using an imaging element are widely used in place of conventional film cameras that perform imaging using film. However, there are cases where the appearance of image reproduction by a conventional film is desired. In order to respond to this, for example, an image processing apparatus that can obtain a film-like image by performing image conversion on an image photographed by a digital camera has been proposed (see Patent Document 1). This image processing apparatus sequentially performs white balance conversion, color conversion matrix, and gradation conversion processing on an image photographed by a digital camera and subjected to Bayer interpolation processing to obtain a film-tone image. In this image processing apparatus, processing is performed by optimizing the parameters of each process of white balance conversion, color conversion matrix, and gradation conversion so as to reproduce film tone.

JP 2001-346218 A

  When it is desired to reproduce film tone with higher accuracy than the above-mentioned conventional technology, image conversion to film tone is performed without performing white balance conversion processing in the same manner as when white balance conversion processing is not performed with a film camera. It is possible. In this case, the achromatic portion in the image before conversion does not always become an achromatic color after conversion. For this reason, when it is desired to adjust a specific area in an image to an achromatic color, even if the specific area is adjusted to an achromatic color in an image before conversion, it does not always become an achromatic color after image conversion to film tone. However, even in image conversion in which an achromatic color portion does not always become an achromatic color after conversion, it is possible to adjust a specified color such as a color of an area designated by the user to an achromatic color. It is hoped that.

(1) A computer program according to the first aspect of the present invention is a computer program for causing a computer to execute a predetermined image conversion process, and a target image input step of inputting target image data to be subjected to the image conversion process Achromatic color correspondence information input for inputting the achromatic color correspondence information indicating the correspondence between the achromatic color output image data of the achromatic color in the output image data of the image conversion processing and the input image data of the corresponding image conversion processing A converted reference data acquisition step for performing image conversion processing on the reference data serving as the reference for the achromatic color in the target image data and acquiring the converted reference data; and the achromatic color corresponding to the converted reference data Achromatic color input that acquires achromatic input image data corresponding to the image data based on the achromatic color correspondence information An image acquisition step; a data ratio calculation step for calculating a data ratio for each channel between the reference data and the achromatic input image data; an image conversion step for multiplying the target image data by the data ratio and further executing an image conversion process; Are executed by a computer.
(2) A computer program according to a third aspect of the invention is a computer program for causing a computer to execute a predetermined image conversion process, and a target image input step for inputting target image data to be subjected to the image conversion process A reference input step for inputting reference data serving as a reference for an achromatic color in the target image data, a post-conversion reference data acquisition step for executing image conversion processing on the reference data to obtain post-conversion reference data, By executing sequential image conversion processing on the virtual input image data, achromatic color output image data of the output image data of the image conversion processing and input image data of the image conversion processing corresponding thereto Achromatic color correspondence information indicating the correspondence relationship with the achromatic color image data corresponding to the converted reference data, An achromatic color input image acquisition step for acquiring the achromatic color input image data corresponding thereto based on the achromatic color correspondence information, and a data ratio calculation step for calculating a data ratio for each channel between the reference data and the achromatic color input image data And an image conversion step of multiplying the target image data by the data ratio and further executing an image conversion process.
(3) According to a sixth aspect of the present invention, there is provided a computer program according to a sixth aspect of the present invention, in a predetermined image conversion process, an achromatic color output image data which is an achromatic color among output image data of the image conversion process, and an input of an image conversion process corresponding to the achromatic color An achromatic color correspondence information creating step for creating achromatic color correspondence information indicating a correspondence relationship with image data and a recording step for recording the achromatic color correspondence information on a recording medium are executed by a computer.
(4) A digital camera according to a ninth aspect of the present invention includes an imaging unit that captures a subject image, and achromatic output image data that is an achromatic color among output image data of the image conversion process in a predetermined image conversion process. Storage means for storing achromatic color correspondence information indicating a correspondence relationship with input image data of image conversion processing corresponding to this, and white balance adjustment data acquisition for acquiring white balance adjustment data of captured image data captured by the imaging means Means, reference data calculation means for calculating reference data for achromatic colors in the captured image data based on the white balance adjustment data, and obtaining converted reference data by executing image conversion processing on the reference data Corresponding to the converted reference data acquisition means and the achromatic image data corresponding to the converted reference data Achromatic input image acquisition means for acquiring chromatic color input image data based on achromatic color correspondence information, data ratio calculation means for calculating a data ratio for each channel between reference data and achromatic color input image data, and data ratio And image conversion means for multiplying the captured image data and executing image conversion processing.
(5) A digital camera according to a tenth aspect of the present invention includes an imaging unit that captures a subject image, and achromatic output image data that is an achromatic color among output image data of the image conversion process in a predetermined image conversion process. Storage means for storing achromatic color correspondence information indicating a correspondence relationship with input image data of image conversion processing corresponding to this, and white balance adjustment data acquisition for acquiring white balance adjustment data of captured image data captured by the imaging means Means, reference data calculation means for calculating reference data for achromatic colors in the captured image data based on the white balance adjustment data, and obtaining converted reference data by executing image conversion processing on the reference data The post-conversion reference data acquisition means and the achromatic image data corresponding to the post-conversion reference data correspond to this. Achromatic color input image acquisition means for acquiring achromatic color input image data based on achromatic color correspondence information, a data ratio calculation means for calculating a data ratio for each channel between reference data and achromatic color input image data, and data ratio Recording control means for recording the image data together with the captured image data on a recording medium.

  In the predetermined image conversion, it is possible to adjust so that the color balance of the entire area of the image is maintained and the specified color is converted to an achromatic color.

1 is a diagram illustrating a configuration example of an image processing system. And FIG. 11 is a block diagram illustrating a configuration example of a personal computer. It is a block diagram explaining the structural example of a digital camera. (A) is a figure explaining the spectral density characteristic of a cyan layer, (B) is a figure explaining the spectral density characteristic of a magenta layer, (C) is a figure explaining the spectral density characteristic of a yellow layer. . It is a flowchart explaining the flow of a LUT creation process. It is a flowchart explaining the flow of an image process. It is a figure explaining the structural example of PC of the modification 8. FIG.

  Hereinafter, an embodiment for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating the configuration of an image processing system according to an embodiment of the present invention. A personal computer (PC) 1 is connected to an external device such as a digital camera 2, a display 3, and an input device 4 via a connection interface.

  In the present embodiment, the PC 1 performs image processing for converting photographed image data acquired from the digital camera 2 into an image that looks close to an image reproduced by a positive film (that is, a positive film-like image), and displays it on the display 3. Output. In this image processing, when the user designates an arbitrary region in the image via the input device 4 such as a mouse or a keyboard, the PC 1 adjusts the designated region to an achromatic color. Is supposed to do. Details of such image processing will be described later.

-PC configuration-
FIG. 2 is a block diagram illustrating a configuration example of the PC 1. The PC 1 includes a PC control device 10, a connection interface 11, and a recording medium 12 such as a hard disk. The PC 1 is connected to an external device such as the digital camera 2, the display 3, and the input device 4 via the connection interface 11. The connection interface 11 includes, for example, a USB interface that performs wired connection and a wireless LAN module that performs wireless connection. The PC control device 10 includes a CPU, a memory, and other peripheral circuits, and controls the operation of the PC 1 by executing a program stored in the memory.

  The PC control device 10 captures an image captured by the digital camera 2 from the memory card 26 (FIG. 3) attached to the digital camera 2 via the connection interface 11. Note that the PC control device 10 may capture an image captured by the digital camera 2 via a card interface (not shown) when a memory card on which an image captured by the digital camera 2 is recorded is attached to the PC 1.

-Digital camera configuration-
FIG. 3 is a block diagram illustrating a configuration example of the digital camera 2. The digital camera 2 includes an imaging optical system 20, an imaging device 21, a camera control device 22, a connection interface 23, a card interface 24, and a recording medium 25. A memory card 26 that is a removable recording medium is attached to the card interface 24.

  The digital camera 2 is connected to an external device such as the PC 1 (FIG. 1) via the connection interface 23. The connection interface 23 includes, for example, a USB interface that performs wired connection and a wireless LAN module that performs wireless connection.

  The camera control device 22 includes a CPU, a memory, and other peripheral circuits, and controls the operation of the digital camera 2 by executing a program stored in the memory.

  The imaging optical system 20 includes a plurality of lens groups including a zoom lens and a focusing lens, and forms a subject image on the light receiving surface of the imaging element 21. In order to simplify FIG. 3, the imaging optical system 20 is illustrated as a single lens.

  The image sensor 21 is configured by, for example, a CMOS image sensor. The image sensor 21 photoelectrically converts the subject image to generate an image signal. A well-known mosaic color filter is provided on the light receiving surface of the image sensor 21. The mosaic color filter has a mosaic arrangement (for example, a Bayer arrangement) in which primary color filters that pass light of any one of red (R), blue (B), and green (G) correspond to pixel positions. Is a color separation filter constituted by The imaging element 21 outputs a color image signal for each of the three primary colors of light by capturing a subject image through such a color filter.

  An image signal generated by the image sensor 21 is converted into a digital signal by an A / D conversion unit (not shown), and is output to the camera control device 22 as RAW data. The camera control device 22 performs, for example, only Bayer interpolation processing on the RAW data and records it on the memory card 26 via the card interface 24.

-Image processing to convert to positive film tone image-
In the present embodiment, image processing for conversion into a positive film tone image is performed by, for example, inputting RAW data (RGB values) into a positive film tone image and converting it into a device independent color space L ^* a ^* b. ^{* This is performed} using a three-dimensional LUT (Look Up Table) (hereinafter referred to as a positive film conversion LUT) output as a value. The PC control device 10 creates a positive film conversion LUT in advance and records it on the recording medium 12 of the PC 2.

  Here, a process for creating a positive film conversion LUT will be described. The PC control device 10 first generates an input value for the positive film conversion LUT. The input value is, for example, a grid point obtained by dividing the data range of each of the RGB values output from the image sensor 21 of the digital camera 2 at equal intervals in a plurality of stages. For example, when each of RGB is divided into 18 stages, (R, G, B) = (R [0], G [0], B [0]), (R [0], G [0], B [1] ), ... (R [0], G [0], B [17]), (R [0], G [1], B [0]), (R [0], G [1], B [1]), ... (R [0], G [1], B [17]), ... (R [17], G [17], B [0]), (R [17 ], G [17], B [1]),... (R [17], G [17], B [17]), 18 × 18 × 18 input values are generated. For example, when the RGB values output from the image sensor 21 are each 8 bits (maximum level 255), (R [0], G) for (R, G, B) = (0, 0, 0). [0], B [0]), (R, G, B) = (255, 255, 255) corresponds to (R [17], G [17], B [17]) .

Then, the PC control device 10 obtains an output value corresponding to each of the generated 18 × 18 × 18 input values. For example, when calculating an output value for an input value (R [1], G [2], B [3]), for this input value (R [1], G [2], B [3]) Output values (L ^* [1], a ^* [2], b ^* [3]) are calculated by performing a predetermined calculation process for converting to a positive film tone image.

Here, the calculation process for converting to a positive film tone image will be specifically described. First, image reproduction of positive film will be considered. The positive film has a cyan dye layer, a magenta dye layer, a yellow dye layer, and a base layer. Therefore, the color of the positive film after development is large and can be considered to be determined by the characteristics obtained by adding the spectral density characteristics of the base to the spectral density characteristics of the cyan, magenta, and yellow dyes. Therefore, the spectral density characteristic of the cyan layer is represented as Dc (λ), the spectral density characteristic of the magenta layer is represented as Dm (λ), the spectral density characteristic of the yellow layer is represented as Dy (λ), and the spectral density characteristic of the base is represented as Db (λ). The overall spectral density characteristic D (λ) of the positive film can be expressed by the following formula (1).
D (λ) = Dc (λ) + Dm (λ) + Dy (λ) + Db (λ) (1)

  The base spectral density characteristic Db (λ) is always constant regardless of what kind of subject is photographed. On the other hand, the spectral density characteristics Dc (λ), Dm (λ), and Dy (λ) of the cyan layer, magenta layer, and yellow layer are portions that vary depending on the subject, and this portion mainly constitutes the color of the image. Yes. For example, if the color of the photographed subject has a strong blue element, the spectral density characteristic Dc (λ) of the cyan layer has a large value, and if the red element is strong, the spectral density characteristic Dm of the magenta layer and the yellow layer. (λ) and Dy (λ) are generally large values.

  Therefore, in general, in order to consider the developed color of a positive film, the spectral density characteristics Dc (λ), Dm (λ), and Dy (λ) of the cyan layer, magenta layer, and yellow layer are the strong colors of the subject. It is important to know how it changes. In order to know this, a positive film is exposed and developed so that the density of only one dye changes, a chart is created, and this is measured spectroscopically. Thereby, the spectral density characteristic of each pigment | dye can be known.

  As an example, FIG. 4 shows the result of spectroscopic measurement of a chart created with 10 levels of density for each color. 4A shows the spectroscopic measurement result of the cyan chart, FIG. 4B shows the spectroscopic measurement result of the magenta chart, and FIG. 4C shows the spectroscopic measurement result of the yellow chart. Since the cyan layer absorbs red, which is a complementary color of cyan, the density at the red wavelength is high (that is, the transmittance is low). The same principle applies to the magenta layer and the yellow layer. Note that the results shown in FIG. 4 are conceptual, not actual measurement values. In FIG. 4, the spectral density characteristics of each density in the cyan layer are Dc [1] (λ) to Dc [10] (λ), and the spectral density characteristics of each density in the magenta layer are Dm [1] (λ) ˜ Dm [10] (λ) is assumed, and spectral density characteristics of the respective densities in the yellow layer are Dy [1] (λ) to Dy [10] (λ). [1] to [10] indicate the numbers (indexes) of the curves of the spectral density characteristics shown in FIG. 4, and the larger the value, the higher the density.

  Here, consider the case of obtaining the spectral density characteristic Dc (C, λ) of the cyan layer with the density parameter C indicating the density of the cyan layer. The density parameter C indicates the density value of the cyan layer at the wavelength λc that is the peak density. For example, assuming that the density value at the wavelength λc in Dc [9] (λ) shown in FIG. 4 is 1.8 (that is, Dc [9] (λc) = 1.8), the spectral density characteristic at the density parameter C = 1.8. Dc (1.8, λ) is Dc [9] (λ) itself. For example, if the density value at the wavelength λc in Dc [10] (λ) shown in FIG. 4 is 2.0 (Dc [10] (λc) = 2.0), the spectral density characteristic Dc at the density parameter C = 2.0. (2.0, λ) is Dc [10] (λ) itself.

When the density parameter C is 1.8 <C <2.0 (that is, Dc [9] (λc) <C <Dc [10] (λc)), Dc [9] (λ) and Dc [10] Using (λ), the spectral density characteristic Dc (C, λ) in the density parameter C can be obtained approximately. For example, the spectral density characteristic Dc (1.9, λ) at the density parameter C = 1.9 can be obtained approximately by the following equation.
Dc (1.9, λ) = Dc [9] (λ) + [(1.9−1.8) / (2.0−1.8)] · [Dc [10] (λ) −Dc [9] (λ)]

As described above, the spectral density characteristic Dc (C, λ) of the cyan layer at an arbitrary density parameter C is calculated as Dc from the spectral density characteristics Dc [1] (λ) to Dc [10] (λ) measured in advance. Find Dc [i] (λ) and Dc [i + 1] (λ) such that [i] (λc) <C <Dc [i + 1] (λc), and this Dc [i] (λ) And Dc [i + 1] (λ) can be approximately obtained by the following equation (2). In the following equation (2), Dc [i] (λc) = C1 and Dc [i + 1] (λc) = C2.
Dc (C, λ) = Dc [i] (λ) + [(C-C1) / (C2-C1)] · [Dc [i + 1] (λ) −Dc [i] (λ)] ( 2)

  Similarly, the spectral density characteristic Dm (M, λ) of the magenta layer at the density parameter M indicating the density of the magenta layer is also approximated using Dm [1] (λ) to Dm [10] (λ) measured in advance. Can be obtained. The density parameter M indicates the density value of the magenta layer at the wavelength λm that is the peak density. Similarly, Dy [1] (λ) to Dy [10] (λ) measured in advance are also used for the spectral density characteristic Dy (Y, λ) of the yellow layer in the density parameter Y indicating the density of the yellow layer. Can be obtained approximately. The density parameter Y indicates the density value of the yellow layer at the wavelength λy that is the peak density.

  Thus, from the density parameters C, M, and Y indicating the density of each color (cyan, magenta, yellow), the spectral density characteristics Dc (λ), Dm (λ), and Dy (λ ) And the total spectral density characteristic D (λ) of the positive film can be obtained using the above-described equation (1).

Then, once the observation light source is determined, the tristimulus values X, Y, and Z indicating the appearance are expressed based on the spectral characteristic I (λ) of the observation light source and the overall spectral density characteristic D (λ) of the positive film. It is computable by Formula (3)-(5). In the following formulas (3) to (5), x (λ), y (λ), and z (λ) indicate color matching functions.
X = ∫ {I (λ) · 10 ^{(−D (λ))} · x (λ)} dλ (3)
Y = ∫ {I (λ) · 10 ^{(−D (λ))} · y (λ)} dλ (4)
Z = ∫ {I (λ) · 10 ^{(−D (λ))} · z (λ)} dλ (5)

Therefore, in order to convert an image photographed by the digital camera 2 into a positive film tone, the image data photographed by the digital camera 2 is converted into density parameters C, M, Y indicating the density of each color (cyan, magenta, yellow). It only has to be converted to. When the density parameters C, M, and Y of each color are obtained, the spectral density characteristics Dc (λ), Dm (λ), and Dy (λ) of each color are obtained, and the total spectral density characteristic D (λ) of the positive film is obtained. Therefore, tristimulus values X, Y, and Z are obtained. The tristimulus values X, Y, and Z can be converted into L ^* a ^* b ^* values using a known conversion formula. In this way, a positive film-like image can be reproduced from the image data photographed by the digital camera 2 in consideration of the physical characteristics of the positive film.

  Therefore, the PC control apparatus 10 creates in advance a conversion matrix for converting image data (RAW data) taken by the digital camera 2 into density parameters C, M, and Y for each color (cyan, magenta, yellow). . Note that the RAW data used in the present embodiment is image data that has undergone only Bayer interpolation processing, and is image data that has not undergone image processing such as white balance conversion, color matrix conversion, and gradation conversion. To do.

The RAW data is image data having linearity with respect to the amount of light (linear). On the other hand, the density is a value obtained by performing logarithmic transformation with 10 as the base with respect to the reciprocal of light reflectance (or transmittance). Based on this, in the present embodiment, the RGB values are converted into a logarithmic space similar to the density by performing logarithmic conversion with 10 as the base of the RGB values of the RAW data by the following equations (6) to (8). To do. In the following formulas (6) to (8), the values after this conversion are R ₁₀ , G ₁₀ , and B ₁₀ .
R ₁₀ = Log ₁₀ R (6)
G ₁₀ = Log ₁₀ G (7)
B ₁₀ = Log ₁₀ B (8)

Then, a density conversion matrix for converting each color (cyan, magenta, yellow) into density parameters C, M, Y using the converted values R ₁₀ , G ₁₀ , B ₁₀ is created. Specifically, the conversion from R ₁₀ , G ₁₀ , B ₁₀ to C, M, Y is performed by the following equation (9). In the following equation (9), coefficients m1 to m9 are transformation matrix parameters, and coefficients n1 to n3 are offset amounts. The offset amount is an amount corresponding to the value normalization.

  The optimization of the coefficients m1 to m9 and n1 to n3 is performed by performing both shooting using a positive film and shooting using the digital camera 2 on a color chart composed of a plurality of color patches. This can be performed based on both data of spectral colorimetry data of the film and image data taken by the digital camera 2.

First, consider shooting using positive film. As described above, the total spectral density characteristic D (λ) of the positive film can be expressed by Expression (1), and is determined by the density parameters C, M, and Y of each color. Here, when a color chart is photographed using a positive film and developed, a result obtained by measuring the color patch portion with a spectral densitometer is defined as Dp (λ). Then, C, M, and Y in the color patch can be obtained by obtaining C, M, and Y that Dp (λ) and D (λ) are closest to each other. For example, if the difference between Dp (λ) and D (λ) is F, it can be obtained as C, M, and Y values that minimize F calculated by the following equation (10).
F = Σ | D (λ) −Dp (λ) | (10)

For shooting using the digital camera 2, RAW data (RGB) of a certain color patch when the color chart is shot with the digital camera 2 is acquired, and logarithmic conversion is performed using equations (6) to (8). By doing, R ₁₀ , G ₁₀ , and B ₁₀ in the color patch can be obtained.

In this way, a set of C, M, Y and R ₁₀ , G ₁₀ , B ₁₀ can be obtained for one color patch. Then, by photographing a plurality of color patches, a set of a plurality of C, M, Y and R ₁₀ , G ₁₀ , B ₁₀ can be obtained. Therefore, the coefficients m1 to m9 and n1 to n3 in the equation (9) can be obtained based on the combination of the plurality of C, M, Y and R ₁₀ , G ₁₀ , B ₁₀ . Specifically, R ₁₀ , G ₁₀ , B ₁₀ obtained by taking a color patch with a digital camera 2 and logarithmically converted by Equation (9), and color patches are taken with a positive film and measured. The coefficients m1 to m9 and n1 to n3 that reduce the difference from C, M, and Y obtained by coloring are optimized using a least square method or the like.

  The density conversion matrix (Equation (9)) created in this way and the spectral density characteristics Dc (λ), Dm (λ), Dy (λ) and the base of each dye of the positive film in a plurality of density parameters. Using the spectral density characteristic Db (λ) (hereinafter referred to as spectral density characteristic information), it is possible to perform arithmetic processing for conversion into a positive film tone image.

Specifically, when the PC control device 10 obtains the output values (L ^* , a ^* , b ^* ) for the input values (R, G, B), the equations (6) to (8) are calculated for the input values. To perform logarithmic conversion. Next, the PC control device 10 converts the logarithmically converted values (R ₁₀ , G ₁₀ , B ₁₀ ) into density parameters C, M, and Y using equation (9). Next, the PC control device 10 uses the density parameters C, M, and Y and the spectral density characteristic information, and the spectral density characteristics Dc (λ), Dm (λ), and Dy (λ) of each color according to equation (2). Is calculated. Next, the PC control device 10 substitutes the calculated spectral density characteristics Dc (λ), Dm (λ), and Dy (λ) of each color into the formula (1), thereby obtaining the total spectral density characteristics D on the positive film. Calculate (λ). Next, the PC control apparatus 10 calculates the tristimulus values X, Y, and Z by using the calculated spectral density characteristic D (λ) according to the equations (3) to (5). Incidentally, observation light source to be set in calculating tristimulus values X, Y, and Z may be, for example, a daylight such as D _50. In addition, assuming that a positive film is projected onto a screen by a projector for viewing, the observation light source may be a light source for the projector, or may be a combination of the light source of the projector and the spectral reflectance of the screen. . Then, the PC control device 10 converts the tristimulus values X, Y, and Z into L ^* a ^* b ^* values using a known conversion formula. As a result, output values (L ^* , a ^* , b ^* ) for the input values (R, G, B) are obtained.

The PC control device 10 sequentially performs the above arithmetic processing on all input values (R, G, B), and outputs values (L ^* , G, B) for all input values (R, G, B). a ^* and b ^* ) are obtained. The PC control device 10 can create a positive film conversion LUT from a set (input / output data set) of each input value (R, G, B) and output value (L ^* , a ^* , b ^* ) corresponding thereto. it can.

In the arithmetic processing described above, when the ratio of the input values (RGB values) is 1: 1: 1 (achromatic color), a ^* = b ^* = in the output value (L ^* a ^* b ^* value). It is not always 0 (achromatic).

-Conventional White Balance Adjustment By the way, RAW data is image processing data that has not been subjected to image processing such as white balance conversion, matrix conversion, and gradation conversion in a digital camera. Therefore, for example, white balance conversion is performed on RAW data using application software on a PC. In such a case, for example, a method of performing white balance conversion by designating a geometric area in an image by a user is known. In this method, a geometric area that the user desires to reproduce as an achromatic color, for example, a white area of clothes or a white area of a signboard is specified, and image conversion is performed so that the specified area becomes an achromatic color. Specifically, the average value of the image data in the designated area is calculated for each channel (RGB), and the white balance gain (gainR, gainB) is calculated from the data ratio between the channels of the average value and applied to the image data. .

  An example of such white balance conversion processing will be shown. Here, it is assumed that the image conversion is performed in the order of Bayer interpolation, white balance conversion, matrix conversion, and γ conversion. For example, it is assumed that the average value of RGB channels after Bayer interpolation processing in an area designated by the user is (R, G, B) = (100, 200, 160). In this case, in the average value of the RGB channels, the data ratio gainR of the G channel to the R channel and the data ratio gainB of the G channel to the B channel are gainR = 200/100 = 2.00 and gainB = 200/160 = 1.25, respectively. Calculated. In the image data after Bayer interpolation, white balance conversion is performed by multiplying the R channel and B channel image data by gainR and gainB, respectively (assuming nothing is multiplied on the G channel). By this white balance conversion, the average values (R, G, B) of the above RGB channels (100, 200, 160) are converted to R: 100 x 2.00 = 200, G: 200, B: 160 x 1.25 = 200 Is done. In the subsequent matrix conversion and γ conversion, image data having a RGB channel data ratio of 1: 1: 1 is converted so as to maintain the ratio. By converting in this way, the image data of the area specified by the user can be finally rendered achromatic with the RGB channel data ratio of 1: 1: 1.

  Here, when performing image conversion without using the white balance gain as in the above-described image conversion processing to positive film tone, the region designated by the user is achromatic while generally retaining the image reproduction characteristics. Let's consider implementing a process that adjusts to For example, as described above, it is assumed that the white balance conversion is performed on the image data by calculating the data ratio between the RGB channels from the average value for each RGB channel in the region designated by the user. In this case, even if the subsequent image data is converted into a positive film tone, the area designated by the user is not always achromatic. In the conventional image conversion, in the matrix conversion and the γ conversion after the white balance conversion process, the image data having a data ratio between the RGB channels of 1: 1: 1 is converted so as to maintain the ratio. In particular, the image data of the area designated by the user can be achromatic. However, in the image conversion process to the positive film tone, the image data in which the data ratio between the RGB channels is 1: 1: 1 is not always converted so as to maintain the ratio. For this reason, even if the image conversion processing to positive film tone is performed after the white balance conversion processing, the region designated by the user is not necessarily achromatic.

  Therefore, in the present embodiment, the area designated by the user can be adjusted to an achromatic color when performing image conversion processing to positive film tone based on the above-described points. Next, the process for adjusting the achromatic color will be described in detail.

-LUT creation process-
First, the flow of LUT creation processing for creating an LUT used in the present embodiment will be described using the flowchart shown in FIG. It is assumed that a computer program (LUT creation program) for causing the PC control device 10 to execute this LUT creation processing is stored in, for example, a memory (not shown) in the PC control device 10.

  In step S1, the PC control device 10 creates a positive film conversion LUT as described above. The input value of the positive film conversion LUT created here is a grid point obtained by dividing the data range of each of the RGB values output from the image sensor 21 at equal intervals in 18 steps. Therefore, the positive film conversion LUT created here has a set of 18 × 18 × 18 input values and output values.

In step S2, the PC control apparatus 10 determines a plurality of virtual input values, for example, input values from (R, G, B) = (0, 0, 0) to (255, 255, 255) (that is, 256 × 256 × 256 input values), the arithmetic processing for converting into the above-described positive film tone image is sequentially performed to obtain the output value L ^* a ^* b ^* value. Here, it is assumed that the RGB values output from the image sensor 21 of the digital camera 2 are each 8 bits (maximum level 255). As a result, the PC control apparatus 10 obtains a set of 256 × 256 × 256 input values (RGB values) and output values (L ^* a ^* b ^* values). That is, in step S2, the PC control device 10 creates a positive film conversion LUT (referred to as a high-definition LUT) with higher definition than in step S1.

  Since the positive film conversion LUT created in step S1 is stored in the recording medium 12, the number of input values is reduced by dividing the RGB data range at equal intervals in order to save data capacity. On the other hand, since the high-definition LUT created in step S2 is used for creating an achromatic color LUT in step S3 described later (not stored in the recording medium 12), the input value is set to increase the accuracy of the achromatic color LUT. Is increased from step S1. In steps S1 and S2, the number of input values is not limited to the number described above.

In step S3, the PC control device 10 selects (L ^* , a ^* , b) out of a set of input values (RGB values) and output values (L ^* a ^* b ^* values) of the high-definition LUT created in step S2. ^* ) A pair that is closest to (iL, 0, 0) (ie, closest to an achromatic color) is extracted. IL is changed step by step from 0 to 100. In other words, the PC control apparatus 100 determines the input value (RGB value) and the output value (L ^* a ^* ) with the L ^* a ^* b ^* value closest to an achromatic color for each of 101 iLs with iL = 0 to 100 ^. b) ^* Value) and a pair are extracted. Then, the PC control apparatus 10 uses a set (namely, 101 sets) of each iL and the input values (RGB values) extracted from the iL to use an LUT (referred to as an achromatic color LUT) used for achromatic color adjustment. Create

  In step S4, the PC control device 10 records the positive film conversion LUT created in step S1 and the achromatic color LUT created in step S3 in the same file and records them on the recording medium 12.

-Image processing-
Next, the flow of image processing for performing image conversion to positive film tone and achromatic color adjustment will be described using the flowchart shown in FIG. It is assumed that a computer program (image processing program) for causing the PC control device 10 to execute this image processing is stored in, for example, a memory (not shown) in the PC control device 10.

  In step S11, the PC control apparatus 10 inputs the positive film conversion LUT and the achromatic color LUT recorded in the same file on the recording medium 12.

  In step S <b> 12, the PC control device 10 inputs RAW data captured by the digital camera 2.

In step S13, the PC control device 10 performs image conversion on the RAW data (input image data) input in step S12 using a positive film conversion LUT, and displays the image data after the image conversion on the display 3. To do. In this image conversion, RAW data (RGB values) are converted to L ^* a ^* b ^* values using a positive film conversion LUT, and the converted L ^* a ^* b ^* values are displayed on the display. 3 is converted into a color space (RGB value) that depends on 3. The PC control device 10 describes the relationship between the color space L ^* a ^* b ^* that does not depend on the display 3 and the color space RGB that depends on the display 3 for conversion from L ^* a ^* b ^* to the display RGB, for example. Based on the specified color profile. As a result, the input image (RAW data) is converted into a positive film tone image and displayed on the display 3.

  In step S <b> 14, the PC control device 10 determines whether or not an area to be converted to an achromatic color is designated by the user. The user confirms the image displayed on the display 3, and if there is a region that is desired to be reproduced with an achromatic color in the image, the user designates the region via the input device 4. The PC control device 10 repeats the determination process of step S14 until the user designates an area to be converted to an achromatic color. When the area is designated, the determination in step S14 is affirmative and the process proceeds to step S15.

  In step S15, the PC control apparatus 10 calculates the average value of each channel (RGB) in the RAW data of the pixels included in the region specified in step S14. The average value calculated here is (Ra, Ga, Ba). In this image processing, this average value (Ra, Ga, Ba) is used as RGB data (reference data that serves as a reference for the achromatic color) that should be achromatic, and this (Ra, Ga, Ba) is an image for a positive film tone. Processing is performed so that an achromatic color is obtained after conversion.

In step S16, the PC control apparatus 10 converts the average values (Ra, Ga, Ba) calculated in step S15 into L ^* a ^* b ^* values using a positive film conversion LUT. The L ^* a ^* b ^* value converted here is (L ^* ₀ , a ^* ₀ , b ^* ₀ ).

In step S17, the PC control device 10 refers to the achromatic color LUT and has the same brightness as the L ^* a ^* b ^* value obtained in step S16 (that is, iL = L ^* ₀ ). Search for a pair. Let the RGB value of the searched set be (Rs, Gs, Bs). That is, in step S17, the L ^* a ^* b ^* value obtained by converting the average value (RGB value) in the RAW data of the designated area by the positive film conversion LUT has the same brightness and is achromatic L ^* a ^{For the *} b ^* value, the corresponding RGB value is obtained based on the achromatic LUT.

In step S18, the PC control device 10 calculates the RGB value calculated based on the achromatic color LUT in step 17 with respect to the RGB average value (Ra, Ga, Ba) in the RAW data of the designated area calculated in step S15. The data ratio (GainR, GainG, GainB) for each channel of Rs, Gs, Bs) is calculated from the following equations (11) to (13).
GainR = Rs / Ra (11)
GainG = Gs / Ga (12)
GainB = Bs / Ba (13)

  In step S19, the PC control apparatus 10 multiplies the RAW data by the data ratio (GainR, GainG, GainB) obtained in step S18.

  In step S20, the PC control apparatus 10 performs image conversion again on the image data obtained in step S19 using the positive film conversion LUT, and displays the image data after the image conversion on the display 3. Note that this image conversion is the same image conversion as in step S13.

In the above processing, in step S19, the average value (Ra, Ga, Ba) of the area specified in step S14 is converted into (Rs, Gs, Bs) obtained in step S17 in the RAW data. This (Rs, Gs, Bs) is converted to an achromatic color by image conversion using the positive film conversion LUT in step S20. This is because in step S17, (Rs, Gs, Bs) is obtained using the achromatic LUT in which the RGB values corresponding to the achromatic L ^* a ^* b ^* values are described.

  Therefore, by the above processing, it is possible to reproduce the region designated by the user in an achromatic color while leaving the reproduction tendency of the image processing using the positive film conversion LUT.

According to the embodiment described above, the following operational effects can be obtained.
(1) In accordance with the LUT creation program, the PC control device 10 performs achromatic output image data (a ^* = b ^*) that is achromatic among output image data of the image conversion process in the image conversion process to positive film tone ^. Achromatic color LUT indicating a correspondence relationship between L ^* a ^* b ^* value ⁽ = 0) and input image data (RGB value) of image conversion processing corresponding to this is created. Further, the PC control device 10 creates a positive film conversion LUT indicating the correspondence between the input image data (RGB values) and the output image data (L ^* a ^* b ^* values) of the image conversion process. The PC control device 10 records the achromatic color LUT and the positive film conversion LUT in the same file. Further, the PC control device 10 inputs RAW data, an achromatic color LUT, and a positive film conversion LUT according to the image processing program. Then, the PC control device 10 executes the image conversion process on the reference data (average values (Ra, Ga, Ba)) serving as the reference for the achromatic color in the RAW data, and the converted reference data (L ^* ₀ , a ^* ₀ , b ^* ₀ ). The PC control device 10 then outputs achromatic image data corresponding to the converted reference data (L ^* ₀ , a ^* ₀ , b ^* ₀ ) (L ^* a having the same lightness and achromatic color as the converted reference data). ^{As for *} b ^* value, corresponding achromatic color input image data (Rs, Gs, Bs) is acquired based on the achromatic color LUT. Then, the PC control device 10 calculates a data ratio (GainR, GainG, GainB) for each channel between the reference data (Ra, Ga, Ba) and the achromatic color input image data (Rs, Gs, Bs). Then, the PC control apparatus 10 multiplies the data ratio (GainR, GainG, GainB) by the RAW data, and further executes the image conversion process. Thereby, in the image conversion to the positive film tone, the color balance of the entire area of the image can be maintained, and the color corresponding to the reference data can be adjusted to an achromatic color.

(2) When an arbitrary area in the image data is designated by the user operation according to the image processing program, the PC control apparatus 10 calculates an average value of each channel (RGB) of the RAW data in the designated area. The average value was used as the reference data. Thereby, in the image conversion to a positive film tone, the area designated by the user can be adjusted to an achromatic color.

(Modification 1)
In the above-described embodiment, an example in which the positive film conversion LUT and the achromatic color LUT are recorded in the same file in the LUT generation processing has been described. However, separate files may be used. The achromatic color LUT may be recorded in association with each other.

(Modification 2)
In the above-described embodiment, the positive film conversion LUT and the achromatic color LUT are created in the LUT generation process. However, for example, when the positive film conversion LUT has already been recorded on the recording medium 12, there is no need to do so. Only the color LUT may be created.

  Further, the PC control device 10 may input a positive film conversion LUT that has already been created from an external device. In this case, the PC control apparatus 10 uses the input positive film conversion LUT when creating the achromatic color LUT in step S3. Then, the PC control apparatus 10 adds the achromatic color LUT created in step S3 to the input positive film conversion LUT and records it on the recording medium 12.

(Modification 3)
In the above-described embodiment, the example in which the achromatic color LUT created in advance is used in the image processing shown in FIG. 6, but the achromatic color LUT may be created during the image processing. In this case, the PC control device 10 performs steps S2 and S3 of the LUT generation process shown in FIG. 5 in step S17 of FIG. 6 to create an achromatic color LUT.

(Modification 4)
In the above-described embodiment, the example of performing image conversion to positive film tone using the positive film conversion LUT created in advance in the image processing shown in FIG. 6 has been described. Image conversion to positive film tone may be performed by calculation processing using matrix conversion. Further, the image conversion to the positive film tone is not limited to the processing described above, and may be performed by other arithmetic processing.

(Modification 5)
In the embodiment described above, the user designates an area desired to be achromatic in the image, and the average value of the RAW data in the area designated by the user is converted to RGB data (Ra, Ga, The example determined as Ba) has been described. However, RGB data that should be achromatic after image conversion may be calculated using white balance adjustment data (white balance gain value) of RAW data.

  The white balance gain value is recorded together with the RAW data (for example, in the same file) according to the white balance setting set by the digital camera 2 at the time of shooting. For example, when the preset white balance is set, the user of the digital camera 2 performs temporary shooting of an achromatic object such as a gray card before shooting, and the white balance gain value is determined and recorded from the shooting data. ing. Assuming that the gray card is captured as a subject in actual shooting, the data ratio between the RGB channels is 1: 1: 1 by multiplying the captured image data of the gray card portion by the recorded white balance gain value. . If the subsequent image processing is conversion that maintains a data ratio of 1: 1: 1 between the RGB channels, the reproduced image will be achromatic.

  However, when the image processing after multiplying the recorded white balance gain value by the photographed image data is image conversion using the positive film conversion LUT, the photographed image data of the gray card portion is achromatic. It is not guaranteed.

Therefore, in the fifth modification, the PC control device 10 determines RGB data (Ra, Ga, Ba) to be achromatic from the white balance gain values (gainPr, gainPb) recorded together with the RAW data. For example, RGB data (Ra, Ga, Ba) to be achromatic can be calculated from the white balance gain value by the following equations (14) to (16). In the following formulas (14) to (16), Gref is a predetermined value determined in advance.
Ra = Gref x (1 / gainPr) (14)
Ga = Gref (15)
Ba = Gref x (1 / gainPb) (16)

  In this way, after determining the RGB data (Ra, Ga, Ba) to be achromatic color, the PC control device 10 performs the processing after step S16 in FIG. Thereby, the RGB data (Ra, Ga, Ba) to be achromatic can be processed to be achromatic after image conversion to positive film tone.

  Further, in the image processing program for executing the image processing shown in FIG. 6, white balance setting can be performed by a user operation, and RGB data (Ra, Ga, Ba) to be achromatic color according to the setting. May be determined. In this case, for example, options such as clear sky, shade, and light bulb are provided as white balance settings, and white balance gain values corresponding to the options are stored in advance in the computer program. Then, the PC control device 10 uses the above formulas (14) to (16) to calculate RGB data (Ra, Ga, Ba) is calculated.

  By processing in this way, a normal image conversion mode (image data in which the data ratio between the RGB channels becomes 1: 1: 1 due to the white balance gain becomes 1: 1 in the data ratio between the RGB channels even in subsequent processing. 1) and the white balance compatibility can be maintained.

(Modification 6)
In the embodiment described above, the example in which the PC control device 10 executes the image processing shown in FIG. 6 according to the image processing program stored in the memory has been described. However, the digital camera 2 may execute the image processing shown in FIG. In this case, for example, the computer program for executing the image processing shown in FIG. 6, the positive film conversion LUT, and the achromatic color LUT are stored in the recording medium 25 of the digital camera 2 or the like. Then, the camera control device 22 of the digital camera 2 executes the image processing shown in FIG. 6 on the RAW data imaged by the imaging device 21 of the digital camera 2 according to the computer program. Then, the camera control device 22 records the image data after the image processing on the memory card 26 in the JPEG format, for example.

  Further, as described in the fifth modification, the camera control device 22 of the digital camera 2 acquires the white balance gain value of the RAW data, and based on the white balance gain value, the RGB data (Ra , Ga, Ba) may be determined. Further, the camera control device 22 of the digital camera 2 determines RGB data (Ra, Ga, Ba) to be achromatic based on the white balance gain value as described above, and then performs steps S16 to S18 in FIG. Processing may be performed to calculate a data ratio (GainR, GainG, GainB), and this data ratio may be recorded in the memory card 26 together with the RAW data imaged by the imaging device 21. In this case, if the computer program executed on the PC 1 acquires the RAW data and the data ratio from the memory card 26 and executes steps S19 and S20 in FIG. 6, the same effects as in the above embodiment can be obtained. .

(Modification 7)
In the above-described embodiment, the example in which the present invention is applied when adjusting the achromatic color in the case of performing the image conversion to the positive film tone using the positive film conversion LUT has been described. However, the present invention is not limited to image conversion to positive film tone, and the present invention may be applied to various other image conversions. In particular, by applying the present invention to image conversion in which an achromatic portion before conversion does not always become an achromatic color after conversion, an area designated by the user can be adjusted to an achromatic color.

(Modification 8)
In the above-described embodiment, the case where the program for the PC control device 10 to execute each process is stored in the memory of the PC control device 10 has been described. However, the loading of the program to the PC 1 may be performed by setting a recording medium 40 such as a CD-ROM storing the program in the PC 1 as shown in FIG. 7, or via a communication line 41 such as a network. It may be loaded into PC1. When going through the communication line 41, the program is stored in the hard disk device 43 of the server (computer) 42 connected to the communication line 41. The program can be supplied as various types of computer program products such as provision via the recording medium 40 or the communication line 41.

  The above description is merely an example, and is not limited to the configuration of the above embodiment. Further, the configuration of each modification example may be appropriately combined with the above-described embodiment, or the structure of each modification example may be appropriately combined. For example, modification 3 and modification 6 may be combined. In this case, when the camera control device 22 of the digital camera 2 executes the image processing shown in FIG. 6, steps S2 and S3 of the LUT generation processing shown in FIG. Create a coloring LUT.

DESCRIPTION OF SYMBOLS 1 ... PC, 2 ... Digital camera, 3 ... Display, 10 ... PC control apparatus, 12, 25 ... Recording medium, 21 ... Image sensor, 22 ... Camera control apparatus, 26 ... Memory card

Claims (10)

A computer program for causing a computer to execute predetermined image conversion processing,
A target image input step of inputting target image data to be subjected to the image conversion processing;
Achromatic color correspondence information input for inputting the achromatic color correspondence information indicating the correspondence between the achromatic color output image data of the achromatic color in the output image data of the image conversion processing and the corresponding input image data of the image conversion processing. Steps,
A post-conversion reference data acquisition step that executes the image conversion process on reference data that is a reference for an achromatic color in the target image data, and acquires post-conversion reference data;
For the achromatic color image data corresponding to the converted reference data, the achromatic color input image acquisition step for acquiring the achromatic color input image data corresponding thereto based on the achromatic color correspondence information;
A data ratio calculating step for calculating a data ratio for each channel between the reference data and the achromatic input image data;
An image conversion step of multiplying the target image data by the data ratio and further executing the image conversion process;
A computer program for causing a computer to execute. The computer program according to claim 1,
Causing the computer to further execute a lookup table input step of inputting an image conversion lookup table indicating a correspondence relationship between the input image data and the output image data of the image conversion processing;
The computer program according to claim 1, wherein the image conversion process is performed based on the image conversion lookup table. A computer program for causing a computer to execute predetermined image conversion processing,
A target image input step of inputting target image data to be subjected to the image conversion processing;
A reference input step of inputting reference data serving as a reference for an achromatic color in the target image data;
A post-conversion reference data acquisition step of performing the image conversion process on the reference data to acquire post-conversion reference data;
By executing the image conversion process sequentially on a plurality of virtual input image data, the achromatic color output image data that is an achromatic color among the output image data of the image conversion process and the image conversion process corresponding thereto Achromatic color correspondence information indicating a correspondence relationship with the input image data, and for the achromatic color image data corresponding to the converted reference data, the achromatic color input image data corresponding thereto is determined based on the achromatic color correspondence information. An achromatic color input image acquisition step to be acquired;
A data ratio calculating step for calculating a data ratio for each channel between the reference data and the achromatic input image data;
An image conversion step of multiplying the target image data by the data ratio and further executing the image conversion process;
A computer program for causing a computer to execute. In the computer program as described in any one of Claims 1-3,
When an arbitrary area in the target image data is specified by a user operation, the computer further executes an average value calculating step of calculating an average value of each channel of the target image data in the specified area,
A computer program characterized in that the average value is used as the reference data. In the computer program as described in any one of Claims 1-3,
A white balance adjustment data input step for inputting white balance adjustment data of the target image data;
A reference data determination step for determining the reference data based on the white balance adjustment data;
Is further executed by a computer. In predetermined image conversion processing, achromatic color correspondence information indicating a correspondence relationship between the achromatic color output image data of the achromatic color in the output image data of the image conversion processing and the corresponding input image data of the image conversion processing is displayed. Achromatic color correspondence information creation step to create,
A recording step of recording the achromatic color correspondence information on a recording medium;
A computer program for causing a computer to execute. The computer program according to claim 6,
Causing the computer to further execute a lookup table creation step of creating an image conversion lookup table indicating a correspondence relationship between the input image data and the output image data of the image conversion process;
In the recording step, the achromatic color correspondence information and the image conversion lookup table are recorded in the same file. The computer program according to claim 6,
Causing the computer to further execute a lookup table input step of inputting an image conversion lookup table indicating a correspondence relationship between the input image data and the output image data of the image conversion processing;
In the recording step, the achromatic color correspondence information is added to the image conversion lookup table and recorded. Imaging means for capturing a subject image;
In predetermined image conversion processing, achromatic color correspondence information indicating a correspondence relationship between the achromatic color output image data of the achromatic color in the output image data of the image conversion processing and the corresponding input image data of the image conversion processing is displayed. Storage means for storing;
White balance adjustment data acquisition means for acquiring white balance adjustment data of imaged image data imaged by the imaging means;
Based on the white balance adjustment data, reference data calculation means for calculating reference data serving as a reference for an achromatic color in the captured image data;
A post-conversion reference data acquisition unit that executes the image conversion process on the reference data to acquire post-conversion reference data;
Achromatic color input image acquisition means for acquiring achromatic color input image data corresponding to the achromatic color image data corresponding to the converted reference data based on the achromatic color correspondence information;
Data ratio calculating means for calculating a data ratio for each channel between the reference data and the achromatic input image data;
Image conversion means for multiplying the captured image data by the data ratio and further executing the image conversion processing;
A digital camera comprising: Imaging means for capturing a subject image;
In predetermined image conversion processing, achromatic color correspondence information indicating a correspondence relationship between the achromatic color output image data of the achromatic color in the output image data of the image conversion processing and the corresponding input image data of the image conversion processing is displayed. Storage means for storing;
White balance adjustment data acquisition means for acquiring white balance adjustment data of imaged image data imaged by the imaging means;
Based on the white balance adjustment data, reference data calculation means for calculating reference data serving as a reference for an achromatic color in the captured image data;
A post-conversion reference data acquisition unit that executes the image conversion process on the reference data to acquire post-conversion reference data;
Achromatic color input image acquisition means for acquiring achromatic color input image data corresponding to the achromatic color image data corresponding to the converted reference data based on the achromatic color correspondence information;
Data ratio calculating means for calculating a data ratio for each channel between the reference data and the achromatic input image data;
Recording control means for recording the data ratio together with the captured image data on a recording medium;
A digital camera comprising:

Images