JP2004194303A - Imaging apparatus, imaging method, computer readable recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of easily setting the optimal chromaticity of a photographed image. <P>SOLUTION: In the imaging apparatus, an indication relating to a black body radiation axis like a chromaticity chart is performed, such that a user may set the optimal chromaticity of a picked-up image corresponding to his/her linking, intention or location with visual confirmation. The user designates an arbitrary white balance followup range and next determines a followup range parameter for achromatic color discrimination in accordance with the designated followup range. The determined followup range parameter is used to perform white balance processing to an imaging signal outputted from an imaging device, and an achromatic color discrimination range for white balancing may be then set or confirmed to generate and output a chrominance signal and a luminance signal based upon the imaging signal after white balancing. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

  The present invention relates to an image pickup apparatus including an image pickup device having a plurality of types of color filters and having a setting function relating to a color of an image pickup signal.

  2. Description of the Related Art A method of setting white balance (hereinafter, referred to as WB) correction in a digital camera will be described as an example in which a user performs a setting related to a hue in an imaging apparatus.

  In a conventional digital camera, the settings relating to WB correction performed by a user are roughly classified into an auto white balance (hereinafter, referred to as AWB) mode and a manual mode. When the former AWB mode is set, the light source or color temperature is automatically determined, and WB correction is performed accordingly. When the latter manual mode is set, a predetermined WB correction is performed by specifying a method of specifying a light source such as a light bulb or a fluorescent lamp or a color temperature. Furthermore, when the manual mode is used, it is possible to set a scene where it is difficult to automatically determine the light source or to set a different color temperature.

  Next, a method for performing the above-described AWB will be described.

  As a method of automatically performing WB correction according to a change in color temperature, a region that should be viewed as an achromatic color is determined from image data (hereinafter, achromatic color determination process), and WB correction is performed so that the determined region becomes achromatic. There is a method (first method) or a method of averaging the image signal of the entire image data for each color component and performing WB correction using the average value (second method). In the achromatic color determination, an area that should be viewed as an achromatic color is defined as an achromatic color determination range on a predetermined chromaticity plane, and an achromatic color is determined by determining whether or not the area is within the achromatic color determination range. A region to be powered is determined.

  However, in the above-described first method, an area (hereinafter simply referred to as an area) occupied in image data of an object (subject) to be viewed as an achromatic color in a video signal obtained from an image sensor or the like is sufficiently large. This is effective in such a case, but it is difficult to perform WB correction with high accuracy when there is no region of an object (subject) that should appear achromatic, or when there is a very small region.

  Also, even if the area of an object that should appear achromatic can be largely determined, for example, achromatic color at low color temperature and skin color at high color temperature, achromatic color at high color temperature and blue at low color temperature In some cases, the result of the achromatic color discrimination processing is close to each other, and the achromatic color discrimination that has been erroneously discriminated may be performed. In particular, when the achromatic color determination range is set to be wider, AWB can be applied to a wide range of light sources, but on the other hand, erroneous determination is made when the result of the achromatic color determination process described above is close. More cases.

  For example, an achromatic fluorescent lamp has a stronger green component than a solar light source. Therefore, by setting the chromaticity diagram so that it can follow the green direction more broadly (= the achromatic color determination range is wider), most fluorescent lamps follow the AWB. The fluorescent lamp is erroneously determined to be a fluorescent lamp, and accurate WB correction cannot be performed. Conversely, if the following range is narrowed, it becomes impossible to follow various light sources.

  In the above-described second method, the color components in the image data are integrated to make the value an achromatic color. For example, when the color component is biased toward a specific color such as a sunset scene, the average value obtained by the integration is calculated. It has been pointed out that although the color shown is not an achromatic color, the integral value is determined to be an achromatic color, which results in a significant erroneous determination.

  On the other hand, a third technique using both the first technique and the second technique described above, and a fourth technique for separately estimating the color temperature of the light source using information on the brightness of the subject have been proposed. ing.

  For example, detecting means for detecting the brightness of the subject, limiting means for limiting the white balance correction range for at least a low color temperature subject, and the limited white balance correction range according to the output of the detecting means, There has been proposed an imaging apparatus having a correction range changing unit for changing the range so as to be widened when it is dark (for example, see Patent Document 1). In other words, if the subject is relatively bright, it is predicted that the probability of being outdoors is high, and white balance correction is performed as an outdoor sun light source. When it becomes dark, it becomes cloudy or shaded. Is incorporated as an AWB tracking range (achromatic color determination range). By adopting such a method, it is possible to automatically correct the white balance to a memory color or a hue close to human sensation based on experience according to a shooting situation.

However, although the problems in the first and second methods are alleviated to some extent by the third and fourth methods, the light source cannot be determined, so that it is not a fundamental solution.
JP-A-7-23400

  In other words, it is possible to cope with light sources and environments within a range assumed in advance, but it becomes impossible to perform appropriate WB correction as the light source and environment deviate from the assumed range. For example, there is a problem that the AWB does not function well under a very bright bulb light source or when photographing a thin green in a dark place.

  In addition, there are cases in which the WB correction is desired to completely follow the change in the light source, and conversely, there is a case in which the WB correction is desired to be performed without following the change so as to leave the atmosphere of the place. It depends on the taste and intention of the person. For example, under a light source of a light bulb, a description that leaves a reddish image and a description that is completely WB-corrected are both often used depending on the shooting scene. As described above, it is difficult to determine the optimal following range of the assumed light source, and it depends on the intention of the photographer. Therefore, it is very difficult for the user to uniformly obtain the optimal WB correction setting. There is a problem. Further, conventionally, the user cannot visually confirm the setting of the WB correction as a color.

  In addition, when setting the hue (hue) of an image signal, for example, in the case of an RGB space, when a user sets each independent parameter of RGB, an RGB setting value may be visually grasped. Since it is not possible, it is difficult to find an optimal color balance. Also, when checking the color of a captured image as in the case of a histogram display, it is not possible to visually grasp the overall color balance simply by looking at the RGB independent levels.

An image pickup device according to claim 1, wherein the image pickup device converts light from a subject into an image pickup signal,
A signal processing circuit for processing the imaging signal, wherein:
Display means for displaying a black body radiation axis;
User interface means for a user to perform settings related to image processing of the imaging signal in the signal processing circuit,
The user interface unit performs the setting in a state where a display relating to a blackbody radiation axis is displayed on the display unit.
Further, the image pickup apparatus according to claim 18 is an image pickup device that converts light from a subject into an image pickup signal,
An imaging device having a signal processing circuit that processes the imaging signal,
A tracking intensity setting means for arbitrarily setting the tracking intensity of the white balance by a user;
White balance processing means for performing white balance processing on the image signal based on the degree of tracking set by the degree of tracking setting means.

The imaging device according to claim 28,
An image sensor that converts light from a subject into an image signal;
An imaging device having a signal processing circuit that processes the imaging signal,
Display means for displaying a setting screen representing the tracking strength of white balance;
Achromatic color discrimination parameter determining means for determining an achromatic color discrimination parameter corresponding to a tracking intensity arbitrarily set by a user on the setting screen,
White balance processing means for performing white balance processing on the image pickup signal using the achromatic color determination parameter determined by the achromatic color determination parameter determination means.

The imaging method according to claim 35,
The light from the subject is converted into an image signal by an image sensor,
Processing an image signal from the image sensor;
Display the diagram related to the blackbody radiation axis,
The setting is performed in a state where a display related to a blackbody radiation axis is displayed on the display unit.

The imaging method according to claim 36,
The light from the subject is converted into an image signal by an image sensor,
Processing an image signal from the image sensor;
Display the setting screen indicating the degree of white balance tracking,
Determine the achromatic color discrimination parameter corresponding to the follow-up degree arbitrarily set by the user on the setting screen,
White balance processing is performed on the imaging signal using the determined achromatic color determination parameter.

  Since the display relating to the blackbody radiation axis is performed in the image pickup apparatus, the user can set an optimum color tone in accordance with a preference, an intention, and a location, and also can set a white balance.

<First embodiment>
(Selection of preset follow range)
FIG. 1 is a block diagram illustrating a schematic configuration of the imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, reference numeral 100 denotes an imaging device, specifically, a digital still camera or the like. Since the imaging device 100 uses an imaging device having a plurality of color filters, the imaging device 100 has an auto white balance (hereinafter, referred to as AWB) function for automatically correcting a change in color tone due to a difference in light source. Hereinafter, the internal configuration of the imaging device 100 will be described.

  Reference numeral 101 denotes an image sensor, which is, for example, a CCD (Charge Coupled Device) sensor or a CMOS sensor, and has a color filter on each pixel. FIG. 2 shows an example of an arrangement of color filters included in the image sensor 101. As shown in FIG. 2, the color filter of the image sensor 101 is such that the next row of R, G1, R, G1,... (Hereinafter referred to as R row) is G2, B, G2, B,. The next row is a primary color Bayer filter array in which the row is repeated again and the row B is repeated next. With the above configuration, the image sensor 101 outputs an image signal (digital signal) corresponding to the amount of light passing through the color filter. The image sensor 101 includes an A / D conversion circuit if necessary, and outputs an image signal as a digital signal.

  A signal processing circuit 102 processes an image signal output from the image sensor 101 to generate image data. In addition, the signal processing circuit 102 has a function of performing AWB processing. The detailed configuration of the signal processing circuit 102 will be described later. A buffer memory 103 stores the image data output from the signal processing circuit 102. Reference numeral 104 denotes a compression circuit that reads out image data from the buffer memory 103 and outputs a compressed image obtained by compressing the image data using a predetermined compression method. The predetermined compression method is, for example, a JPEG (Joint Photographic Experts Group) method, and at this time, the compression circuit 104 outputs a JPEG image (compressed image) obtained by compressing the image data.

  When image recording is performed, the recording device 105 performs a process of recording the compressed image output from the compression circuit 104 on the recording medium 106. When image recording is not performed, such as during EVF (Electric View Finder), the image data stored in the buffer memory 103 is input to the display control circuit 108. The display control circuit 108 converts the image data into an image for a monitor display device. Reference numeral 109 denotes a D / A converter, which outputs an image signal (analog) obtained by D / A (digital / analog) conversion of a digital image signal for a monitor display device output from the display control circuit 108. Reference numeral 110 denotes a monitor device as a display unit, which displays an image based on an image signal output from the D / A converter 109.

  A system controller 111 controls the operation of the compression circuit 104, the recording device 105, the display control circuit 108, the following range determination circuit 512, and the flow of data in the imaging device 100. Reference numeral 112 denotes a user interface as a user interface unit, which transmits to the system controller 111 a change command regarding various settings input by a user who operates the imaging apparatus 100, a display process in the display control circuit 108, and the like. Reference numeral 113 denotes a data memory which stores a program which can be read and executed by the system controller 111 and various data to be processed by the system controller 111.

  Reference numeral 512 denotes a tracking range determination circuit that determines a tracking range parameter that is a parameter that determines an achromatic color determination range in a chromaticity diagram described later. The tracing range parameter determined by the tracing range determination circuit 512 is also a parameter indicating the tracing range in AWB correction. For example, a tracking range parameter that broadly defines the achromatic color determination range in the chromaticity diagram indicates that the tracking range is wide and AWB processing corresponding to various light sources is performed. Conversely, a tracking range parameter that narrows the achromatic color determination range indicates that the AWB processing is performed so that the tracking range is narrow and the type of light source is small.

  For the determination of achromatic color, for example, from a coordinate plane in which the vertical axis is an axis of an evaluation value indicating a change in hue from magenta to a green through an achromatic color, and the horizontal axis is an axis of an evaluation value indicating a change in color temperature Desired.

  Here, for example, a chromaticity diagram or the like is used to display the black body radiation axis. The chromaticity diagram is a diagram in which colors represented by a predetermined color space and a color reproduction area are two-dimensionally mapped and the colors are visually represented as coordinates. The axes of the chromaticity diagram are, for example, one in the color temperature direction and the other orthogonal axis is in the green and magenta directions. Alternatively, the expression of the chromaticity diagram may be a multidimensional display of the color space of the color space, and the parameter of the axis may be the RB direction instead of the color temperature direction.

  The color temperature and color of various light sources are expressed by plotting the coordinates of the chromaticity diagram based on the blackbody radiation axis (curve) characteristics. Are suitable.

  Reference numeral 513 denotes a tracking range parameter candidate, which stores a tracking range parameter candidate determined by the tracking range determination circuit 512. The tracking range determination circuit 512 and the tracking range parameter candidate 513 will be described later in detail.

  With the configuration described above, the imaging apparatus 100 can record a white-balanced compressed image on the recording medium 106 and display a white-balanced image on the monitor device 110. In addition, the imaging device 100 can perform AWB correction according to the set tracking range by setting the predetermined tracking range.

  Next, setting processing of AWB correction in the above-described imaging apparatus 100 will be described.

  First, the setting of the WB correction follow-up range by the user interface 112 will be described. FIG. 3 is a diagram showing an example of an AWB tracking range selection screen displayed on the imaging device 100 shown in FIG. In the image displayed on the monitor device 110 shown in FIG. 3, reference numeral 31 denotes a tracking range setting column for AWB correction, which includes “high color temperature side tracking range”, “low color temperature side tracking range”, and “green side tracking range”. This is a column in which a desired tracking range can be selected from four types of tracking ranges of “range” and “magenta side tracking range”. Specifically, a desired follow-up range is designated by selection by a cursor or a selection button provided in the imaging device 100 (not shown) or by selection by a user's touch on the screen if the monitor device 110 is a touch panel. Further, the above-described user interface is realized by the processing of the user interface 112.

FIG. 3 also shows the change of the color tone of the image displayed on the monitor device 110 according to the selected tracking range, simultaneously with the display of the tracking range setting column 31. In FIG. 3, the background of the subject is white, and the light source is a light bulb. Here, in FIG. 3A, the low color temperature tracking range is set to “4000k”, and the light bulb is set so as not to be included in the tracking range. Instead, an image that is a reddish color background is displayed. FIG. 3B shows an example in which the low color temperature tracking range is set to “2500K”, the WB correction is completely performed, and a white background image is displayed. In addition, the following range in the green / magenta direction includes a green correction filter and a magenta correction filter for correcting green cast of a fluorescent lamp when shooting with a positive film, and a number corresponding to a correction strength is provided. Existing. In the present embodiment, the deviation from the black body radiation corresponding to the correction intensity is indicated by a number, and the numerical value is designated to set the following range in the green / magenta direction.
The following range is set on the chromaticity diagram by displaying a chromaticity diagram for representing the black body radiation axis on the user interface screen. For example, FIG. 4 shows a tracking range designated by numerical values in FIGS. 3A, 3B, and 3C on a user interface screen. In the chromaticity diagrams of FIGS. 4A, 4B, and 4C, the vertical axis indicates a change in chromaticity from green through white to magenta, and the horizontal axis indicates a change in color temperature. Further, the chromaticity diagram of FIG. 4 is described together with names of various light sources.

  When the chromaticity diagram is displayed alone as a user interface screen as shown in FIG. 4A, the captured image and the user interface screen (a) are switched by an operation switch (not shown) to change the captured image. Confirm. Also, a transparent display (superimposed) may be displayed on the captured image as shown in FIG. Alternatively, the captured image may be displayed together with the chromaticity diagram as shown in (c), and the change in the set following range may be reflected on the captured image and displayed. As a result, the user can check the change in the chromaticity of the captured image in real time with the setting of the tracking range.

  Next, an operation of determining the following range parameter in the imaging apparatus 100 when the user sets an arbitrary following range on the above-described AWB following intensity selection screen shown in FIG. FIG. 5A is a flowchart showing a process of determining the following range parameter in the imaging apparatus 100 shown in FIG. Note that the operation of the imaging apparatus 100 illustrated in FIG. 5A is started when the user operates the user interface 112 or the like.

  First, when the user operates the user interface 112, a screen for selecting an AWB tracking range as shown in FIG. 3 is displayed on the monitor device 110 (step S101). At this time, if the camera is in the image capturing state, the EVF display of the captured image is switched to the selection screen. This prompts the user to set the follow-up range. Next, when the follow-up range is not set on the selection screen of FIG. 3 (NO in step S102), the display of the selection screen is maintained until it is set (loop processing). When the tracing range is set on the setting screen in FIG. 3 (YES in step S102), the setting information regarding the set tracing range is transmitted from the user interface 112 to the system controller 111. Next, the system controller 111 outputs an AWB tracking range setting signal corresponding to the transmitted tracking range to the tracking range determination circuit 512 (step S103).

  Next, the tracking range determination circuit 512 determines a tracking range parameter corresponding to the input AWB tracking range setting signal by referring to the tracking range parameter from the tracking range parameter candidate 513, and performs signal processing on the tracking range parameter. The data is transmitted to the circuit 102 (step S104). As described above, the imaging apparatus 100 can set the following range parameter corresponding to the following range set by the user on the AWB following range setting screen. Thus, the imaging apparatus 100 can perform the AWB correction using the following range parameter corresponding to the following range set by the user.

  Note that the internal configuration of the above-described imaging device 100 is one embodiment, and any other configuration may be used as long as there is an imaging device 100 having a plurality of types of color filters and a configuration for performing the above-described AWB processing. . In the above-described embodiment, four types of tracking ranges are set. However, the number of types is not limited thereto, and an arbitrary number of types may be set as the tracking range. Further, the tracking range is defined within the achromatic color determination range, and the achromatic color determination area on the chromaticity diagram corresponding to the tracking range is not limited to the above, but is an achromatic color determination area of various sizes and shapes. You may.

  Next, details of the WB correction processing in the above-described imaging device 100 will be described below.

  First, the internal configuration of the signal processing circuit 102 shown in FIG. 1 will be described. FIG. 6 is a diagram showing an example of the internal configuration of the signal processing circuit 102 shown in FIG. In FIG. 6, reference numeral 502 denotes a WB (white balance) circuit that receives an image signal from the image sensor 101 and performs WB correction so that a subject that looks white to a human becomes white even in an image output by the image capturing apparatus 100. Do. At this time, the WB circuit 502 performs WB correction according to the following range parameter input from the following range determining circuit 512. Next, the color signal generation circuit 503 generates and outputs color difference signals U and V based on the image data subjected to the WB correction. The luminance signal generation circuit 504 generates and outputs a luminance signal Y based on the image data subjected to the WB correction. As described above, the signal processing circuit 102 includes the WB circuit 502, the chrominance signal generation circuit 503, and the luminance signal generation circuit 504. Thus, the signal processing circuit 102 can output WB-corrected YUV format image data according to the following range parameter.

  Next, a detailed configuration of the WB circuit 502 shown in FIG. 6 will be described. FIG. 7 is a diagram showing a detailed configuration of WB circuit 502 shown in FIG. In FIG. 7, reference numeral 510 denotes a WB correction coefficient determination circuit, which calculates and outputs a WB correction coefficient based on an imaging signal output from the imaging element 101 and a tracking range parameter input from the tracking range determination circuit 512. Reference numeral 511 denotes a WB control circuit that multiplies an image signal output from the image sensor 101 by a WB correction coefficient output from the WB correction coefficient determination circuit 510 by a WB correction coefficient corresponding to each color signal. The image data after the WB correction is output. As described above, the WB circuit 502 includes the WB correction coefficient determination circuit 510 and the WB control circuit 511. Thus, the WB circuit 502 can output the image data after the WB correction according to the following range parameter based on the imaging signal from the imaging device 101. The image data after the WB correction is output to the monitor device 110, so that the user can check the image.

  Next, the following range parameter will be described. In the present embodiment, the achromatic color determination range in the chromaticity diagram is changed according to the value of the following range parameter in accordance with the above described following range set by the user. Here, specific examples of the chromaticity diagram and the achromatic color determination range will be described.

  On the vertical axis of the chromaticity diagram shown in FIG. 4, a change in chromaticity from green to white to magenta can be represented by a color evaluation value Ey. Further, on the horizontal axis of the chromaticity diagram, a change in color temperature can be represented by a color evaluation value Ex. When the color filters of the image sensor 101 are arranged in the primary color Bayer filter shown in FIG. 2, the color evaluation values Ex and Ey are defined by the following equations.

Ex = (RB) / Y
Ey = ((R + B) / 2− (G1 + G2) / 2) / Y
However, Y = 0.3R + 0.59G (= G1 or G2) + 0.11B. Further, R, G1, G2, and B in the above equation correspond to the color filters (R, G1, G2, and B) in FIG.

  FIG. 8 is a chromaticity diagram with the color evaluation values Ex and Ey as the horizontal axis and the vertical axis, and a diagram showing the coordinates of a plurality of types of light sources. The method of obtaining the coordinates for the plurality of types of light sources is, for example, a method in which a white point is photographed at the color temperatures of the plurality of types of light sources existing in the natural world, and the color evaluation values Ex and Ey at each color temperature are obtained.

  In the chromaticity diagram having the above-described color evaluation values Ex and Ey as the vertical axis and the horizontal axis, the above-described achromatic color determination range is set according to the following intensity parameter. Further, by plotting the coordinates of a plurality of types of light sources on the chromaticity diagram as described above, the set achromatic color determination range can follow the WB correction corresponding to any type of light source. Can be grasped. As described above, the following range parameter in the present embodiment is a parameter that specifies the color evaluation values Ex and Ey that specify the achromatic color determination range in the chromaticity diagram.

  Next, an example of a method in which the above-described WB correction coefficient determination circuit 510 determines a WB correction coefficient will be described.

  (1). First, the system controller 111 outputs an AWB tracking range signal corresponding to the tracking range set by the user to the tracking range determining circuit 512. The tracking range determination circuit 512 selects and determines a tracking range parameter from the tracking range parameter candidates 513 based on the input AWB tracking range signal, and inputs and sets the tracking range parameter to the WB correction coefficient determination circuit 510. Next, the WB correction coefficient determination circuit 510 sets an achromatic color determination range in the chromaticity diagram based on the set following range parameter. As described above, the achromatic color determination range according to the follow-up range set by the user is set.

  (2). Next, the WB correction coefficient determination circuit 510 divides the image signal input from the image sensor 101 into a plurality of small area (hereinafter, sample area) signals. Here, the sample area indicates one of the areas obtained by dividing one screen into m areas. Further, the WB correction coefficient determination circuit 510 calculates the color evaluation values Ex and Ey by using values obtained by averaging the R, G, and B signals existing in each sample area for each pixel.

  (3). Next, the WB correction coefficient determination circuit 510 determines whether or not the coordinates of the color evaluation values Ex and Ey obtained at each sample point are included in the specified white determination range. As a result, it is possible to determine a sample point required for obtaining a WB correction coefficient included in the achromatic color determination range set according to the following range.

  (4). When it is determined that the above-described color evaluation values Ex and Ey are included in the white determination range of the chromaticity diagram, the WB correction coefficient determination circuit 510 determines the R, G, and B values output for each pixel from the sample point for each color. Add each time.

  (5). Next, the WB correction coefficient determination circuit 510 performs the above-described processes (1) to (4) for all sample points on the screen.

  (6). Next, the WB correction coefficient determination circuit 510 obtains a gain for each color (for each RGB) so that the average value of the RGB signals added in the above (4) becomes the same level, and calculates the gain by the WB correction coefficient. And

  With the method described above, the WB correction coefficient determination circuit 510 can obtain the WB correction coefficient. That is, the imaging apparatus 100 changes the tracking range on the chromaticity diagram in the AWB correction process by changing the achromatic color determination range on the chromaticity diagram according to the setting of the AWB tracking range.

  According to the above-described embodiment, the user can arbitrarily set the following range, and performs the auto white balance correction according to the user's intention using the following range parameter according to the set following range. It becomes possible.

<Second embodiment>
(Arbitrarily specify the tracking range on the chromaticity diagram)
In the second embodiment, in the method of setting the following range parameter, a chromaticity diagram (including an achromatic color / white discrimination range) is displayed on the monitor device 110, and the user can view the displayed color while viewing the chromaticity diagram. A setting method for setting the achromatic color determination range to an arbitrary range on the degree diagram will be described. Note that the schematic configuration of the imaging device 100 is the same as that of the above-described embodiment, and thus the description is omitted. Hereinafter, only portions different from the first embodiment will be described.

  First, the user interface 112 displays, on the monitor device 110, a chromaticity diagram in which the achromatic color determination range as shown in FIG. 10 can be changed to an arbitrary range. FIG. 10 is an example showing an AWB following range setting screen. As shown in FIG. 10, the following range is set using the following variables in order to specify the upper or lower four sides of the achromatic color determination range on the chromaticity diagram. That is, in the chromaticity diagram, the achromatic color discrimination range is set by defining the following range in the low color temperature direction as T1, the following range in the high color temperature direction as T2, the following range in the magenta direction as Gr1, and the following range in the green direction as Gr2. I have. Also, in FIG. 10, the names of the various light sources are described in coordinates corresponding to the color evaluation values of the various light sources. The achromatic color determination range in the chromaticity diagram may be transparently displayed on the captured image as shown in FIG. This allows the user to check the set achromatic color determination range and the accompanying change in chromaticity in the captured image in real time.

  Next, when the user sets the achromatic color determination range on the chromaticity diagram of FIG. 10, the user interface 112 transmits the setting information of the achromatic color determination range to the system controller 111. Next, the system controller 111 outputs an AWB following range setting signal according to the setting information. Next, the following range determination circuit 512 specifies the above-described set values (color evaluation values) of T1, T2, Gr1, and Gr2 based on the input AWB following range setting signal. That is, the tracking range determination circuit 512 determines the set values of T1, T2, Gr1, and Gr2 as the tracking range parameters. Subsequent processing is the same as in the above-described embodiment, and a description thereof will be omitted.

  In the imaging device 100 described above, the user can easily set a desired follow-up range by designating the achromatic color determination range on the chromaticity diagram. Further, since the coordinates of various light sources are described in this chromaticity diagram, it is possible to easily set which type of light source is to be followed.

  For example, by changing the value of the tracking range parameter T1 for setting the achromatic color determination range on the high color temperature side, only the tracking range on the high color temperature side can be changed. Conversely, by changing the tracking range parameter T2 for setting the achromatic color determination range on the low color temperature side, only the tracking range on the low color temperature side can be changed. Further, by changing the tracking range parameters Gr1 and Gr2 for setting the achromatic color determination range in the green direction on the fluorescent lamp side, it is possible to change only the tracking range on the fluorescent lamp side.

  Further, here, the achromatic color determination range is set on the upper and lower four sides determined by the user arbitrarily setting T1, T2, Gr1, and Gr2, but either the color temperature axis or the magenta / green axis is set. May be arbitrarily set, or if one of the upper and lower limits is fixed, the other may be arbitrarily set by the user.

  Accordingly, in the imaging apparatus 100, the tracking range can be set for each color temperature or for each chromaticity, so that the free WB correction such as “follows a high color temperature but does not follow a low color temperature” can be performed. Settings can be made. That is, by allowing the user to arbitrarily set the AWB follow-up range, the degree of freedom is further increased, and the optimal AWB correction according to the user's preference, intention, and location can be realized.

  The relationship between the color temperature and the distance from the axis in the green direction, which are the set values (color evaluation values) of T1, T2, Gr1, and Gr2 on the chromaticity diagram described above, are determined by, for example, the following method. . First, as shown in FIG. 9 and FIG. 10, a color evaluation value obtained from photographing data of a white point previously photographed at a color temperature of a plurality of light sources existing in the natural world is plotted. The color evaluation value set in the achromatic color determination range can be calculated from the positional relationship between the coordinates of the various light sources (color temperature and distance in the green direction). At that time, the positional relationship can be specified by performing a mapping method or the like. Alternatively, the positional relationship may be grasped by calculation using a regression calculation process or the like, and may be obtained by the calculation.

  Although the achromatic color determination range is set here by the values (coordinates) of T1, T2, Gr1, and Gr2 on the chromaticity diagram, for example, the range surrounded by the dotted line in FIG. 10 displayed on the monitor device. The user is not limited to a quadrangle, but sets the achromatic color discrimination range by changing the size and size of a closed area such as an ellipse or a cloud, or moving the closed area surrounded by a dotted line to an arbitrary location. You may do so.

  In the above-described embodiment, the follow-up range setting screen for AWB correction shown in FIG. 10 is displayed on the monitor device 110 attached to the imaging device 100, but is not limited thereto. It may be a setting screen displayed on the screen of the computer terminal by an application or the like that can be executed on the terminal or the like. That is, an embodiment of the present invention is not limited to an imaging device, and may be an imaging system including an imaging device and a computer terminal.

<Third embodiment>
(Follow-up strength)
Note that the schematic configuration of the imaging device 100 is the same as that of the above-described embodiment, and thus the description is omitted. Hereinafter, only portions different from the above-described embodiment will be described.

  In the first and second embodiments, the achromatic color discrimination range is changed by the user designating the following range. In the third embodiment, however, the user sets the intensity of the following degree in the achromatic color discrimination range. The achromatic color determination range is set by selection.

  FIG. 3 also shows a change in the color tone of the image displayed on the monitor device 110 according to the selected following intensity, simultaneously with the display of the following intensity selection column 31. In FIG. 3, the background of the subject is white, and the light source is a light bulb. Here, in FIG. 3A, since the following intensity is set to “weak following”, the WB correction does not follow much under the light source of the bulb, and an image which is a background of a reddish color tone is displayed. I have. Further, in FIG. 3B, since the following strength is set to “strong following”, the following of the WB correction is completely performed, and an image with a white background is displayed.

  Further, the three following intensities correspond to, for example, three types of achromatic color determination areas in the chromaticity diagram shown in FIG. FIG. 4D is a diagram illustrating an example of a chromaticity diagram and an example of an achromatic color determination area according to the following intensity in the present embodiment. In the chromaticity diagram of FIG. 4D, the vertical axis indicates a change in chromaticity from green through white to magenta, and the horizontal axis indicates a change in color temperature. Reference numeral 41 denotes an achromatic color determination area in which the following strength corresponds to "weak following". Reference numeral 42 denotes an achromatic color determination area corresponding to the following strength “following”. Reference numeral 43 denotes an achromatic color determination area in which the following strength corresponds to “strong following”. The achromatic color determination areas 41 to 43 shown in FIG. 4D are specified according to the following parameters. In the chromaticity diagram of FIG. 4D, the coordinates of various light sources are written together with the names of the light sources.

As described above, it can be said that the following relationship exists between the following intensity and the following parameter as shown in FIG.
-When the following intensity is "weak following", a following parameter that follows only in the vicinity of sunlight in the sun-When the following intensity is "following", following parameters that follow from the bulb to near the shade-Following intensity Is "high tracking", a tracking parameter that is tracked by almost all light sources. Next, imaging when the user selects an arbitrary tracking strength on the AWB tracking strength selection screen shown in FIG. The operation of determining the following range parameter in the apparatus 100 will be described. FIG. 5B is a flowchart showing a process of determining a following range parameter in the imaging apparatus 100 shown in FIG. Note that the operation of the imaging apparatus 100 illustrated in FIG. 5B is started when the user operates the user interface 112 or the like.

  First, the user interface 112 causes the monitor device 110 to display an AWB following strength selection screen as shown in FIG. 3D (step S201). This prompts the user to select the following strength. Next, when the follow-up strength is not selected on the selection screen of FIG. 3D (NO in step S202), the display of the selection screen is maintained until it is selected (loop processing). When the following strength is selected on the selection screen of FIG. 3 (YES in step S202), the setting information regarding the selected following strength is transmitted from the user interface 112 to the system controller 111. Next, the system controller 111 outputs an AWB following intensity setting signal corresponding to the transmitted following intensity to the following range determining circuit 512 (step S203).

  Next, the tracking range determination circuit 512 determines a tracking range parameter corresponding to the input AWB tracking strength setting signal by referring to the tracking range parameter from the tracking range parameter candidate 513, and performs signal processing on the tracking range parameter. The data is transmitted to the circuit 102 (step S204). As described above, the imaging device 100 can set the following range parameter according to the following intensity selected by the user on the AWB following intensity selection screen. Thereby, the imaging apparatus 100 can perform the AWB correction using the following range parameter corresponding to the following intensity selected by the user.

  Note that the internal configuration of the above-described imaging device 100 is one embodiment, and any other configuration may be used as long as there is an imaging device 100 having a plurality of types of color filters and a configuration for performing the above-described AWB processing. . Further, in the above-described embodiment, there are three types of tracking intensity, but this is not a limitation, and an arbitrary number of types may be set as the tracking intensity. Further, the achromatic color determination area on the chromaticity diagram corresponding to the following intensity is not limited to the above, and may be an achromatic color determination area of various sizes and shapes.

  Further, even when a switch (not shown) corresponding to each light source is pressed to set an achromatic color determination area by combining areas corresponding to a plurality of light sources, the coordinates of the light sources can be designated by a selection key (not shown). Alternatively, the user may enter a numerical value as desired.

  The details of the WB correction processing for determining the WB correction coefficient by the WB correction coefficient determination circuit 510 in the above-described imaging apparatus 100 are the same as described above. In other words, the imaging apparatus 100 varies the achromatic color discrimination range on the chromaticity diagram according to the setting of the AWB following intensity, thereby varying the following range on the chromaticity diagram in the AWB correction process.

  Next, a description will be given of a specific example of the process in which the tracking range determination circuit 512 changes the tracking range (achromatic color determination range) on the chromaticity diagram.

  FIG. 10 is a diagram illustrating an example of a tracking range parameter for setting a tracking range (achromatic color determination range) on the chromaticity diagram by the tracking range determination circuit 512. As shown in FIG. 9, the achromatic color determination range on the chromaticity diagram is set using the following range parameters xs_1 to 3, xe_1 to 3, ys_1 to 3, and ye_1 to 3.

As shown in FIG. 10, a following range parameter corresponding to the following strength is set.
Tracking intensity = Achromatic color determination range with weak tracking [xs_1 <Ex <xe_1], [ys_1 <Ey <ye_1]
Tracking intensity = Achromatic color determination range during tracking [xs_2 <Ex <xe_2], [ys_2 <Ey <ye_2]
Tracking intensity = Achromatic color determination range with strong tracking [xs_3 <Ex <xe_3], [ys_3 <Ey <ye_3]
At this time, as is clear from FIG. 9, the magnitude relationship of each following range parameter according to the following strength is as follows.

xs_1>xs_2> xs_3
xe_1 <xe_2 <xe_3
ys_1>ys_2> ys_3
ye_1 <ye_2 <ye_3
In other words, it can be said that the relationship is such that the achromatic color determination range of “strong following”> the achromatic color determination range of “under tracking”> the achromatic color determination range of “weak following”. This is because the wider the achromatic color determination range, the wider the tracking area during WB correction, and is an inequality expression indicating the relationship between the tracking intensity during WB correction and the achromatic color determination range.

  Further, the following range parameters can be independently adjusted. For example, by changing the values of the tracking range parameters xs_1 to 3 for setting the achromatic color determination range on the high color temperature side, only the tracking range on the high color temperature side can be changed. Conversely, by changing the tracking range parameters xe_1 to xe_3 which set the achromatic color determination range on the low color temperature side, only the tracking range on the low color temperature side can be changed. Further, by changing the following range parameters ye_1 to 3 for setting the achromatic color determination range in the green direction on the fluorescent lamp side, only the following range for the fluorescent lamp side can be changed.

  As described above, according to the above-described embodiment, by displaying the setting screen capable of displaying the blackbody radiation axis capable of setting the degree of tracking of the white balance to an arbitrary intensity, the user can optimize the white balance correction. Can be selected by visually judging the color.

<Fourth embodiment>
(Combination of brightness and shooting mode)
Next, a fourth embodiment of the present invention will be described.
In the second embodiment, the automatic white balance is set by the user in the achromatic color determination range, and in the third embodiment, the achromatic color determination range is changed from the setting of the following intensity set by the user. explained. Further, a case where the achromatic color determination range is set based on a combination of a user setting and a shooting condition such as a shooting mode and the brightness of a subject will be described.

  For example, when the subject is bright as in the outdoor mode, the area that can be set on the high color temperature side portion is limited, and the user sets the achromatic color determination range within that area. Similarly, in the case of shooting when the subject is dark as in the indoor mode, the range that can be set on the low color temperature side is limited.

  Thereafter, in the achromatic color determination area limited to a predetermined range according to the brightness, the user further sets an arbitrary range in the chromaticity diagram from green on the vertical axis to the magenta direction and the color temperature direction on the horizontal axis.

  According to the above-described embodiment, the area that can be set according to the brightness is limited, so that the user can set a desired achromatic color determination range in consideration of the brightness. Here, it has been described that the achromatic color determination area that can be set is limited according to the shooting mode based on the brightness. However, in addition to the brightness, a mode according to the type of the subject such as a landscape or a portrait, etc. The achromatic color determination area that can be set may be changed according to the shooting conditions.

  In the chromaticity diagrams described in Examples 1 to 4, the vertical axis indicates the chromaticity changing from green to magenta, and the horizontal axis indicates the change in color temperature. Various colors such as xy chromaticity diagram of chromaticity diagram, uv chromaticity diagram of Luv chromaticity diagram, u'v 'chromaticity diagram of Lu'v' chromaticity diagram, ab chromaticity diagram of Lab chromaticity diagram A diagram may be used. For example, the chromaticity diagram may be such that a black body radiation axis or a white axis corresponding thereto is used as a first coordinate axis, and a change in chromaticity that changes from green to magenta is used as a second coordinate axis. Also, the achromatic color determination area described above is not limited to the area defined by the line parallel to the horizontal axis indicating the change in color temperature, and the black plane radiating axis (curve) or the corresponding white axis (Curve) may be drawn, and the area may be equidistant in the vertical axis direction from the curve.

  In the case of a monitor device capable of three-dimensional display, the chromaticity diagram may display a diagram showing a three-dimensional or multidimensional color space.

  Each process performed by the system controller 111 shown in FIG. 1 may be realized by dedicated hardware. The system controller is configured by a memory and a CPU, and is a program for realizing each process. May be realized by reading the program into a memory and executing the program.

  The memory is a non-volatile memory such as a hard disk device, a magneto-optical disk device, or a flash memory, a recording medium such as a CD-ROM, which is readable only, or a volatile memory such as a RAM (Random Access Memory). Or a computer-readable and writable recording medium by a combination thereof.

  Although the above embodiments have been described by taking a digital camera as an example, the present invention is not limited to a digital camera, but also includes a digital video camera, a mobile phone with a digital camera, a scanner, and the like. In addition, a camera that performs imaging by a remote operation such as issuing a release instruction from the PC while the camera and the PC are connected by wire or wirelessly is also included.

  Further, an object of the present invention is to supply a recording medium (or a storage medium) recording a program for realizing the functions of the above-described embodiments to an imaging system or an imaging device, and to execute the computer of the imaging system or the imaging device on the recording medium. This is also achieved by reading and executing the program stored in the. Also, based on the instructions of the program read by the computer, an operating system (OS) or the like running on the computer performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments. The case is also included.

  Further, after the program code read from the storage medium is written into the memory provided on the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. A CPU or the like provided in the board or the function expansion unit may perform part or all of the actual processing, and the functions of the above-described embodiments may be realized by the processing.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the embodiments, and includes a design and the like within a range not departing from the gist of the present invention.

FIG. 1 is a block diagram illustrating a schematic configuration of an imaging device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of an array of color filters included in the image sensor 101 illustrated in FIG. 1. FIG. 2 is a diagram illustrating an example of a screen for selecting an AWB tracking intensity displayed on the imaging device 100 illustrated in FIG. 1. FIG. 3 is a diagram illustrating an example of a chromaticity diagram and an example of an achromatic color determination area according to a tracking intensity according to the embodiment. FIG. 2 is a flowchart showing a process of determining a follow-up range parameter in the imaging apparatus 100 shown in FIG. 1. FIG. 2 is a diagram illustrating an example of an internal configuration of a signal processing circuit 102 illustrated in FIG. 1. FIG. 7 is a diagram illustrating a detailed configuration of a WB circuit 502 illustrated in FIG. 6. FIG. 3 is a chromaticity diagram with color evaluation values Ex and Ey as abscissa and ordinate, and a diagram showing coordinates of a plurality of types of light sources. FIG. 9 is a diagram illustrating an example of a tracking range parameter for setting a tracking range (achromatic color determination range) on a chromaticity diagram by a tracking range determination circuit 512; It is a figure showing the example showing the AWB following range specification screen in this embodiment.

Explanation of reference numerals

Reference Signs List 101 imaging device 102 signal processing circuit 110 monitoring device 112 user interface

Claims (40)

An image sensor that converts light from a subject into an image signal;
A signal processing circuit for processing the imaging signal, wherein:
Display means for displaying a diagram related to the blackbody radiation axis;
User interface means for a user to perform settings related to image processing of the imaging signal in the signal processing circuit,
The imaging apparatus according to claim 1, wherein the user interface unit performs the setting while displaying a display regarding a blackbody radiation axis on the display unit.   The display related to the black body radiation axis is a chromaticity diagram.   The imaging apparatus according to claim 2, wherein the display unit displays a display based on blackbody radiation in a chromaticity diagram together with the imaging signal.   The imaging apparatus according to claim 2, wherein the display unit switches between an imaging signal from the imaging device and a display screen of the chromaticity diagram based on an operation of the user interface unit.   The imaging device according to claim 2, wherein the display unit displays the chromaticity diagram transparently on the captured image. Furthermore, it has a white balance processing means for performing white balance processing of the imaging signal,
The user interface unit sets an achromatic color determination range for white balance processing in the chromaticity diagram,
The imaging apparatus according to claim 2, wherein the white balance processing unit performs white balance processing based on the achromatic color determination range set by the user interface unit.   3. The imaging apparatus according to claim 2, wherein in the chromaticity diagram, one coordinate axis is a color temperature, and the other coordinate axis is a coordinate axis different from the color temperature.   The coordinate axis of a chromaticity different from the color temperature in the chromaticity diagram is a green / magenta direction, and the coordinate axis of the color temperature is a blackbody radiation axis or an achromatic color axis corresponding thereto. Item 8. The imaging device according to Item 7.   The imaging apparatus according to claim 2, wherein the chromaticity diagram is an xy chromaticity diagram, a uv chromaticity diagram, or a u'v 'chromaticity diagram, which is a chromaticity diagram of the CIE color system.   The imaging apparatus according to claim 6, wherein the user interface unit sets the size of the achromatic color determination range in the color temperature direction.   The imaging apparatus according to claim 6, wherein the user interface unit sets the size of the achromatic color determination range in the hue direction.   The imaging apparatus according to claim 6, wherein the user interface unit arbitrarily sets an upper limit and / or a lower limit of the achromatic color determination range.   The imaging apparatus according to claim 6, wherein the user interface unit sets the achromatic color determination range using coordinates on the chromaticity diagram.   The imaging apparatus according to claim 6, wherein the user interface unit changes the shape of the achromatic color determination range represented by a closed area on the chromaticity diagram.   The imaging apparatus according to claim 6, wherein the user interface unit sets the achromatic color determination range by changing a position of a closed area on the chromaticity diagram.   The imaging apparatus according to claim 6, wherein an achromatic color determination range that can be set by the user interface unit is changed according to a shooting condition of the subject.   7. The imaging apparatus according to claim 6, wherein an achromatic color determination range that can be set by the user interface means is limited to a high color temperature side and / or a low color temperature side according to brightness of a subject. An image sensor that converts light from a subject into an image signal;
An imaging device having a signal processing circuit that processes the imaging signal,
A tracking intensity setting means for arbitrarily setting the tracking intensity of the white balance by a user;
An image capturing apparatus comprising: a white balance processing unit configured to perform a white balance process on the imaging signal based on the tracking degree set by the tracking degree setting unit.   19. The imaging apparatus according to claim 18, wherein the following strength is a size of a following range.   20. The imaging apparatus according to claim 19, further comprising display means for displaying the size of the following range together with a chromaticity diagram.   19. The imaging apparatus according to claim 18, wherein the following strength setting means displays the setting screen on which a user can select a desired following strength from a plurality of kinds of following strengths.   19. The tracking intensity corresponds to a color temperature range in which achromatic color can be determined, and the tracking intensity setting means displays a tracking intensity setting screen capable of changing the color temperature range. An imaging device according to item 1.   23. The imaging apparatus according to claim 22, wherein the following intensity setting unit displays a following intensity setting screen for changing the color temperature range by changing the upper limit of the color temperature.   23. The imaging apparatus according to claim 22, wherein the following intensity setting unit displays a following intensity setting screen for changing the color temperature range by changing the lower limit of the color temperature.   19. The tracking intensity according to claim 18, wherein the tracking intensity corresponds to a hue range in which achromatic color can be determined, and the tracking intensity setting means displays a tracking intensity setting screen for changing the hue range. Imaging device.   26. The imaging apparatus according to claim 25, wherein the following intensity setting means displays a following intensity setting screen for changing a hue range in a green direction as the hue range.   The following intensity corresponds to the width of an achromatic color determination range on a chromaticity diagram using a change in color temperature and hue as a coordinate axis, and the following intensity setting means is used to change the width of the achromatic color determination range. 21. The imaging apparatus according to claim 20, wherein a setting screen is displayed. An image sensor that converts light from a subject into an image signal;
An imaging device having a signal processing circuit that processes the imaging signal,
Display means for displaying a setting screen representing the tracking strength of white balance;
Achromatic color discrimination parameter determining means for determining an achromatic color discrimination parameter corresponding to a tracking intensity arbitrarily set by a user on the setting screen,
White balance processing means for performing white balance processing on the imaging signal using the achromatic color determination parameter determined by the achromatic color determination parameter determination means,
An imaging device comprising:   29. The imaging apparatus according to claim 28, wherein the degree of following the white balance corresponds to the size of an achromatic color determination range on a chromaticity diagram.   29. The imaging apparatus according to claim 28, wherein the display unit displays a color corresponding to the setting of the degree of following on the setting screen.   29. The imaging apparatus according to claim 28, wherein the user interface unit arbitrarily sets an upper limit and / or a lower limit of the achromatic color determination range.   29. The imaging apparatus according to claim 28, wherein the user interface unit sets the achromatic color determination range using coordinates on the hue diagram.   29. The imaging apparatus according to claim 28, wherein the user interface unit changes the shape of the achromatic color determination range represented by a closed area on the chromaticity diagram.   29. The imaging apparatus according to claim 28, wherein the user interface unit changes a position of a closed region on the chromaticity diagram in the achromatic color determination range of the achromatic color determination range. The light from the subject is converted into an image signal by an image sensor,
Processing an image signal from the image sensor;
Display about the blackbody radiation axis,
An imaging method, wherein the setting is performed in a state where a display regarding a blackbody radiation axis is displayed on the display unit. The light from the subject is converted into an image signal by an image sensor,
Processing an image signal from the image sensor;
Display the setting screen indicating the degree of white balance tracking,
Determine the achromatic color discrimination parameter corresponding to the follow-up degree arbitrarily set by the user on the setting screen,
An imaging method, wherein white balance processing is performed on the imaging signal using the determined achromatic color determination parameter. A computer-readable recording medium recording a display program of an imaging device that processes an imaging signal from an imaging device,
A computer-readable recording medium storing a program for causing a computer of the imaging apparatus to execute the imaging method according to claim 35. A computer-readable recording medium that records a program for an imaging device that processes an imaging signal from an imaging device,
A computer-readable recording medium storing a program for causing a computer of the imaging apparatus to execute the imaging method according to claim 36. A program for an imaging device that processes an imaging signal from an imaging device,
A program for causing a computer of the imaging apparatus to execute the imaging method according to claim 35. A program for an imaging device that processes an imaging signal from an imaging device,
A program for causing a computer of the imaging apparatus to execute the imaging method according to claim 36.

Images