JP2022034185A - Color adjustment method - Google Patents

Abstract

To perform proper white balance control independently of a subject size and a background color.SOLUTION: Provided is a color adjustment method including first determination processing SA110, first adjustment processing SA120, and second adjustment processing SA130 as follows. At the first determination processing SA110, it is determined whether or not a change amount ΔL in a brightness adjustment amount of a picked-up image captured by an imaging device is larger than a first threshold th1. The first adjustment processing SA120 is executed in a case where the determination result at the first determination processing SA110 is negative. At the first adjustment processing SA120, a color of the picked-up image is adjusted on the basis of an average value of a color temperature in the picked-up image. The second adjustment processing SA130 is executed in a case where the determination result at the first determination processing SA110 is affirmative. At the second adjustment processing SA130, the color of the picked-up image is adjusted on the basis of an average value of a color temperature in a correction image obtained by applying processing of decreasing brightness to the picked-up image.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a color adjustment method for captured images.

For example, in a digital camera, the type of light source at the time of imaging is estimated from the captured image, and color adjustment is performed so that a region that should be white in the captured image is correctly white. This color adjustment is called white balance control. White balance control refers to control that makes the appearance of gray scale seen under a light source other than a fluorescent lamp such as a red lamp the same as when viewed under a fluorescent lamp. Examples of the prior art relating to white balance control include the technique disclosed in Patent Document 1 and the technique disclosed in Patent Document 2.

Japanese Unexamined Patent Publication No. 201672916 Japanese Unexamined Patent Publication No. 2-272892

In the prior art, the type of light source may be erroneously determined due to the relationship between the size of the subject and the color of the background, and the white balance control may not be performed correctly. For example, suppose that a small piece of white paper placed on a reddish-brown desk, which is close to the color of a red light, is imaged under a fluorescent light, and most of the captured image is a desk that is the background rather than the paper that is the subject. In this case, it is erroneously determined that the light source is a red light, and the paper that should be originally white is displayed with a bluish tint in the captured image.

In order to solve the above problems, the color adjustment method of the present disclosure determines whether or not the amount of change in the adjustment amount of the brightness of the captured image captured by the image pickup device exceeds the first threshold value, and the determination is made. If the result is negative, the color of the captured image is adjusted based on the color temperature of the captured image, and if the result of the determination is positive, the captured image is subjected to a process of reducing the brightness. The color of the captured image is adjusted based on the color temperature of the corrected image obtained by the application.

Further, in order to solve the above-mentioned problems, in another aspect of the present disclosure, the color adjustment method determines whether or not the amount of change in the adjustment amount of the brightness of the image captured by the image pickup device exceeds the first threshold value. When it is determined that the amount of change in the adjustment amount exceeds the first threshold value, the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and more than the second threshold value. It is determined whether or not the color is small, and if the value is determined to be zero, if the value is determined to be equal to or higher than the second threshold value, or if the change amount of the adjustment amount is determined to be equal to or lower than the first threshold value. In this case, the color of the captured image is adjusted based on the color temperature of the captured image, and when it is determined that the value is larger than zero and smaller than the second threshold value, the captured image is bright. The color of the captured image is adjusted based on the color temperature in the corrected image obtained by performing the process of reducing the color.

Further, in order to solve the above-mentioned problems, the color adjustment method of still another aspect of the present disclosure determines whether or not the amount of change in the amount of adjustment of the brightness of the image captured by the image pickup device exceeds the first threshold value. When it is determined that the amount of change in the adjustment amount exceeds the first threshold value, the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and the second threshold value is determined. If it is determined that the value is smaller than or equal to the second threshold value, it is determined whether or not the change amount of the value is smaller than the third threshold value, and the change of the adjustment amount is determined. When the amount is determined to be equal to or less than the first threshold value, the value is determined to be zero, or the amount of change in the value is determined to be equal to or greater than the third threshold value, the color temperature in the image pickup apparatus is determined. Based on this, the color of the captured image is adjusted, and it is determined that the amount of change in the value is smaller than the third threshold value, or the value is larger than zero and smaller than the second threshold value. In the case, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness.

Further, in order to solve the above-mentioned problems, the color adjustment method of still another aspect of the present disclosure determines whether or not the amount of change in the amount of color adjustment of the image captured by the image pickup device exceeds the first threshold value. If the result of the determination is negative, the color of the captured image is adjusted based on the color temperature of the captured image, and if the result of the determination is positive, the brightness of the captured image is adjusted. The color of the captured image is adjusted based on the color temperature in the corrected image obtained by performing the process of reducing.

It is a block diagram which shows the structural example of the camera 1A which executes the color adjustment method which concerns on 1st Embodiment of this disclosure. It is a figure for demonstrating white balance control. It is a flowchart which shows the flow of the color adjustment method which the processing apparatus 10 of a camera 1A executes according to a program PA. It is a figure for demonstrating the problem of the conventional white balance control. It is a figure for demonstrating the erroneous determination of a light source. It is a figure for demonstrating the effect of this embodiment. It is a block diagram which shows the structural example of the camera 1B which executes the color adjustment method which concerns on 2nd Embodiment of this disclosure. It is a flowchart which shows the flow of the color adjustment method which the processing apparatus 10 of a camera 1A executes according to a program PB. It is a figure for demonstrating the problem when the light source actually changes. It is a block diagram which shows the structural example of the camera 1C which executes the color adjustment method which concerns on 3rd Embodiment of this disclosure. It is a flowchart which shows the flow of the color adjustment method which the processing apparatus 10 of a camera 1A executes according to a program PC. It is a figure for demonstrating a problem when there is a pattern having an edge in the background. It is a figure for demonstrating the effect of this embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below are subject to various technically preferred limitations. However, the embodiments of the present disclosure are not limited to the embodiments described below.

1. 1. First Embodiment FIG. 1 is a block diagram showing a configuration example of the camera 1A of the first embodiment of the present disclosure. The camera 1A is a digital camera having a function of automatically executing exposure compensation and white balance control. Exposure compensation refers to adjusting the brightness of the entire captured image according to, for example, the average value of the brightness constituting the captured image. As shown in FIG. 1, the camera 1A includes a processing device 10, an image pickup device 20, an image processing device 30, an output device 40, and a storage device 50.

The processing device 10 includes a processor such as a CPU (Central Processing Unit). The processing device 10 may be configured by a single processor or may be configured by a plurality of processors. The processing device 10 functions as a control center of the camera 1A by operating according to the program PA described later. The processing device 10 operating according to the program PA automatically executes exposure compensation and white balance control.

The image pickup device 20 is, for example, an image sensor. The image pickup device 20 takes an image of a subject according to the operation of the user of the camera 1A, and writes image data representing the captured image in the storage device 50. Hereinafter, the image data output from the image pickup apparatus 20 will be referred to as the original image data. The image processing device 30 is, for example, an image processing LSI (Large Scale Integration). The image processing device 30 reads the original image data from the storage device 50 under the control of the processing device 10, and performs image processing for exposure compensation according to the exposure compensation value stored in the storage device 50. Apply to. The image processing device 30 writes image data representing the exposure-corrected image in the storage device 50. In the following, image data representing an image with exposure compensation may be referred to as corrected image data.

Further, the image processing device 30 performs image processing of white balance control on the corrected image data under the control of the processing device 10 to generate image data representing the color-adjusted image. In the following, image data representing a color-adjusted image may be referred to as adjusted image data. Although the details will be described later, in the present embodiment, the type of the light source is specified according to the average value of the color temperature of the pixels in the image. For example, if the average value of the color temperature in the image captured with exposure compensation is around 2400K, the type of the light source is specified as a red lamp. Further, if the average value of the color temperature in the image captured with exposure compensation is a value in the vicinity of 4150K, the type of the light source is specified as a fluorescent lamp. Then, the processing device 10 generates a filter coefficient of the color adjusting filter according to the type of the light source specified based on the average value of the color temperature. The color adjustment filter is a filter that adjusts the colors of the pixels constituting the captured image for each of the red, green, and blue color components so that the appearance of the subject is the same as that under fluorescent lighting. Adjusted image data is generated by applying the filter processing by this color adjustment filter to the corrected image data.

For example, as shown in FIG. 2, it is assumed that the image G01 is imaged under a red light with an A4 size white paper OB1 placed on a gray desk TBL1 as a subject. The color of the paper OB1 is white, but since the image G01 is imaged under a red light, the color of the paper OB1 in the image G01 is red. In FIG. 2, the hatching of the diagonal line rising to the right represents red, and the hatching of the vertical line represents reddish gray. When the average value of the color temperature of the image G01 is calculated to be around 2400K, the processing apparatus 10 determines that the light source is a red lamp. As a result, the processing apparatus 10 generates a filter coefficient of a color adjustment filter for a red lamp, that is, a filter coefficient of a color adjustment filter that decreases red and increases blue. When the image processing device 30 performs the filter processing according to the filter coefficient on the image G01, the color of the desk TBL1 as the background becomes gray and the color of the paper OB1 as the subject becomes gray as shown in FIG. The white image G02 is generated.

The output device 40 is a collection of output interfaces conforming to various standards such as USB (Universal Serial Bus). Various external devices such as a personal computer are connected to the output device 40. The output device 40 reads the adjusted image data stored in the storage device 50 under the control of the processing device 10, and outputs the read adjusted image data to the external device.

The storage device 50 includes a non-volatile memory such as a flash memory and a volatile memory such as a RAM (Random Access Memory). The non-volatile memory stores the program PA executed by the processing device 10. In the present embodiment, the program PA is stored in the non-volatile memory in advance at the time of shipment from the factory of the camera 1A. The program PA may be acquired from a server (not shown) or the like after the camera 1A is shipped from the factory, and then stored in the non-volatile memory. Further, in the non-volatile memory, data and the like indicating a first threshold value to be described later and a reduction width when the exposure is reduced and re-corrected are stored in advance.

The volatile memory of the storage device 50 stores the original image data, the exposure-corrected image data, and the adjusted image data. Further, the exposure compensation value is stored in the storage device 50. This exposure compensation value is updated by the processing device 10 according to the average value of the brightness of the pixels constituting the image represented by the original image data. When updating the exposure compensation value, the processing device 10 separately stores the exposure compensation value before the update in the storage device 50, and then updates the exposure compensation value. Further, the storage device 50 stores the filter coefficient of the color adjustment filter as a parameter defining the white balance control. When updating the parameter defining the white balance control, the processing device 10 separately stores the parameter before the update in the storage device 50, and then updates the parameter.

The processing device 10 starts executing the program PA when the power of the camera 1A is turned on. The processing device 10 operating according to the program PA executes a color adjustment method that remarkably exhibits the characteristics of the present embodiment each time an image is captured by the image pickup device 20. FIG. 3 is a flowchart showing a flow of a color adjustment method executed by the processing device 10 of the camera 1A according to the program PA. As shown in FIG. 3, the color adjustment method of the present embodiment includes the first determination process SA110, the first adjustment process SA120, and the second adjustment process SA130. The processing contents of the first determination processing SA110, the first adjustment processing SA120, and the second adjustment processing SA130 are as follows.

In the first determination processing SA110, the processing apparatus 10 determines whether or not the change amount ΔL of the exposure compensation value exceeds the first threshold value th1. The change amount ΔL of the exposure compensation value is an example of the change amount of the brightness adjustment amount of the captured image in the present disclosure. More specifically, the processing device 10 sets the absolute value of the difference between the current value of the exposure compensation value stored in the storage device 50 and the value before the update as the change amount ΔL of the exposure compensation value. When the change amount ΔL of the exposure compensation value exceeds the first threshold value th1, the judgment result of the first determination process SA110 is “Yes”, and when the change amount ΔL of the exposure compensation value is equal to or less than the first threshold value th1. 1 The result of the determination of the determination process SA110 is “No”. Even when the exposure compensation value before the update is not stored in the storage device 50, that is, when the exposure compensation value is not updated, the determination result of the first determination process SA110 is “No”.

When the determination result of the first determination process SA110 is "No", that is, when the determination result of the first determination process SA110 is negative, the processing apparatus 10 executes the first adjustment process SA120. When the determination result of the first determination process SA110 is "Yes", that is, when the determination result of the first determination process SA110 is affirmative, the processing apparatus 10 executes the second adjustment process SA130. As described above, in the present embodiment, the processing is switched according to whether or not the change amount ΔL of the exposure compensation value exceeds the first threshold value th1, in other words, whether or not the exposure compensation value changes abruptly. It is known that the exposure compensation value changes abruptly when an erroneous determination of the light source occurs, and this is to deal with the erroneous determination of the light source. The specific value of the first threshold value th1 is described. An appropriate value may be set by conducting an appropriate experiment.

In the first adjustment processing SA120, the processing apparatus 10 adjusts the color of the captured image based on the average value of the color temperature in the image represented by the corrected image data. That is, in the first adjustment processing SA120, the processing device 10 specifies the type of the light source from the average value of the color temperature of the pixels in the image represented by the corrected image data, and the color adjustment filter according to the specified type of the light source. To generate. Using this color adjustment filter, color adjustment of the image represented by the corrected image data, that is, white balance control is executed. The first adjustment process SA120 is the same process as the conventional white balance control.

In the second adjustment process SA130, the processing device 10 first applies a process of reducing the brightness to the corrected image data to generate re-corrected image data representing the corrected image. This is to generate a corrected image close to monochrome so that the influence of the background color is reduced. Next, the processing device 10 adjusts the color of the captured image based on the average value of the color temperature in the corrected image represented by the re-corrected image data. That is, in the second adjustment processing SA130, the processing device 10 specifies the type of the light source from the average value of the color temperature of the pixels in the image represented by the recorrected image data, and for color adjustment according to the specified type of the light source. Generate a filter. Using this color adjustment filter, color adjustment of the image represented by the corrected image data, that is, white balance control is executed.

As described above, it is known that when an erroneous determination of a light source occurs, a sudden change in the exposure compensation value occurs. For example, as shown in FIG. 4, it is assumed that an image is taken under a fluorescent lamp with a business card-sized white paper OB2 placed on a reddish brown desk TBL2 as a subject, and an image G03 is imaged. The hatching of the horizontal line in FIG. 4 represents reddish brown. When exposure compensation is performed based on this image G03, the background color of reddish brown occupies most of the image G03, so that it is erroneously determined that the light source is a red lamp as shown in FIG. The exposure compensation value is raised as described above. Each value of R, G, and B in the image to which the exposure compensation is performed according to the exposure compensation value after this update becomes larger than that when the exposure is low. In this example, the value of R becomes large. When the conventional white balance control is executed in this state, a color adjustment filter for a red light is generated, and a bluish image G04 is generated as a color-adjusted image. In FIG. 4, the hatching of the grid lines represents bluish reddish brown, and the hatching of the diagonal lines descending to the right shoulder represents blue.

On the other hand, in the present embodiment, when there is a sudden change in the exposure compensation value such that the change amount ΔL of the exposure compensation value exceeds the first threshold value th1, the determination result of the first determination process SA110 is The result is affirmative, and the second adjustment process SA130 is executed. As described above, in the second adjustment process SA130, the corrected image data is generated by performing a process of reducing the brightness of the corrected image data, and the color temperature in the corrected image represented by the re-corrected image data is measured. A color adjustment filter is generated based on the average value. In the corrected image, the values of R, G, and B are reduced as compared with the adjusted image, and the color adjustment filter for fluorescent lamps as the color adjustment filter, that is, any of the colors of red, green, and blue is also reduced. A color adjustment filter is generated that does not cause or increase. Since the white balance control is executed using this color adjustment filter, as shown in FIG. 6, the paper OB2 which should be originally white is white for the image G03, and should be originally reddish brown. An image in which the desk TBL2 is reddish brown, that is, an image G05 unchanged from the image G03 is generated as a color-adjusted image. The hatching of the horizontal line in FIG. 6 represents reddish brown.

As described above, according to the camera 1A of the present embodiment, the white balance control can be correctly performed even when the subject is small with respect to the background and the background color is close to the color of the red light. In the present embodiment, the case where the background color is close to the color of the red lamp has been described, but the reason why the corrected image is generated by the second adjustment process SA130 is to reduce the influence of the background color. , The same effect can be obtained even if the background color is a bluish color. As described above, according to the camera 1A of the present embodiment, it is possible to correctly perform the white balance control regardless of the size of the subject and the color of the background.

2. 2. Second Embodiment Next, the camera 1B of the second embodiment of the present disclosure will be described. The camera 1B of the present embodiment is a digital camera that automatically performs exposure compensation and white balance control in the same manner as the camera 1A. FIG. 7 is a block diagram showing a configuration example of the camera 1B. In FIG. 7, the same components as those in FIG. 1 are designated by the same reference numerals. As is clear from the comparison between FIGS. 7 and 1, the difference in the configurations of the camera 1B and the camera 1A is that the program PB is stored in the storage device 50 instead of the program PA.

The processing device 10 of the camera 1B starts executing the program PB when the power of the camera 1B is turned on. The processing device 10 operating according to the program PB executes a color adjustment method that remarkably exhibits the characteristics of the present embodiment each time an image is captured by the image pickup device 20. FIG. 8 is a flowchart showing a flow of a color adjustment method executed by the processing device 10 of the camera 1B according to the program PB. In FIG. 8, the same components as those in FIG. 3 are designated by the same reference numerals. That is, the processing contents of the first determination processing SA110, the first adjustment processing SA120, and the second adjustment processing SA130 in FIG. 8 are the same as the processing contents of each of these processes in the color adjustment method of the first embodiment. As is clear from the comparison between FIGS. 8 and 3, the color adjustment method in the present embodiment is different from the color adjustment method in the first embodiment in that the second determination process SA112 is included.

As shown in FIG. 8, the second determination process SA112 is executed when the first determination process SA110 determines that the change amount ΔL of the exposure compensation value exceeds the first threshold value th1. When the first determination process SA110 determines that the change amount ΔL of the exposure compensation value is equal to or less than the first threshold value th1, the first adjustment process SA120 is executed as in the first embodiment.

In the second determination process SA112, in the processing device 10 of the camera 1B, the value S corresponding to the size of the region partitioned by the edges in the image represented by the corrected image data is larger than zero, and the value S is the second threshold value. It is determined whether or not it is smaller than th2. When it is determined that the value S is larger than zero and smaller than the second threshold value th2, the processing device 10 executes the second adjustment processing SA130. On the other hand, when the value S is zero, or when the value S is the second threshold value th2 or more, the processing apparatus 10 executes the first adjustment processing SA120.

An example of an edge is a boundary point between a subject and a background in an image, specifically, a change point in which the brightness changes significantly from bright to dark or from dark to bright. By connecting these boundary points, the contour line of the subject is formed. In the following, the value corresponding to the size of the area partitioned by the edge is referred to as an edge size. Specific examples of the edge size include the area of the area partitioned by the edges, the length of the contour line of the area, or the number of pixels included in the area. Existing techniques may be appropriately used for edge detection and edge size calculation. Further, the second threshold value th2 may be appropriately set by conducting an experiment as appropriate.
The above is the flow of the color adjustment method in this embodiment.

In the color adjustment method of the first embodiment, the second adjustment process SA130 is executed even when the light source actually changes from the fluorescent lamp to the red lamp and the exposure compensation value is updated due to the change of the light source. , Color-adjusted images that do not match the actual light source may be generated. For example, it is assumed that the light source is changed from a fluorescent lamp to a red lamp when an image is taken with a white A4 size paper OB1 placed on a gray desk TBL1 as a subject, and the image G06 of FIG. 9 is obtained. In FIG. 9, the hatching of the diagonal line rising to the right represents red, and the hatching of the vertical line represents reddish gray. In this case, in the color adjustment method of the first embodiment, the second adjustment process SA130 is executed even if the exposure compensation value is increased according to the change of the light source. As a result, the white balance control is executed using the color adjustment filter for the fluorescent lamp, and a color-adjusted image in which the color of the subject is reddish as shown in the image G06 of FIG. 9 is generated.

On the other hand, in the color adjustment method of the present embodiment, when the first determination process SA110 determines that the change amount ΔL of the exposure compensation value exceeds the first threshold value th1, the second determination process SA112 is executed. Will be done. As described above, the second determination process SA112 is a process for determining whether or not the size of the edge size, that is, the size of the subject in the captured image is larger than zero and smaller than the second threshold value. If the second threshold value is set to a value corresponding to the A4 size, the first adjustment process SA120, that is, the conventional white balance control is executed when the image G06 of FIG. 9 is captured. As a result, as in the image G02 of FIG. 2, a color-adjusted image is generated in which the color of the desk TBL1 as the background is gray and the color of the paper OB1 as the subject is white.

Similarly to the camera 1A of the first embodiment, the camera 1B of the present embodiment can correctly perform white balance control regardless of the size of the subject and the color of the background. In addition, according to the camera 1B of the present embodiment, it is possible to avoid erroneous determination when the light source actually changes.

3. 3. Third Embodiment Next, the camera 1C of the third embodiment of the present disclosure will be described. The camera 1C of the present embodiment is a digital camera that automatically performs exposure compensation and white balance control in the same manner as the camera 1B. FIG. 10 is a block diagram showing a configuration example of the camera 1C. In FIG. 10, the same components as those in FIG. 7 are designated by the same reference numerals. As is clear from the comparison between FIGS. 10 and 7, the difference in the configurations of the camera 1C and the camera 1B is that the program PC is stored in the storage device 50 instead of the program PB.

The processing device 10 of the camera 1C starts executing the program PC when the power of the camera 1C is turned on. The processing device 10 operating according to the program PC executes a color adjustment method that remarkably exhibits the characteristics of the present embodiment each time an image is captured by the image pickup device 20. FIG. 11 is a flowchart showing a flow of a color adjustment method executed by the processing device 10 of the camera 1C according to the program PC. In FIG. 11, the same components as those in FIG. 8 are designated by the same reference numerals. That is, the processing contents of the first determination processing SA110, the second determination processing SA112, the first adjustment processing SA120, and the second adjustment processing SA130 in FIG. 11 are the processing of each of these processes in the color adjustment method of the second embodiment. It is the same as the content. As is clear from comparing FIGS. 11 and 8, the color adjustment method in the present embodiment is different from the color adjustment method in the second embodiment in that the third determination process SA114 is included.

As shown in FIG. 11, the third determination process SA114 determines in the second determination process SA112 that the value S corresponding to the size of the closed region partitioned by the edges in the captured image is equal to or greater than the second threshold value th2. Will be executed if it is done. When the second determination process SA112 determines that the value S is larger than zero and smaller than the second threshold value th2, the second adjustment process SA130 is executed as in the second embodiment. Further, when the value S is determined to be zero by the second determination process SA112, the first adjustment process SA120 is executed as in the second embodiment.

In the third determination process SA114, the processing device 10 of the camera 1C determines whether or not the change amount ΔS of the edge size in the captured image is smaller than the third threshold value th3. When the change amount ΔS of the edge size is determined by the third determination process SA114 as the third threshold value th3 or more, the processing apparatus 10 executes the first adjustment process SA120. On the other hand, when the third determination process SA114 determines that the change amount ΔS of the edge size is smaller than the third threshold value th3, the processing device 10 executes the second adjustment process SA130. In addition, when it is determined by the third determination process SA114 that the change amount ΔS of the edge size is smaller than the third threshold value th3, and when the value S is larger than zero and smaller than the second threshold value th2, the second determination process SA112 The amount of reduction in brightness when generating a corrected image may be different depending on the case determined by. For example, the reduction amount in the former case may be smaller than the reduction amount in the latter case.
The above is the flow of the color adjustment method in this embodiment.

In the color adjustment method of the second embodiment, when a subject suddenly inserted into a background having a pattern is imaged, the edge size is increased by the amount of the pattern and the first adjustment process SA120 is executed, which causes a problem. There is. For example, it is assumed that an image G07 is obtained by photographing a state in which a white business card-sized paper OB2 is inserted on a desk TBL3 having a rectangular pattern on a reddish brown background as shown in FIG. 12 under a fluorescent lamp. .. In FIG. 12, the hatching of the horizontal line represents reddish brown. In this case, in the color adjustment method of the second embodiment, the edge size is increased by the amount of the pattern, and when the edge size is equal to or larger than the second threshold value, the first adjustment process SA120 is executed. As a result, as shown in the image G08 of FIG. 12, a color-adjusted image having a bluish tint as a whole is generated. In FIG. 12, the hatching of the grid line represents a bluish reddish brown color, and the hatching of the diagonal line downward to the right shoulder represents blue color.

If the third threshold value th3 in the present embodiment is set to a value larger than the business card size, for example, A4 size, if the subject suddenly inserted into the background having a pattern is business card size paper, the subject is inserted. The amount of change ΔS in the edge size due to the above is smaller than the third threshold value th3, and the second adjustment process SA130 is executed. As a result, it is possible to avoid generating a bluish color-adjusted image as in the image G08 of FIG. 12.

Further, it is assumed that an image G09 is obtained by photographing a state in which A4 size white paper OB1 is inserted on a desk TBL4 having a rectangular pattern on a gray ground color as shown in FIG. 13 under a red light. In FIG. 13, the hatching of the vertical line represents reddish gray, and the hatching of the diagonal line rising to the right represents red. In this case, the amount of change ΔS in the edge size due to the insertion of the subject becomes the third threshold value th3 or more, and the first adjustment process SA120 is executed. As a result, as in the image G10 of FIG. 13, a color-adjusted image is generated in which the ground color portion of the desk TBL4 is gray and the color of the paper OB1 as the subject is white.

Similarly to the camera 1A of the first embodiment and the camera 1B of the second embodiment, the camera 1C of the present embodiment can correctly perform white balance control regardless of the size of the subject and the color of the background. Become. In addition, according to the camera 1C of the present embodiment, it is possible to avoid erroneous determination when the light source actually changes, as in the case of the camera 1B of the second embodiment. Further, according to the camera 1C of the present embodiment, it is possible to avoid erroneous determination due to a pattern on the background of the subject.

4. Modifications Each of the above embodiments may be modified as follows.
(1) The adjustment amount of the brightness of the captured image in each of the above embodiments is an exposure compensation value indicating the correction amount when the exposure of the captured image is corrected by image processing, but the aperture or shutter speed of the image pickup device 20 It may be a value indicating. Further, in the first adjustment processing SA120 of each of the above embodiments, the processing apparatus 10 specifies the type of the light source based on the average value of the color temperature in the image represented by the corrected image data. However, the type of light source may be specified based on other statistics such as the median or most frequent color temperature in the image represented by the corrected image data. In short, any aspect may be used as long as the type of the light source is specified based on the color temperature in the image represented by the corrected image data. The same applies to the specification of the type of the light source in the second adjustment process SA130.

(2) In the first determination processing SA110 in each of the above embodiments, the processing device 10 determines whether or not the change amount ΔL of the adjustment amount of the brightness of the captured image captured by the imaging device 20 exceeds the first threshold value th1. Although the determination has been made, it may be determined whether or not the amount of change in the color adjustment amount of the captured image captured by the image pickup apparatus 20 exceeds the first threshold value. As a specific example of the color adjustment amount of the captured image captured by the image pickup apparatus 20, a filter coefficient defining a color adjustment filter can be mentioned.

(3) In the first embodiment, the program PA for causing the processing device 10 to execute the color adjustment method of the present disclosure is stored in the storage device 50 in advance. However, the program PA may be provided alone. As a specific mode of providing the program PA, the program PA is written on a computer-readable recording medium such as a flash ROM (Read Only Memory), and the recording medium on which the program PA is written is distributed, or electricity such as the Internet. An embodiment in which the program PA is distributed by downloading via a communication line can be mentioned. By operating the processor of an existing digital camera according to the program PA provided by these aspects, it becomes possible to cause the digital camera to perform the color adjustment method of the present disclosure. Similar to the program PA, the program PB in the second embodiment and the program PC in the third embodiment may be written and distributed on a computer-readable recording medium such as a flash ROM, and may be distributed by writing to a telecommunication line such as the Internet. It may be distributed by downloading via.

(4) The processing device 10 in the first embodiment is a processor, that is, a computer that executes the first determination processing SA110, the first adjustment processing SA120, and the second adjustment processing SA130 according to the program PA. However, each of the following first determination circuit, first adjustment circuit, and second adjustment circuit may be configured by an ASIC (Application Specific Integrated Circuit) or the like, and these circuits may be combined to form the processing device 10. .. The first determination circuit executes the first determination process SA110. The first adjustment circuit executes the first adjustment process SA120. The second adjustment circuit executes the second adjustment process SA130. Similarly, for the processing device 10 in the second embodiment, a first determination circuit composed of an ASIC or the like, a second determination circuit for executing the second determination process SA112, a first adjustment circuit, and a second adjustment circuit are provided. It may be configured in combination. Similarly, for the processing device 10 in the third embodiment, the first determination circuit, the second determination circuit, the third determination circuit for executing the third determination process SA114, the first adjustment circuit, and the first adjustment circuit, respectively, which are composed of ASICs and the like. The two adjustment circuits may be combined and configured.

(5) In each of the above embodiments, an example of application of the present disclosure to a digital camera has been described, but the present invention includes an image pickup device such as a smartphone equipped with an image pickup device, a tablet terminal equipped with an image pickup device, and a projector equipped with an image pickup device. The present disclosure may be applied to the color adjustment of the captured image in the electronic device.

5. Aspects Ascertained from At least One of the Embodiments and Each Modification The present disclosure is not limited to the above-described embodiments and modifications, and can be realized in various embodiments without departing from the spirit thereof. For example, the present disclosure can also be realized by the following aspects. The technical features in the above embodiments that correspond to the technical features in each of the embodiments described below are for solving some or all of the problems of the present disclosure, or some or all of the effects of the present disclosure. It is possible to replace or combine as appropriate to achieve this. Further, if the technical feature is not described as essential in the present specification, it can be appropriately deleted.

One aspect of the color adjustment method of the present disclosure includes a first determination process, a first adjustment process, and a second adjustment process. In the first determination process, it is determined whether or not the amount of change in the amount of adjustment of the brightness of the image captured by the image pickup device exceeds the first threshold value. The first adjustment process is executed when the result of the determination is negative, and the second adjustment process is executed when the result of the determination is affirmative. In the first adjustment process, the color of the captured image is adjusted based on the color temperature of the captured image. In the second adjustment process, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness. According to this aspect, it is possible to correctly perform white balance control regardless of the size of the subject and the color of the background. As a specific example of the adjustment amount of the brightness of the captured image, an exposure compensation value indicating the correction amount of the brightness of the captured image can be mentioned.

Further, another aspect of the color adjustment method of the present disclosure includes a first determination process, a second determination process, a first adjustment process, and a second adjustment process. In the first adjustment process, it is determined whether or not the amount of change in the adjustment amount of the brightness of the captured image captured by the image pickup device exceeds the first threshold value. The second determination process is executed when it is determined that the amount of change in the adjustment amount exceeds the first threshold value. In the second determination process, it is further determined whether or not the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and smaller than the second threshold value. In the first adjustment process, when the value is determined to be zero by the second determination process, when it is determined by the second determination process that the value is larger than the second threshold value, or the adjustment amount is determined. Is executed when the change amount of is determined by the first determination process to be equal to or less than the first threshold value. In the first adjustment process, the color of the captured image is adjusted based on the color temperature of the captured image. The second adjustment process is executed when it is determined by the second determination process that the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and smaller than the second threshold value. To. In the second adjustment process, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by applying the process of reducing the brightness to the captured image. Also in this embodiment, it is possible to correctly perform white balance control regardless of the size of the subject and the color of the background. In addition, according to this aspect, it is possible to avoid erroneous determination when the light source actually changes.

Further, another aspect of the color adjustment method of the present disclosure includes a first determination process, a second determination process, a third determination process, a first adjustment process, and a second adjustment process. In the first determination process, it is determined whether or not the amount of change in the amount of adjustment of the brightness of the image captured by the image pickup device exceeds the first threshold value. The second determination process is executed when it is determined by the first determination process that the amount of change in the adjustment amount exceeds the first threshold value. In the second determination process, it is determined whether or not the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and smaller than the second threshold value. The third determination process is executed when it is determined by the second determination process that the value is larger than the second threshold value. In the third determination process, it is determined whether or not the amount of change in the value is smaller than the third threshold value. In the first adjustment process, the change amount of the adjustment amount is determined to be equal to or less than the first threshold value by the first determination process, the value is determined to be zero by the second determination process, or the first adjustment process is performed. This is executed when it is determined by the third determination process that the amount of change in the value is equal to or greater than the third threshold value. In the first adjustment process, the color of the captured image is adjusted based on the color temperature in the image pickup device. In the second adjustment process, the third determination process determines that the amount of change in the value is smaller than the third threshold value, or the value is larger than zero and smaller than the second threshold value. It is executed when it is determined by the second determination process. In the second adjustment process, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness. Also in this embodiment, it is possible to correctly perform white balance control regardless of the size of the subject and the color of the background. In addition, according to this aspect, it is possible to avoid erroneous determination when the light source actually changes, and further, it is possible to avoid erroneous determination due to a pattern in the background of the subject. Become.

In addition, another aspect of the color adjustment method of the present disclosure includes a first determination process, a first adjustment process, and a second adjustment process. In the first determination process, it is determined whether or not the amount of change in the color adjustment amount of the image captured by the image pickup device exceeds the first threshold value. If the result of the determination is negative, the first adjustment process is executed, and if the result of the determination is affirmative, the second adjustment process is executed. In the first adjustment process, the color of the captured image is adjusted based on the color temperature of the captured image. In the second adjustment process, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness. Also in this embodiment, it is possible to correctly perform white balance control regardless of the size of the subject and the color of the background.

1A, 1B, 1C ... camera, 10 ... processing device, 20 ... image pickup device, 30 ... image processing device, 40 ... output device, 50 ... storage device, PA, PB, PC ... program.

Claims (5)

It is determined whether or not the amount of change in the amount of adjustment of the brightness of the captured image captured by the image pickup device exceeds the first threshold value.
If the result of the determination is negative, the color of the captured image is adjusted based on the color temperature of the captured image.
If the result of the determination is affirmative, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness.
Color adjustment method. The adjustment amount is an exposure compensation value indicating an exposure correction amount of the captured image.
The color adjustment method according to claim 1. It is determined whether or not the amount of change in the amount of adjustment of the brightness of the captured image captured by the image pickup device exceeds the first threshold value.
When it is determined that the amount of change in the adjustment amount exceeds the first threshold value, the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and smaller than the second threshold value. Judge whether or not
When the value is determined to be zero, the value is determined to be equal to or higher than the second threshold value, or the amount of change in the adjustment amount is determined to be equal to or lower than the first threshold value, the color in the captured image is determined. Adjust the color of the captured image based on the temperature,
When it is determined that the value is larger than zero and smaller than the second threshold value, the captured image is based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness. Adjust the color,
Color adjustment method. It is determined whether or not the amount of change in the amount of adjustment of the brightness of the captured image captured by the image pickup device exceeds the first threshold value.
When it is determined that the amount of change in the adjustment amount exceeds the first threshold value, the value corresponding to the size of the region partitioned by the edges in the captured image is larger than zero and smaller than the second threshold value. Judge whether or not
When it is determined that the value is larger than the second threshold value, it is determined whether or not the amount of change in the value is smaller than the third threshold value.
When the change amount of the adjustment amount is determined to be equal to or less than the first threshold value, the value is determined to be zero, or the change amount of the value is determined to be equal to or higher than the third threshold value, the image pickup is performed. The color of the captured image is adjusted based on the color temperature in the device.
When it is determined that the amount of change in the value is smaller than the third threshold value, or when it is determined that the value is larger than zero and smaller than the second threshold value, the captured image is given brightness. The color of the captured image is adjusted based on the color temperature in the corrected image obtained by performing the reducing process.
Color adjustment method. It is determined whether or not the amount of change in the color adjustment amount of the captured image captured by the image pickup device exceeds the first threshold value.
If the result of the determination is negative, the color of the captured image is adjusted based on the color temperature of the captured image.
If the result of the determination is affirmative, the color of the captured image is adjusted based on the color temperature in the corrected image obtained by subjecting the captured image to a process of reducing the brightness.
Color adjustment method.

Images