JP2018207283A - Image processing device - Google Patents

Abstract

To provide an image processing circuit which faithfully reproduces an actual hue underwater.SOLUTION: A CPU in an image processing device serves as a determination part which determines a water depth and the state of water quality at a position where a subject is imaged on the basis of color distribution information of an image of the subject taken underwater by an imaging part. Further, the CPU serves also as a white balance correction coefficient adjustment part which determines a white balance correction coefficient for correcting the white balance of the image, in accordance with the determined water depth and the determined state of water quality. A white balance adjustment circuit 22b executes white balance control on the image by using the white balance correction coefficient.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus that performs white balance adjustment on an image.

  An image processing apparatus such as a digital camera performs white balance adjustment processing on a captured image. The white balance adjustment processing is processing for adjusting the color balance of an image so that white is accurately projected white even under light sources having various color temperatures. Specifically, the white balance of the image is adjusted by multiplying the R component and the B component by a gain so that the levels of the R component, G component, and B component signals in the image have a predetermined relationship.

  Further, when a subject is imaged underwater, the image is bluish according to the water depth. This is because water molecules absorb R component light that has a longer wavelength than the G and B components. On the other hand, Patent Document 1 is disclosed as a technique for white balance adjustment in an image captured underwater. In Patent Document 1, white balance adjustment is performed on an image on an axis different from the black body radiation axis in accordance with the water depth at the position of the imaged subject.

Japanese Patent No. 5679113

  However, phytoplankton exists in the water. Since phytoplankton has the property of absorbing the light of the B component, the image becomes more green as the amount of phytoplankton increases.

  Then, in the said patent document 1, even if the bluishness of an image is improved by the white balance adjustment according to the water depth, the color of the said image will still be influenced by the underwater phytoplankton. Therefore, it is difficult to say that the color can be faithfully reproduced in Patent Document 1.

  The present invention has been made in view of the above, and an object thereof is to perform white balance adjustment capable of faithfully reproducing the color of an image captured in water.

  According to a first aspect of the present invention, there is provided an imaging unit that images a subject in water, a water depth at a position where the subject is captured based on color distribution information of an image of the subject captured by the imaging unit, and a water quality at the position. A white balance correction coefficient adjustment for determining a white balance correction coefficient for correcting the white balance of the image according to the water depth and the water quality state determined by the determination unit. And a white balance control unit that performs white balance control on the image using the white balance correction coefficient.

  Examples of the water quality include phytoplankton content in water. Here, white balance control is performed on an image captured underwater in accordance with the water depth at the position where the subject is captured and the state of water quality at the position. As a result, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, in which the influence of the water quality as well as the water depth is reduced.

  According to a second invention, in the first invention, a first memory for storing white balance correction coefficient information in which the white balance correction coefficient is set in advance corresponding to a combination pattern of the water depth and the water quality state. The white balance control unit corrects the white balance of the image using the white balance correction coefficient information.

  Thereby, since the white balance correction coefficient is quickly determined according to the actual water depth and the actual water quality state, the white balance control process is performed relatively quickly. In addition, it is possible to finely adjust the white balance correction coefficient according to the actual water depth and the actual water quality.

  According to a third invention, in the first invention or the second invention, at least one of lightness, saturation, and hue of the image is corrected according to the water depth and the water quality determined by the determination unit. An image processing apparatus further comprising a color attribute adjustment unit.

  Thus, tuning is performed so that at least one of the brightness, saturation, and hue of the image is appropriate according to the actual water depth and the actual water quality. Accordingly, it is possible to obtain an image in which the actual color in water is reproduced more faithfully.

  In a fourth aspect based on the third aspect, a color attribute coefficient for correcting at least one of lightness, saturation, and hue of the image corresponding to a combination pattern of the water depth and the water quality state is provided in advance. A second storage unit that stores set color attribute coefficient information; and the color attribute adjustment unit adjusts the color attribute coefficient using the color attribute coefficient information. It is a processing device.

  As a result, the color attribute coefficient is quickly determined according to the actual water depth and the actual water quality state, so that at least one of the saturation, lightness, and saturation adjustment processing of the image is performed relatively quickly. . In addition, the color attribute coefficient can be finely adjusted according to the actual water depth and the actual water quality.

  According to the present invention, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, in which the influence of not only the water depth but also the water quality is reduced.

FIG. 1 is a block diagram schematically showing the configuration of the image processing apparatus. FIG. 2 is a block diagram schematically showing the configuration of the signal processing circuit. FIG. 3 is a conceptual diagram of the weighting coefficients set in the weighting table. FIG. 4 is a diagram illustrating a color distribution state. FIG. 5 is a diagram illustrating an example of an external view of a subject and (b) an example of a screen when a captured subject is displayed. FIG. 6 is a diagram for explaining the concept of operation during white balance adjustment. FIG. 7 is a conceptual diagram of the reference value table. FIG. 8 is a conceptual diagram of the target value table. FIG. 9 is a diagram illustrating a distribution state of the reference value and the target value. FIG. 10 is a diagram illustrating a part of the reference value and the target value. FIG. 11 is a diagram showing a flow of operation of the region discriminating circuit. FIG. 12 is a diagram for explaining a part of the operation for adjusting the hue. FIG. 13 is a diagram for explaining a part of the operation for adjusting the saturation. FIG. 14 is a diagram for explaining a part of the operation for adjusting the brightness. FIG. 15 is a diagram showing a main flow of the CPU. FIG. 16 is a diagram showing the flow of the color tone correction process in FIG. FIG. 17 is a diagram showing the concept of white balance adjustment and hue adjustment when FIG. 16 is executed. FIG. 18 is a chromaticity diagram of the R component, G component, and B component in water. FIG. 19 is a diagram for explaining the concept of the operation at the time of white balance adjustment in the underwater imaging mode. FIG. 20 is a diagram illustrating a flow of color adjustment operation in the underwater imaging mode. FIG. 21 is a conceptual diagram of a water depth-water quality table. FIG. 22 is a conceptual diagram of a parameter table.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments are essentially preferable examples, and are not intended to limit the scope of the present invention, its application, or its use.

<Embodiment>
<Overview>
The image processing apparatus (10) according to the present embodiment is a digital camera that can capture an image not only on the ground but also underwater. That is, in the image processing apparatus (10), a mode for imaging on the ground and a mode for imaging underwater can be selected. Regardless of which mode is selected, the image processing apparatus (10) performs color adjustment including white balance adjustment on the captured image, and outputs the adjusted image to a monitor (30), which will be described later.

  By the way, when a general image processing apparatus performs white balance adjustment on an image captured underwater, the color of the image after white balance adjustment and the color of the actual subject may deviate. . The factors include the distance from the water surface to the subject (that is, the water depth) and the water quality near the subject. This is because the depth of water and the degree of water quality vary the light absorption rates of the R component, the G component, and the B component.

  The water quality includes the phytoplankton content in water.

  In contrast, when the subject is imaged underwater, the image processing apparatus (10) according to the present embodiment estimates the water depth and the water quality state (phytoplankton content) at the position of the imaged subject. By performing color adjustment including white balance adjustment using the result obtained, the actual color of the subject can be reproduced. In particular, the image processing apparatus (10) according to the present embodiment greatly exerts the effect when imaging is performed without using a strobe.

  In the following, after describing the configuration of the image processing apparatus (10) and the basic color adjustment processing in advance, the color adjustment processing in the underwater imaging mode according to the present embodiment is described as <color adjustment in the underwater imaging mode. Processing> will be described in detail.

<Configuration>
As shown in FIG. 1, the image processing apparatus (10) includes an imaging unit (11), a signal processing circuit (22), a memory control circuit (24), an SDRAM (26), a video encoder (28), and a monitor (30). , An I / F circuit (34), a memory card (36), and a CPU (38). The imaging unit (11) includes a focus lens (12), an image sensor (14), a TG (timing generator) (16), a CDS / AGC circuit (18), and an A / D converter (20). Each of the focus lens (12) and the image sensor (14) is driven by a driver (not shown).

  The optical image of the subject is incident on the light receiving surface of the image sensor (14) via the focus lens (12). On the light receiving surface, photoelectric conversion is performed on the incident optical image, thereby generating a camera signal (raw image signal) corresponding to the optical image. The light-receiving surface is covered with a primary color Bayer array color filter (not shown), and each pixel signal forming the camera signal is one of R (Red), G (Green), and B (Blue). It has only information component.

  When the power is turned on, a processing command is given from the CPU (38) to each of the TG (16), the signal processing circuit (22), and the video encoder (28). Thereby, the through image processing is started.

  The TG (16) repeatedly performs exposure of the image sensor (14) and reading of the camera signal. The camera signal of each frame read from the image sensor (14) is subjected to noise removal and level adjustment in the CDS / AGC circuit (18), and then converted into a digital signal in the A / D converter (20). .

  The signal processing circuit (22) performs color separation processing, color adjustment processing, YUV conversion processing, and the like on the camera data of each frame output from the A / D converter (20) to generate image data. This image data includes Y data representing a luminance component, U data representing a color difference component, and V data. The image data generated by the signal processing circuit (22) is output to the memory control circuit (24) and stored in the SDRAM (26) by the memory control circuit (24).

  In the color adjustment process, the white balance is adjusted for the camera data of each frame, and then the brightness, saturation, and hue are corrected. Hereinafter, an adjustment coefficient used in white balance adjustment is referred to as “gain x1, y1”, and correction coefficients used in lightness, saturation, and color difference correction are sequentially set to “target L component value, target C component value”. , Referred to as “target H component value”. These adjustment coefficients and correction coefficients are determined by the CPU (38).

  When the video encoder (28) causes the memory control circuit (24) to read the image data of each frame stored in the SDRAM (26), the video encoder (28) encodes the image data into a composite image signal in the NTSC format. The video encoder (28) outputs the composite image signal to the monitor (30). As a result, a real-time moving image (through image) of the subject is displayed on the monitor (30).

  When the photographer fully presses the shutter button (40), the image processing device (10) executes imaging and recording processing. First, the CPU (38) outputs a compression command to the JPEG codec (32). When the JPEG codec (32) reads out the image data stored in the SDRAM (26) via the memory control circuit (24), the JPEG codec (32) compresses the image data by the JPEG method. The compressed image data is stored again in the SDRAM (26).

  The I / F circuit (34) reads the image data stored in the SDRAM (26) via the memory control circuit (24), and records the read image data in the memory card (36). As a result, an image file is created in the memory card (36).

  The memory card (36) can be formed of a non-volatile recording medium that is detachable from the housing (not shown) of the image processing apparatus (10).

<Configuration of signal processing circuit>
Here, the configuration of the signal processing circuit (22) will be described in detail.

  As shown in FIG. 2, the signal processing circuit (22) includes a color separation circuit (22a), a white balance adjustment circuit (22b) corresponding to a white balance control unit, an integration circuit (22c), an LCH conversion circuit (22d), An L correction circuit (22e), a C correction circuit (22f), an H correction circuit (22g), a YUV conversion circuit (22h), an area discrimination circuit (22i), and a plurality of tables (22m, 22n,...) Are stored. A memory (22j) (corresponding to a first storage unit and a second storage unit) is included.

-Color separation circuit-
The color separation circuit (22a) performs color separation processing on the camera data output from the A / D converter (20). Since each pixel data constituting the camera data has only one of the R component, the G component, and the B component, the color separation circuit (22a) includes the other two color components that each pixel does not have. Complement. Thereby, RGB format image data in which each pixel has all the color information of R, G, and B is generated.

-White balance adjustment circuit-
The white balance adjustment circuit (22b) has two amplifiers (221b, 222b). The amplifier (221b) is given a gain x1, and the amplifier (222b) is given a gain y1. Of the generated image data, the R component is input to the amplifier (221b), and is amplified by the gain x1 by the amplifier (221b). Of the generated image data, the B component is input to the amplifier (222b) and amplified by the gain y1 by the amplifier (222b).

  The R component and B component amplified by the white balance adjustment circuit (22b) and the G component of the unamplified image data are input to the integrating circuit (22c) and the LCH conversion circuit (22d).

  The screen is divided into, for example, 16 in the horizontal direction and the vertical direction. In this case, 256 divided areas are formed on the screen. The integrating circuit (22c) integrates the R component, G component, and B component for each divided area and for each same color component. For this reason, 256 integrated values Ir (i), 256 integrated values Ig (i), and 256 integrated values Ib (i) are obtained for each frame period. Here, “i” is an integer from 1 to 256. That is, 256 color evaluation values individually corresponding to 256 divided areas are obtained for each frame period.

  Based on the color evaluation value, the CPU (38) adjusts the gains x1 and y1 used for white balance adjustment to calculate optimum gains xs and ys.

  Specifically, the CPU (38) converts the integrated value Ir (i) into an integrated value Iry (i), and converts the integrated value Ib (i) into an integrated value Iby (i). The integrated value Iry (i) and the integrated Iby (i) are equal to the values obtained by integrating the color difference components RY and BY for each divided area, and can be said to be the color evaluation values of each divided area.

  Subsequently, the CPU (38), based on the weighting table (38a) having the numerical distribution according to FIG. 3 and the integrated values Iry (i) and Iby (i) of each divided area, the weighting coefficient k ( i). The weighting table (38a) is stored in a memory (not shown). In FIG. 3, each weighting coefficient k (i) is stored as “0” or “1”. The weighting coefficient k (i) in FIG. 3 corresponds to the color distribution state shown in FIG.

  In the weighting table (38a) in FIG. 3, each of the BY axis as the horizontal axis and the RY axis as the vertical axis is divided into 17 equal parts and separated by the BY axis and the RY axis. An example in which the quadrant is divided into 64 parts is shown. As a result, 290 areas are formed in FIG. 3, and “0” or “1” is assigned to the weighting coefficient K (i) for each area. Of the 290 areas shown in FIG. 3, the range to which the weighting coefficient k (i) = 1 is assigned (the range surrounded by the solid line in FIG. 3) is the pull-in range.

  After obtaining the weighting coefficient k (i) for each divided area, the CPU (38) uses the weighting coefficient k (i) for each of the following expressions (1) and (2) to obtain the optimum gains xs, ys. Is calculated.

  As described above, since the weighting coefficient k (i) is “0” or “1”, only the divided area where the color evaluation value is within the pull-in range is valid according to the above equation (1) and the above equation (2). Become. Therefore, the sum total of the integrated values Ir (i), Ig (i), and Ib (i) of the effective divided area is obtained by the above formulas (1) and (2). From the above equation (1), the optimum gain xs based on the sum of the integrated values Ir (i) and Ig (i) is obtained. From the above equation (2), the integrated values Ig (i) and Ib (i ) An optimum gain ys based on each sum is obtained. The optimum gains xs and ys obtained in this way are values such that the average value of the color evaluation values included in the pull-in range converges at the intersection of the RY axis and the BY axis.

  As shown in FIG. 5 (a), when the cylindrical rod (44) placed on the paper (42) is imaged from directly above, the monitor (30) is shown in FIG. 5 (b). A subject image in such a state is displayed. Note that the vertical and horizontal lines shown in FIG. 5B are lines for dividing the screen into 256 divided areas and are not actually displayed. If the paper (42) is light orange and the stick (44) is dark yellow, the integrated values Iry (i) and Iby (i) obtained by integrating only the color of the paper (42) are included in the pull-in range. The integrated values Iry (i) and Iby (i) obtained by integrating only the color of the bar (44) are out of the pull-in range. As a result, the optimum gains xs and ys are determined based on the integrated values Ir (i), Ig (i), and Ib (i) related to the color of the paper (42).

  When the integrated values Iry (i) and Iby (i) relating to the color of the paper (42) before the optimum gains xs and ys are calculated are distributed in the area 1 shown in FIG. 6, the optimum gains xs and ys are calculated and The integrated values Iry (i) and Iby (i) relating to the color of the paper (42) after being set in the amplifiers (221b, 222b) move to the area 2 shown in FIG. That is, the integrated values Iry (i) and Iby (i) relating to the color of the paper (42) move only by the movement amount W1 toward the intersection of the RY axis and the BY axis.

-LCH conversion circuit-
The LCH conversion circuit (22d) converts the R component and B component of each pixel amplified by the white balance adjustment circuit (22b) and the G component of each pixel output from the color separation circuit (22a) into lightness components. To an L component, a C component as a saturation component, and an H component as a hue component. The LCH conversion circuit (22d) outputs the converted LCH data (data having an L component, a C component, and an H component of each pixel). The L component data in the LCH data is input to the L correction circuit (22e), and the C component data is input to the C correction circuit (22f). The H component data is input to the H correction circuit (22g) and the region discrimination circuit (22i).

-Each correction circuit, YUV conversion circuit, and area discrimination circuit-
The L correction circuit (22e) obtains a correction L component based on the L component of the input LCH data. The C correction circuit (22f) obtains a correction C component based on the C component of the input LCH data. The H correction circuit (22g) obtains a correction H component based on the H component of the input LCH data. The correction L component, the correction C component, and the correction H component are input to the YUV conversion circuit (22h).

  The YUV conversion circuit (22h) converts the LCH format image data into YUV format image data based on the input corrected L component, corrected C component, and corrected H component. As a result, the YUV conversion circuit (22h) outputs image data having a Y component that is luminance data, a U component that is a color difference component, and a V component.

  The area determination circuit (22i) refers to the reference value table (22m) stored in the memory (22j) and determines the area to which the H component belongs for each pixel. The area discrimination circuit (22i) further reads a reference value corresponding to the discrimination result from the reference value table (22m), and reads a target value corresponding to the discrimination result from the target value table (22n).

  FIG. 7 is a conceptual diagram of the reference value table (22m). As shown in FIG. 7, in the reference value table (22m), one record includes a combination of a reference H component value indicating hue, a reference C component value indicating saturation, and a reference L component value indicating lightness. A total of six patterns are represented. “N” in FIG. 7 represents a record number, and the reference H component value, the reference C component value, and the reference L component value represented by the same record number define one reference value. The reference values of the six patterns shown in FIG. 7 correspond to each of the six representative colors of Mg (magenta), R (red), Ye (yellow), G (green), Cy (cyan), and B (blue). Then, as shown in FIGS. 9 and 10, it is distributed in the YUV space. FIG. 10 shows reference values corresponding to only Cy.

  FIG. 8 is a conceptual diagram of the target value table (22n). As shown in FIG. 8, in the target value table (22n), one record is a combination of a target H component value indicating hue, a target C component value indicating saturation, and a target L component value indicating brightness. A total of six patterns are represented. “N” in FIG. 8 represents a record number, and the target H component value, the target C component value, and the target L component value represented by the same record number define one target value. The target values of the six patterns shown in FIG. 8 correspond to each of the six representative colors of Mg (magenta), R (red), Ye (yellow), G (green), Cy (cyan), and B (blue). Then, as shown in FIGS. 9 and 10, it is distributed in the YUV space. FIG. 10 shows target values corresponding to only Cy.

  The reference H component value, the reference C component value, and the reference L component value set in the reference value table (22m) are values set in advance at the manufacturing stage of the image processing apparatus (10) and cannot be changed. Value. The target H component value, the target C component value, and the target L component value set in the target value table (22n) are set to default values at the manufacturing stage of the image processing apparatus (10). ) Is a value that is appropriately changed.

  The region discriminating circuit (22i) executes processing according to FIG. 11 using the H correction component value output from the LCH conversion circuit (22d) and the two tables (22m, 22n).

  First, the area discriminating circuit (22i) sets the variable record number N to “1” (step St1), and reads the reference H component value corresponding to the record number N “1” from the reference value table (22m). (Step St2). The region discriminating circuit (22i) compares the read reference H component value with the H component value of the target pixel (hereinafter, the current pixel H component value) (step St3).

  When the reference H component value is larger than the current pixel H component value (YES in step St3), the area determination circuit (22i) determines whether or not the record number N is “1” (step St6). If the record number N is not “1” (NO in step St6), the processes in steps St7 to St10 are performed. When the reference H component value is equal to or smaller than the current pixel H component value (NO in step St3), the area determination circuit (22i) increments the record number N (step St4), and the incremented record number N is It is determined whether or not it is larger than “6” (step St5). When the incremented record number N is equal to or smaller than “6” (NO in Step St5), the process of Step St2 is repeated. When the incremented record number N is larger than “6” (YES in step St5), or when the record number N is “1” in step St6 (YES in step St6), the processing in steps St11 to St14 Is done.

  In step St7, the region discriminating circuit (22i) sets the reference H component value, the reference C component value and the reference L component value corresponding to the current record number N as “Hrα”, “Crα” and “Lrα” as a reference value table ( Select from 22m). In step St8, the area discriminating circuit (22i) sets the target H component value, target C component value and target L component value corresponding to the current record number N as “Htα”, “Ctα” and “Ltα” as target value tables ( Select from 22n).

  In step St9, the area discriminating circuit (22i) calculates the reference H component value, reference C component value and reference L component value corresponding to the number before the current record number N (ie, “N−1”) as “Hrβ. “Crβ” and “Lrβ” are selected from the reference value table (22m). In step St10, the area discriminating circuit (22i) obtains the target H component value, target C component value and target L component value corresponding to the number before the current record number N (ie, “N−1”) as “Htβ. “Ctβ” and “Ltβ” are selected from the target value table (22n).

  On the other hand, in step St11, the region discriminating circuit (22i) sets the reference H component value “Hr1”, the reference C component value “Cr1” and the reference L component value “Lr1” corresponding to the record number N to “1” to “ Hrα ”“ Crα ”“ Lrα ”is selected from the reference value table (22 m). In step St12, the region discriminating circuit (22i) sets the target H component value “Ht1”, the target C component value “Ct1” and the target L component value “Lt1” corresponding to the record number N to “1” to “Htα”. “Ctα” and “Ltα” are selected from the target value table (22n).

  In step St13, the region discriminating circuit (22i) sets the reference H component value “Hr6”, the reference C component value “Cr6” and the reference L component value “Lr6” corresponding to the record number N to “6” to “Hrβ”. “Crβ” and “Lrβ” are selected from the reference value table (22 m). In step St14, the region discriminating circuit (22i) sets the target H component value “Ht6”, the target C component value “Ct6” and the target L component value “Lt6” corresponding to the record number N to “6” to “Htβ”. “Ctβ” and “Ltβ” are selected from the target value table (22n).

  In this way, two reference values each having two reference H component values sandwiching the H component value of the target pixel and two target values corresponding to the two reference values are detected.

  The reference H component values Hrα and Hrβ and the target H component values Htα and Htβ are input to the H correction circuit (22g). The reference C component values Crα and Crβ and the target C component values Ctα and Ctβ are input to the C correction circuit (22f). The reference L component values Lrα and Lrβ and the target L component values Ltα and Ltβ are input to the L correction circuit (22e).

  The H correction circuit (22g) calculates the current pixel H component value Hin input from the LCH conversion circuit (22d), the reference H component values Hrα, Hrβ, and the target H component values Htα, Htβ by the following equation (3): By substituting in (5), the current pixel H component value Hin is converted into a corrected H component value Hout. The corrected H component value Hout has a magnitude indicated by a broken line in FIG.

  The H correction circuit (22g) also outputs the angle data θ1 and θ2 to the C correction circuit (22f) and the L correction circuit (22e), and the angle data θ3 (= | Htβ−Hout |) and angle data θ4 ( = | Htα−Hout |) is output to the L correction circuit (22e).

  The C correction circuit (22f) calculates the current pixel C component value Cin input from the LCH conversion circuit (22d), the reference C component values Crα and Crβ, and the target C component values Ctα and Ctβ by the following equation (6). By substituting into, the current pixel C component value Cin is converted into a corrected C component value Cout. The corrected C component value Hout has a magnitude indicated by a broken line in FIG.

  The C correction circuit (22f) further, based on the following equations (7) and (8), connects the straight line connecting the coordinates (0, 0) and (Cin, Hin) of the CH system to the coordinates (Crα, Hrα), (Crβ , Hrβ) and the C component value Crγ at the intersection coordinates with the straight line connecting the straight lines connecting the coordinates (0, 0) and (Cout, Hout) of the CH system and the coordinates (Ctα, Htα), (Ctβ, Htβ). The C component value Ctγ at the intersection coordinates with the straight line is calculated. The C correction circuit (22f) outputs the calculated C component values Crγ and Ctγ to the L correction circuit (22e) together with the current pixel C component value Cin and the correction C component value Cout described above.

  The L correction circuit (22e) converts the current pixel L component value Lin input from the LCH conversion circuit (22d) into a correction L component value Lout according to the following equations (9) to (15). The corrected L component value Lout is shown in FIG.

  Each of Lmax and Lmin shown in FIG. 14 is a maximum value and a minimum value of L (brightness) that can be reproduced. The current pixel (input pixel value) is a surface formed by LCH coordinates (Lmax, 0, 0), (Lmin, 0, 0), (Lrγ, Crγ, Hin) (YUV space cut out by hue Hin) Surface). On the other hand, the correction pixel value is a surface formed by LCH coordinates (Lmax, 0, 0), (Lmin, 0, 0), (Ltγ, Ctγ, Hout) (a surface obtained by cutting out the YUV space with the hue Hout). Exists on.

  The corrected pixel value is defined by the corrected H component value Hout, the corrected C component value Cout, and the corrected L component value Lout obtained as described above. The current pixel value is defined by the current pixel H component value Hin, the current pixel C component value Cin, and the current pixel L component value Lin output from the LCH conversion circuit (22d).

<Tone correction processing by CPU>
Next, basic color tone correction processing will be described with reference to FIGS. 15 and 16.

  As shown in FIG. 15, first, the CPU (38) sets the reference gains xγ and yγ as gains x1 and y1 (step St31). At this time, gains x1 and x1, that is, reference gains xγ and yγ are set in the amplifiers (221b and 222b) of the white balance adjustment circuit (22b). Note that the reference gains xγ and yγ are gains that allow white balance to be adjusted appropriately when a subject having a color temperature (reference temperature) of about 5100K is imaged. The so-called default values are determined at the manufacturing stage. It is.

  Next, the CPU (38) outputs a through image processing command to the TG (16), the signal processing circuit (22) and the video encoder (28) in order to start through image processing (step St32). Then, the CPU (38) performs color correction (step St33) until the shutter button (40) of the image processing apparatus (10) is pressed (NO in step St34), and the shutter button (40) is pressed. If so (YES in step St34), the imaging / recording process is performed (step St35). Thereby, the compressed image data of the subject image picked up in response to the operation of the shutter button (40) is recorded in the memory card (36) in the file format.

  The color tone correction is executed according to FIG. First, the CPU (38) determines whether or not a vertical synchronizing signal has been generated (step St41). When the vertical synchronization signal is generated (YES in step St41), the CPU (38) determines the integrated values Ir (i) and Ig (i) of the R component, G component, and B component calculated by the integration circuit (22c). , Ib (i) is fetched (step St42). Next, the CPU (38) calculates optimal gains xs and ys using the integrated values Ir (i), Ig (i) and Ib (i) (step St43), and calculates the calculated optimal gains xs and ys as gains. These are held as x2 and y2 (step St44). The CPU (38) calculates a difference Δx between the gain x2 and the gain x1 and a difference Δy between the gain y2 and the gain y1 (step St45).

  The calculated gain difference Δx, Δy is a change amount of the white balance of the subject image, and corresponds to a movement amount W1 from the area 1 to the area 2 indicated by a white circle in FIGS. Based on the gain differences Δx, Δy, the CPU (38) converts the target H component value, target C component value, and target L component value in the target value table (22n) into the H component (hue), C component ( Saturation) and L component (lightness) are changed in this order (step St46). The changed target H component value, target C component value, and target L component value move by a movement amount T1 to the position indicated by the white square in FIG.

  That is, as shown in FIG. 17, the moving direction from area 1 to area 2 in white balance and the moving direction of each target component value are opposite. The movement amount T1 of each target component value is an amount that cancels out the movement amount W1 in white balance (| W1 | = | T1 |).

  After completing the process in step St46, the CPU (38) holds the optimum gains x2 and y2 calculated in step St44 as gains x1 and x1 (step St47), and performs the processes after step St34 in FIG.

<Color adjustment processing in underwater imaging mode>
As described above, the image processing apparatus (10) according to the present embodiment can set the underwater imaging mode. When the imaging unit (11) captures a subject underwater without setting a strobe while the underwater imaging mode is set, the color of the actual subject as described below is more faithfully reproduced in the image. To be processed.

  Hereinafter, for convenience of explanation, the L correction circuit (22e), the C correction circuit (22f), the H correction circuit (22g), and the region determination circuit (22i) are collectively referred to as a color attribute adjustment circuit (22p).

  FIG. 18 is a chromaticity diagram for explaining how the R component, B component, and G component of an image captured in water are shifted. In FIG. 18, the distribution of each color of the reference R component, B component, and G component is represented by a solid line.

  As shown in FIG. 18, the color distribution of an image captured in water is affected by the water depth at the imaging location (subject position) in water and the water quality (specifically, the phytoplankton content in water). To change.

  Specifically, when the water depth is constant, the color distribution transitions generally upward in the vertical axis of FIG. 18 as the amount of phytoplankton increases, and conversely, as the amount of phytoplankton decreases, Transitions downward in the vertical axis. When the amount of phytoplankton is constant, the color distribution changes as a whole in the right direction of the horizontal axis in FIG. 18 as the water depth is shallower. Conversely, as the water depth increases, the color distribution increases in the overall left direction of the horizontal axis. Transition to. Then, when the amount of phytoplankton is relatively large, the deeper the water depth, the stronger the green color (dotted line in FIG. 18), and when the amount of plankton is relatively small, the deeper the water depth, the stronger the blueness (dashed line in FIG. 18). ). This is because the R component is easily absorbed by water and the B component is easily absorbed by phytoplankton in the light incident on the water from the ground.

  Therefore, when imaging is performed with the underwater imaging mode selected, the CPU (38) of the image processing apparatus (10) according to the present embodiment first functions as a determination unit, and the water depth at the position of the subject. And the state of water quality at that position. Next, the CPU (38) functions as a white balance correction coefficient adjustment unit, and corrects the gains x1 and y1 used for white balance adjustment of the image according to the determined water depth and water quality state. Determine the correction factor. In parallel with this, the CPU (38) uses the color attribute coefficient used for correcting each of the H component (hue), C component (saturation), and L component (lightness) according to the determined water depth and water quality. (Specifically, correction parameters for individually correcting the target H component value, the target C component value, and the target L component value described above) are determined.

  FIG. 19 is a diagram illustrating the concept of how white balance adjustment is performed when an image is taken underwater. Since the G component and the B component of the image captured underwater are strengthened by the influence of the water depth and the state of the water quality, the first image on the coordinate plane defined by the U axis and the V axis is used instead of the area 1 in FIGS. An image before white balance adjustment may be located in area 3 located in the third quadrant, area 4 in the fourth quadrant, or the like. As an example, the image is located in area 3 when the amount of phytoplankton is large, and is located in area 4 when the amount of phytoplankton is small. Furthermore, as an example, in either area 3 or area 4, the position of the image approaches the intersection of the V axis and the U axis when the water depth is shallow, but is far from the intersection when the water depth is deep.

  Regardless of the area where the image before white balance adjustment exists, the CPU (38) of this embodiment adjusts the gains x1 and y1 more finely by determining the white balance correction coefficient, and the signal processing circuit (22). Performs white balance adjustment using the gains x1 and y1, thereby moving the image to the area 2 that is the intersection of the U axis and the V axis. Further, the CPU (38) of the present embodiment appropriately determines the color attribute coefficient in accordance with the determination of the white balance correction coefficient, and the color attribute adjustment circuit (22p) uses the determined color attribute coefficient to Perform finer adjustment processing of the color tone, such as slightly enhancing redness.

  With these processes, as shown in FIG. 19, as the image moves from area 3 or area 4 to area 2 by white balance adjustment, Mg (magenta), R (red), Ye (yellow), G The target component value of each color of (green), Cy (cyan), and B (blue) moves in the direction opposite to the moving direction of white balance adjustment. Specifically, when the movement by white balance adjustment moves from area 3 to area 2, the movement direction of the target component value of each color is the direction from area 2 to area 3, and the movement amount is the white balance. Is the amount T1 ′ that offsets the movement amount W1 ′. When the movement by white balance adjustment moves from area 4 to area 2, the movement direction of the target component value of each color is the direction from area 2 to area 4, and the movement amount is the movement amount W1 in white balance. The amount is “T1 that cancels out”.

  According to the above processing, in the present embodiment, an actual color tone can be reproduced as in the case of shooting an image captured with a strobe with respect to an image captured without striking the strobe in water.

  Hereinafter, the flow of the control operation will be described in detail with reference to FIGS.

  First, it is assumed that the underwater imaging mode is set (YES in St61) and the image is taken without striking the strobe (YES in St62). The CPU (38) uses the R component integrated value Ir (i), the G component Ig (i), and the B component integrated value Ib (i) input from the above-described integration circuit (22c) to calculate the ratio coefficient. “R / G” and “B / G” are obtained (step St63). The CPU (38) applies the obtained ratio coefficients “R / G” and “B / G” to the water depth-water quality table (22q) according to FIG. 21 to determine the water depth and water quality at the position of the subject in the water ( The phytoplankton content in water) is calculated from the values in the table (22q) (step St64). That is, the CPU (38) determines the water depth at the position where the subject is imaged and the water quality state at the position based on the color distribution information of the image of the subject imaged by the imaging unit (11).

  Here, FIG. 21 is a conceptual diagram of the water depth-water quality table (22q) stored in the memory (22j). The water depth-water quality table (22q) takes water depth vertically and takes the phytoplankton content in water as the water quality state horizontally, and the ratio coefficient “R / G corresponding to each combination pattern of water depth and water quality state” “B / G” is shown in a matrix. “R / G” represents the ratio of the R component to the G component of the image, and “B / G” represents the ratio of the B component to the G component. The ratio coefficients “R / G” and “B / G” are defined in advance by theory, experiment, and the like on the condition of each combination pattern of water depth and water quality. The CPU (38) easily calculates data representing the water depth and the state of water quality by applying the obtained ratio coefficients “R / G” and “B / G” to the water depth-water quality table (22q) of FIG. be able to.

  Next, the CPU (38) applies the data representing the calculated water depth and the state of the water quality to the parameter table (22r) of FIG. 22 to configure the white balance adjustment circuit (22b) and the color attribute adjustment circuit (22q). Various parameters to be set in the various circuits (22e, 22f, 22g, 22i) to be selected are selected (step St65).

  Here, FIG. 22 is a conceptual diagram of the parameter table (22r) stored in the memory (22j). The parameter table (22r) includes a white balance correction coefficient for correcting the gains x1 and y1 and a target H component value correction for each combination pattern of the water depth and the state of water quality (the content of phytoplankton in water). A parameter, a correction parameter for the target C component value, and a correction parameter for the target L component value are set in advance. In other words, this parameter table (22r) is a fine white balance coefficient and color attribute coefficient for reproducing faithful color appearance, no matter how much the color is affected by water depth and water quality. Can be said to have been prepared in advance. For example, various parameters in the parameter table (22r) may be set such that the correction value increases as the water depth increases, or the correction value increases as the amount of phytoplankton increases. Can do.

  The two tables (22q, 22r) both correspond to white balance correction coefficient information and color attribute coefficient information.

  Next, the CPU (38) sets the various parameters selected in step St65 in the color attribute adjustment circuit (22q) and the white balance adjustment circuit (22b) (step St66). Specifically, the white balance correction coefficient for correcting the gains x1 and y1 is set in the white balance adjustment circuit (22b). The color attribute coefficient is set in the color attribute adjustment circuit (22p).

  The color attribute adjustment circuit (22q) and the white balance adjustment circuit (22b) for which various parameters are set in step St66 perform color adjustment on the target image based on the parameters (step St67). Specifically, the white balance adjustment circuit (22b) performs white balance adjustment on the image using the white balance correction coefficient and the gains x1 and y1 obtained from the two tables (22q and 22r). The color attribute adjustment circuit (22p) uses the color attribute coefficient, the target H component value, the target C component value, and the target L component value obtained from the two tables (22q, 22r) to obtain the brightness, saturation, and Correct the hue.

<Effect>
In this embodiment, when the water depth and water quality state at the position where the subject is imaged are determined from the color distribution information of the image captured underwater, the determination result is used to correct the white balance of the image. A white balance correction coefficient is determined, and white balance adjustment is performed on the image using the coefficient. As a result, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, in which the influence of the water quality as well as the water depth is reduced.

  In this embodiment, the white balance correction coefficient is set in advance corresponding to the combination pattern of the water depth and the water quality state. Therefore, the white balance correction coefficient is quickly determined according to the actual water depth and the actual water quality, and the process of white balance control is performed relatively quickly. In addition, it is possible to finely adjust the white balance correction coefficient according to the actual water depth and the actual water quality.

  In the present embodiment, the brightness, saturation, and hue of the image are tuned to be appropriate according to the determined water depth and water quality. Accordingly, it is possible to obtain an image in which the actual color in water is reproduced more faithfully.

  In the present embodiment, color attribute coefficients for correcting the brightness, saturation, and hue of an image are set in advance corresponding to the combination pattern of the water depth and the water quality state. Therefore, the color attribute coefficient is quickly determined according to the actual water depth and the actual water quality state, and the saturation, brightness, and saturation adjustment processing of the image is performed relatively quickly. In addition, the color attribute coefficient can be finely adjusted according to the actual water depth and the actual water quality.

<< Other Embodiments >>
In the above embodiment, it has been described that the white balance correction coefficient for correcting the gains x1 and y1 is determined according to the water depth and the water quality state. However, the gains x1 and y1 themselves are determined according to the water depth and the water quality state. It may be determined as a white balance correction coefficient.

  In the above embodiment, the target H component value correction parameter, the target C component value correction parameter, and the target L component value correction parameter are determined as the color attribute coefficients according to the water depth and the water quality state. The target H component value itself, the target C component value itself, and the target L component value itself may be determined as color attribute coefficients in accordance with the water depth and the water quality state.

  In the above embodiment, the case where the white balance correction coefficient information and the color attribute coefficient information are represented by the tables (22q, 22r) according to FIGS. 21 and 22 is illustrated, but is represented by the tables of FIGS. It is not limited to a thing. For example, the white balance correction coefficient information and the color attribute coefficient information may be obtained by calculation each time using an expression.

  In the above embodiment, the case where all the brightness, saturation, and hue of the image are corrected according to the state of water depth and water quality is exemplified. However, at least one of the brightness, saturation, and hue of the image may be corrected. .

  In FIG. 20 of the above-described embodiment, the case where imaging is performed without strobing the strobe in water is illustrated. However, the present invention is not limited to the case where the image is taken without strobing. Steps St63 to St67 in FIG. 20 are also applicable when shooting with a strobe in water.

  In FIG. 20 of the above-described embodiment, the underwater imaging mode may be set manually or automatically using a sensor for determining whether or not it is underwater. In the case of automatic setting by the sensor, the sensor is mounted on the image processing device (10), and the CPU (38) automatically sets the underwater imaging mode if the detection result of the sensor is underwater ( Step St61 in FIG. 20).

  The present invention is useful as an image processing apparatus that faithfully reproduces the color of an image captured in water.

10 Image processing device
11 Imaging unit
38 CPU (determination unit, white balance correction coefficient adjustment unit)
22b White balance adjustment circuit (white balance control unit)
22j memory (first storage unit, second storage unit)
22p Color attribute adjustment circuit (color attribute adjustment unit)

Claims (4)

An imaging unit for imaging a subject underwater;
A determination unit that determines, based on color distribution information of an image of the subject imaged by the imaging unit, a water depth at a position where the subject is imaged and a water quality state at the position;
A white balance correction coefficient adjusting unit for determining a white balance correction coefficient for correcting the white balance of the image according to the water depth and the water quality determined by the determination unit;
An image processing apparatus comprising: a white balance adjustment unit that performs white balance adjustment on the image using the white balance correction coefficient. In claim 1,
A first storage unit that stores white balance correction coefficient information in which the white balance correction coefficient is set in advance corresponding to a combination pattern of the water depth and the water quality state;
Further comprising
The image processing apparatus, wherein the white balance adjustment unit corrects the white balance of the image using the white balance correction coefficient information. In claim 1 or claim 2,
An image processing apparatus, further comprising: a color attribute adjustment unit that corrects at least one of brightness, saturation, and hue of the image according to the water depth and the water quality determined by the determination unit. In claim 3,
Stores color attribute coefficient information in which a color attribute coefficient for correcting at least one of brightness, saturation, and hue of the image corresponding to a combination pattern of the water depth and the water quality state is preset. A second storage unit,
Further comprising
The image processing apparatus, wherein the color attribute adjustment unit adjusts the color attribute coefficient using the color attribute coefficient information.

Images