JP6870817B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus that adjusts the white balance of an image.

An image processing device such as a digital camera performs white balance adjustment processing on the captured image. The white balance adjustment process is a process of adjusting the color balance of an image so that white is accurately projected as white even under light sources of various color temperatures. Specifically, the white balance of the image is adjusted by multiplying each of the R component and the B component by a gain so that the levels of the signals of the R component, the G component, and the B component in the image have a predetermined relationship.

Further, when the subject is imaged underwater, the image becomes bluish according to the water depth. This is because the water molecule absorbs the light of the R component, which has a longer wavelength than the G component and the B component. On the other hand, Patent Document 1 is disclosed as a technique for adjusting the white balance in an image captured in water. In Patent Document 1, white balance adjustment is performed on an image on an axis different from the blackbody radiation axis according to the water depth at the position of the captured subject.

Japanese Patent No. 5677113

However, phytoplankton is present in the water. Since phytoplankton has the property of absorbing the light of the B component, the larger the amount of phytoplankton, the stronger the green color of the image.

Then, in Patent Document 1, even if the bluishness of the image is improved by adjusting the white balance according to the water depth, the color of the image is still influenced by the phytoplankton in the water. Therefore, in Patent Document 1, it cannot be said that the color can be faithfully reproduced.

The present invention has been made in view of the above, and an object of the present invention is to perform white balance adjustment capable of faithfully reproducing a color tone of an image captured in water.

The first invention stores an image pickup unit that images a subject in water, and a water depth-water quality table that represents the water depth and water quality state determined by the R component value, the G component value, and the B component value of the image. and a storage unit and, by fitting the values of the B component value and the G component of the R component of the image of the object which the imaging unit has captured the table determination unit and a state of the water quality and the water depth A white balance correction coefficient adjusting unit that determines a white balance correction coefficient for correcting the white balance of the image according to the water depth and the water quality condition determined by the determination unit, and the white balance correction coefficient. using an image processing apparatus comprising: a white balance adjustment unit performs white balance adjustment on the image.

Water quality conditions include the content of phytoplankton in water. Here, in the image captured in water, white balance control is performed according to the water depth at the position where the subject is captured and the state of the water quality at the position. As a result, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, with the influence of not only the water depth but also the state of water quality reduced.

The second invention is the first invention, in the storage unit, a white balance correction coefficient information the white balance correction coefficients corresponding to the combination pattern of the state of the water quality and the water depth is preset Further stored, the white balance control unit is an image processing device characterized in that the white balance of the image is corrected by using the white balance correction coefficient information.

As a result, the white balance correction coefficient is quickly determined according to the actual water depth and the actual water quality condition, so that the white balance control process is performed relatively quickly. In addition, the white balance correction coefficient can be finely adjusted according to the actual water depth and the actual water quality condition.

The third invention corrects at least one of the brightness, saturation, and hue of the image according to the state of the water depth and the water quality determined by the determination unit in the first invention or the second invention. It is an image processing apparatus characterized by further including a color attribute adjusting unit.

As a result, at least one of the brightness, saturation, and hue of the image is tuned to be appropriate according to the actual water depth and the actual water quality condition. Therefore, it is possible to obtain an image that more faithfully reproduces the actual color in water.

A fourth aspect of the present invention is to correct at least one of the brightness, saturation, and hue of the image in the storage unit in accordance with the combination pattern of the water depth and the water quality state in the third invention. The color attribute coefficient information in which the color attribute coefficient of the above is preset is further stored, and the color attribute adjusting unit adjusts the color attribute coefficient by using the color attribute coefficient information. It is a processing device.

As a result, the color attribute coefficient is quickly determined according to the actual water depth and the actual water quality condition, so that at least one adjustment process of the saturation, lightness, and saturation of the image is performed relatively quickly. .. It is also possible to finely adjust the color attribute coefficient according to the actual water depth and the actual water quality condition.

According to the present invention, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, with the influence of not only the water depth but also the state of water quality reduced.

FIG. 1 is a block diagram schematically showing the configuration of an image processing device. FIG. 2 is a block diagram simulating the configuration of the signal processing circuit. FIG. 3 is a conceptual diagram of the weighting coefficient set in the weighting table. FIG. 4 is a diagram showing a color distribution state. FIG. 5 is a diagram showing (a) an external view of a subject and (b) an example of a screen when an captured subject is displayed. FIG. 6 is a diagram for explaining the concept of operation at the time of white balance adjustment. FIG. 7 is a conceptual diagram of a reference value table. FIG. 8 is a conceptual diagram of the target value table. FIG. 9 is a diagram showing the distribution state of the reference value and the target value. FIG. 10 is a diagram showing a part of a reference value and a target value. FIG. 11 is a diagram showing an operation flow of the area discrimination circuit. FIG. 12 is a diagram for explaining a part of the operation of adjusting the hue. FIG. 13 is a diagram for explaining a part of the operation of adjusting the saturation. FIG. 14 is a diagram for explaining a part of the operation of adjusting the brightness. FIG. 15 is a diagram showing a main flow of the CPU. FIG. 16 is a diagram showing the flow of the color tone correction process in FIG. FIG. 17 is a diagram showing the concept of white balance adjustment and hue adjustment when FIG. 16 is executed. FIG. 18 is a chromaticity diagram of the R component, the G component, and the B component in water. FIG. 19 is a diagram for explaining the concept of operation at the time of white balance adjustment in the underwater imaging mode. FIG. 20 is a diagram showing a flow of color adjustment operation in the underwater imaging mode. FIG. 21 is a conceptual diagram of a water depth-water quality table. FIG. 22 is a conceptual diagram of the parameter table.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the following embodiments are essentially preferred examples and are not intended to limit the scope of the present invention, its applications, or its uses.

<< Embodiment >>
<Overview>
The image processing device (10) according to the present embodiment is a type of digital camera capable of capturing an image not only on the ground but also in water. That is, in the image processing apparatus (10), it is possible to select a mode for capturing images on the ground and a mode for capturing images underwater. Regardless of which mode is selected, the image processing apparatus (10) performs color adjustment including white balance adjustment on the captured image, and outputs the adjusted image to a monitor (30) or the like described later.

By the way, when a general image processing device adjusts the white balance of an image captured underwater, the color of the image after the white balance adjustment and the color of the actual subject may deviate from each other. .. The factors include the distance from the water surface to the subject (that is, the water depth) and the state of the water quality in the vicinity of the subject. This is because the water depth and the degree of water quality make the light absorption rates of the R component, the G component, and the B component different.

The state of water quality includes the content of phytoplankton in water and the like.

On the other hand, the image processing apparatus (10) according to the present embodiment estimates and estimates the water depth and the state of water quality (phytoplankton content) at the position of the imaged subject when the subject is imaged in water. By performing color adjustment including white balance adjustment using the resulting result, it is possible to reproduce the actual color tone of the subject. In particular, the image processing apparatus (10) according to the present embodiment greatly exerts the effect when the image is taken without firing the strobe.

In the following, after describing the configuration and basic color adjustment processing of such an image processing apparatus (10) in advance, the color adjustment processing in the underwater imaging mode according to the present embodiment is described as <color adjustment in the underwater imaging mode. Processing> will be described in detail.

<Structure>
As shown in FIG. 1, the image processing apparatus (10) includes an image pickup unit (11), a signal processing circuit (22), a memory control circuit (24), an SDRAM (26), a video encoder (28), and a monitor (30). , I / F circuit (34), memory card (36) and CPU (38). The imaging unit (11) includes a focus lens (12), an image sensor (14), a TG (timing generator) (16), a CDS / AGC circuit (18), and an A / D converter (20). Each of the focus lens (12) and the image sensor (14) is driven by a driver (not shown).

The optical image of the subject is incident on the light receiving surface of the image sensor (14) via the focus lens (12). On the light receiving surface, an incident optical image is subjected to photoelectric conversion, whereby a camera signal (raw image signal) corresponding to the optical image is generated. The light receiving surface is covered with a color filter (not shown) in a primary color Bayer array, and each pixel signal forming the camera signal is one of the colors R (Red), G (Green), and B (Blue). It has only information components.

When the power is turned on, a processing instruction is given from the CPU (38) to each of the TG (16), the signal processing circuit (22), and the video encoder (28). As a result, through image processing is started.

The TG (16) repeatedly exposes the image sensor (14) and reads out the camera signal. The camera signal of each frame read from the image sensor (14) is noise-removed and level-adjusted in the CDS / AGC circuit (18), and then converted into a digital signal in the A / D converter (20). ..

The signal processing circuit (22) performs color separation processing, color adjustment processing, YUV conversion processing, and the like on the camera data of each frame output from the A / D converter (20) to generate image data. This image data includes Y data representing a luminance component, U data representing a color difference component, and V data. The image data generated by the signal processing circuit (22) is output to the memory control circuit (24) and stored in the SDRAM (26) by the memory control circuit (24).

In the color adjustment process, the white balance is adjusted for the camera data of each frame, and then the brightness, saturation, and hue are corrected. In the following, the adjustment coefficient used for white balance adjustment is referred to as "gain x1, y1", and the correction coefficient used for correction of brightness, saturation and color difference is referred to as "target L component value, target C component value" in order. , Target H component value ". These adjustment coefficients and correction coefficients are determined by the CPU (38).

When the video encoder (28) causes the memory control circuit (24) to read the image data of each frame stored in the SDRAM (26), the video encoder (28) encodes the image data into an NTSC format composite image signal. The video encoder (28) outputs the composite image signal to the monitor (30). As a result, the monitor (30) displays a real-time moving image (through image) of the subject.

When the imager fully presses the shutter button (40), the image processing apparatus (10) executes imaging and recording processing. First, the CPU (38) outputs a compression instruction to the JPEG codec (32). When the image data stored in the SDRAM (26) is read out via the memory control circuit (24), the JPEG codec (32) compresses the image data by the JPEG method. The compressed image data is stored in the SDRAM (26) again.

The I / F circuit (34) reads the image data stored in the SDRAM (26) via the memory control circuit (24), and records the read image data in the memory card (36). As a result, an image file is created in the memory card (36).

The memory card (36) can be composed of a non-volatile recording medium that can be attached to and detached from the housing (not shown) of the image processing device (10).

<Structure of signal processing circuit>
Here, the configuration of the signal processing circuit (22) will be described in detail.

As shown in FIG. 2, the signal processing circuit (22) includes a color separation circuit (22a), a white balance adjustment circuit (22b) corresponding to a white balance control unit, an integration circuit (22c), and an LCH conversion circuit (22d). Stores L correction circuit (22e), C correction circuit (22f), H correction circuit (22g), YUV conversion circuit (22h), area discrimination circuit (22i), and a plurality of tables (22m, 22n, ...). It has a memory (22j) (corresponding to a first storage unit and a second storage unit).

-Color separation circuit-
The color separation circuit (22a) performs color separation processing on the camera data output from the A / D converter (20). Since each pixel data constituting the camera data has only one of the R component, the G component, and the B component, the color separation circuit (22a) contains the other two color components that each pixel does not have. Complement. As a result, RGB format image data in which each pixel has all the color information of R, G, and B is generated.

-White balance adjustment circuit-
The white balance adjustment circuit (22b) has two amplifiers (221b, 222b). A gain x1 is given to the amplifier (221b), and a gain y1 is given to the amplifier (222b). Of the generated image data, the R component is input to the amplifier (221b) and amplified by the gain x1 at the amplifier (221b). Of the generated image data, the B component is input to the amplifier (222b) and amplified by the gain y1 at the amplifier (222b).

The R component and B component amplified by the white balance adjustment circuit (22b) and the G component of the unamplified image data are input to the integration circuit (22c) and the LCH conversion circuit (22d).

The screen is divided into, for example, 16 in the horizontal and vertical directions, in which case 256 division areas are formed on the screen. The integrating circuit (22c) integrates the R component, the G component, and the B component for each divided area and for each of the same color components. Therefore, 256 integrated values Ir (i), 256 integrated values Ig (i), and 256 integrated values Ib (i) are obtained for each frame period. Here, "i" is an integer from 1 to 256. That is, 256 color evaluation values corresponding to 256 divided areas are obtained for each frame period.

The CPU (38) adjusts the gains x1 and y1 used for the white balance adjustment based on the color evaluation value, and calculates the optimum gains xs and ys.

Specifically, the CPU (38) converts the integrated value Ir (i) into the integrated value Iry (i), and converts the integrated value Ib (i) into the integrated value Iby (i). The integrated values Iry (i) and integrated Iby (i) are equal to the integrated values of the color difference components RY and BY for each divided area, and can be said to be the color evaluation values of each divided area.

Subsequently, the CPU (38) uses the weighting coefficient k (3) of each division area based on the weighting table (38a) having the numerical distribution according to FIG. 3 and the integrated values Iry (i) and Iby (i) of each division area. i) is sought. The weighting table (38a) is stored in a memory (not shown), and in FIG. 3, each weighting coefficient k (i) is stored as “0” or “1”. The weighting coefficient k (i) in FIG. 3 corresponds to the color distribution state shown in FIG.

In the weighting table (38a) of FIG. 3, the BY axis, which is the horizontal axis, and the RY axis, which is the vertical axis, are each divided into 17 equal parts, and are separated by the BY axis and the RY axis, respectively. An example is shown in which the quadrant of is divided into 64. As a result, 290 areas are formed in FIG. 3, and the weighting coefficient K (i) is assigned “0” or “1” for each area. Of the 290 areas according to FIG. 3, the range to which the weighting coefficient k (i) = 1 is assigned (the range surrounded by the solid line in FIG. 3) is the pull-in range.

After obtaining the weighting coefficient k (i) for each division area, the CPU (38) uses the weighting coefficient k (i) for each of the following equations (1) and (2) to obtain the optimum gain xs, ys. Is calculated.

As described above, since the weighting coefficient k (i) is "0" or "1", according to the above equations (1) and (2), only the divided area whose color evaluation value is within the pull-in range is valid. Become. Therefore, the sum of the integrated values Ir (i), Ig (i), and Ib (i) of the effective division areas can be obtained by the above equations (1) and (2). From the above equation (1), the optimum gain xs based on the sum of the integrated values Ir (i) and Ig (i) can be obtained, and from the above equation (2), the integrated values Ig (i) and Ib (i) are obtained. ) The optimum gain ys based on the sum of each is obtained. The optimum gains xs and ys thus obtained are values such that the average value of the color evaluation values included in the pull-in range converges at the intersection of the RY axis and the BY axis.

As shown in FIG. 5 (a), when the cylindrical rod (44) placed on the paper (42) is imaged from directly above, the monitor (30) shows the image in FIG. 5 (b). The subject image in such a state is displayed. The vertical lines and horizontal lines shown in FIG. 5B are lines for dividing the screen into 256 divided areas, and are not actually displayed. If the paper (42) is light orange and the stick (44) is dark yellow, the integrated values Iry (i) and Iby (i) obtained by integrating only the colors of the paper (42) are included in the pull-in range. The integrated values Iry (i) and Iby (i) obtained by integrating only the colors of the rods (44) are out of the pull-in range. As a result, the optimum gains xs and ys are determined based on the integrated values Ir (i), Ig (i), and Ib (i) for the color of the paper (42).

When the integrated values Iry (i) and Iby (i) relating to the color of the paper (42) before the optimum gains xs and ys are calculated are distributed in the area 1 shown in FIG. 6, the optimum gains xs and ys are calculated and the optimum gains xs and ys are calculated. The integrated values Iry (i) and Iby (i) relating to the color of the paper (42) after being set in the amplifiers (221b, 222b) move to the area 2 shown in FIG. That is, the integrated values Iry (i) and Iby (i) relating to the color of the paper (42) move only by the movement amount W1 toward the intersection of the RY axis and the BY axis.

-LCH conversion circuit-
The LCH conversion circuit (22d) converts the R component and B component of each pixel amplified by the white balance adjustment circuit (22b) and the G component of each pixel output from the color separation circuit (22a) into a brightness component. It is converted into an L component, which is a saturation component, a C component, which is a saturation component, and an H component, which is a hue component. The LCH conversion circuit (22d) outputs the converted LCH data (data having the L component, C component, and H component of each pixel). The L component data in the LCH data is input to the L correction circuit (22e), and the C component data is input to the C correction circuit (22f). The H component data is input to the H correction circuit (22g) and the area discrimination circuit (22i).

-Each correction circuit, YUV conversion circuit, and area discrimination circuit-
The L correction circuit (22e) obtains the correction L component based on the L component of the input LCH data. The C correction circuit (22f) obtains the correction C component based on the C component of the input LCH data. The H correction circuit (22 g) obtains the corrected H component based on the H component of the input LCH data. The correction L component, the correction C component, and the correction H component are input to the YUV conversion circuit (22h).

The YUV conversion circuit (22h) converts LCH format image data into YUV format image data based on the input correction L component, correction C component, and correction H component. As a result, the YUV conversion circuit (22h) outputs image data having a Y component which is luminance data, a U component which is a color difference component, and a V component.

The area determination circuit (22i) determines the area to which the H component belongs for each pixel with reference to the reference value table (22m) stored in the memory (22j). The area discrimination circuit (22i) further reads the reference value corresponding to the discrimination result from the reference value table (22m) and reads the target value corresponding to the discrimination result from the target value table (22n).

FIG. 7 is a conceptual diagram of the reference value table (22 m). As shown in FIG. 7, in the reference value table (22 m), a combination of 1 of the reference H component value indicating hue, the reference C component value indicating saturation, and the reference L component value indicating lightness is set as one record. A total of 6 patterns are represented. “N” in FIG. 7 represents a record number, and the reference H component value, the reference C component value, and the reference L component value represented by the same record number define one reference value. The reference values of the six patterns shown in FIG. 7 correspond to each of the six representative colors of Mg (magenda), R (red), Ye (yellow), G (green), Cy (cyan), and B (blue). Then, as shown in FIGS. 9 and 10, it is distributed in the YUV space. Note that FIG. 10 shows a reference value corresponding only to Cy.

FIG. 8 is a conceptual diagram of the target value table (22n). As shown in FIG. 8, in the target value table (22n), a combination of 1 of a target H component value indicating hue, a target C component value indicating saturation, and a target L component value indicating lightness is set as one record. A total of 6 patterns are represented. “N” in FIG. 8 represents a record number, and the target H component value, the target C component value, and the target L component value represented by the same record number define one target value. The target values of the six patterns shown in FIG. 8 correspond to each of the six representative colors of Mg (magenda), R (red), Ye (yellow), G (green), Cy (cyan), and B (blue). Then, as shown in FIGS. 9 and 10, it is distributed in the YUV space. Note that FIG. 10 shows a target value corresponding only to Cy.

The reference H component value, the reference C component value, and the reference L component value set in the reference value table (22 m) are values set in advance at the manufacturing stage of the image processing apparatus (10) and cannot be changed. It is the value that was set. The target H component value, the target C component value, and the target L component value set in the target value table (22n) are set to default values at the manufacturing stage of the image processing apparatus (10), but after that, the CPU (38) ) Is a value that is appropriately changed.

The region discrimination circuit (22i) executes the process according to FIG. 11 using the H correction component value output from the LCH conversion circuit (22d) and the above two tables (22m, 22n).

First, the area discrimination circuit (22i) sets the variable record number N to “1” (step St1), and reads out the reference H component value corresponding to the record number N “1” from the reference value table (22m). (Step St2). The region discrimination circuit (22i) compares the read reference H component value with the H component value of the pixel of interest (hereinafter, the current pixel H component value) (step St3).

When the reference H component value is larger than the current pixel H component value (YES in step St3), the area determination circuit (22i) determines whether or not the record number N is “1” (step St6). If the record number N is not "1" (NO in step St6), the processes of steps St7 to St10 are performed. When the reference H component value is equal to or smaller than the current pixel H component value (NO in step St3), the area determination circuit (22i) increments the record number N (step St4), and the incremented record number N becomes It is determined whether or not it is larger than "6" (step St5). When the record number N after the increment is equal to or smaller than "6" (NO in step St5), the process of shifting to step St2 is repeated. When the record number N after incrementing is larger than "6" (YES in step St5), or when the record number N is "1" in step St6 (YES in step St6), the processing of steps St11 to St14. Is done.

In step St7, the region discrimination circuit (22i) sets the reference H component value, the reference C component value, and the reference L component value corresponding to the current record number N as “Hrα”, “Crα”, and “Lrα” in the reference value table ( Select from 22m). In step St8, the area discrimination circuit (22i) sets the target H component value, the target C component value, and the target L component value corresponding to the current record number N as “Htα”, “Ctα”, and “Ltα” in the target value table ( Select from 22n).

In step St9, the region discrimination circuit (22i) sets the reference H component value, the reference C component value, and the reference L component value corresponding to the number before the current record number N (that is, “N-1”) to “Hrβ”. Select from the reference value table (22m) as "Crβ" and "Lrβ". In step St10, the region discrimination circuit (22i) sets the target H component value, the target C component value, and the target L component value corresponding to the number before the current record number N (that is, “N-1”) to “Htβ”. Select from the target value table (22n) as "Ctβ" and "Ltβ".

On the other hand, in step St11, the region discrimination circuit (22i) sets the reference H component value “Hr1”, the reference C component value “Cr1”, and the reference L component value “Lr1” corresponding to the record number N “1”. Select from the reference value table (22 m) as Hrα, “Crα”, and “Lrα”. In step St12, the area discrimination circuit (22i) sets the target H component value “Ht1”, the target C component value “Ct1”, and the target L component value “Lt1” corresponding to the record number N “1” to “Htα”. Select from the target value table (22n) as “Ctα” and “Ltα”.

In step St13, the region discrimination circuit (22i) sets the reference H component value “Hr6”, the reference C component value “Cr6”, and the reference L component value “Lr6” corresponding to the record number N “6” to “Hrβ”. Select from the reference value table (22m) as "Crβ" and "Lrβ". In step St14, the region discrimination circuit (22i) sets the target H component value “Ht6”, the target C component value “Ct6”, and the target L component value “Lt6” corresponding to the record number N “6” to “Htβ”. Select from the target value table (22n) as “Ctβ” and “Ltβ”.

In this way, two reference values having two reference H component values sandwiching the H component value of the pixel of interest and two target values corresponding to these two reference values are detected.

The reference H component values Hrα and Hrβ and the target H component values Htα and Htβ are input to the H correction circuit (22 g). The reference C component values Crα and Crβ and the target C component values Ctα and Ctβ are input to the C correction circuit (22f). The reference L component values Lrα and Lrβ and the target L component values Ltα and Ltβ are input to the L correction circuit (22e).

The H correction circuit (22g) uses the following equation (3) to convert the current pixel H component value Hin input from the LCH conversion circuit (22d), the reference H component values Hrα, Hrβ, and the target H component values Htα, Htβ. By substituting into (5), the current pixel H component value Hin is converted into the corrected H component value Hout. The corrected H component value Hout has a magnitude shown by a broken line in FIG.

The H correction circuit (22g) also outputs the angle data θ1 and θ2 to the C correction circuit (22f) and the L correction circuit (22e), and also outputs the angle data θ3 (= | Htβ-Hout |) and the angle data θ4 (= | Htβ-Hout |). = | Htα-Hout |) is output to the L correction circuit (22e).

The C correction circuit (22f) uses the following equation (6) for the current pixel C component value Cin input from the LCH conversion circuit (22d), the reference C component values Crα and Crβ, and the target C component values Ctα and Ctβ. By substituting into, the current pixel C component value Cin is converted into the corrected C component value Cout. The corrected C component value Hout has a size shown by a broken line in FIG.

The C correction circuit (22f) further has a straight line connecting the coordinates (0,0) and (Cin, Hin) of the CH system and the coordinates (Crα, Hrα), (Crβ) based on the following equations (7) and (8). , Hrβ) and the C component value Crγ at the coordinates of the intersection with the straight line, and the straight line connecting the coordinates (0,0) and (Cout, Hout) of the CH system and the coordinates (Ctα, Htα), (Ctβ, Htβ). The C component value Ctγ at the coordinates of the intersection with the straight line is calculated. The C correction circuit (22f) outputs the calculated C component values Crγ and Ctγ to the L correction circuit (22e) together with the current pixel C component value Cin and the correction C component value Cout described above.

The L correction circuit (22e) converts the current pixel L component value Lin input from the LCH conversion circuit (22d) into the correction L component value Lout according to the following equations (9) to (15). The corrected L component value Lout is shown in FIG.

Lmax and Lmin shown in FIG. 14 are the maximum and minimum values of L (brightness) that can be reproduced, respectively. The current pixel (input pixel value) is a surface (YUV space cut out with hue Hin) formed by LCH-based coordinates (Lmax, 0,0), (Lmin, 0,0), (Lrγ, Crγ, Hin). Face) exists on. On the other hand, the correction pixel value is a surface formed by LCH-based coordinates (Lmax, 0,0), (Lmin, 0,0), (Ltγ, Ctγ, Hout) (a surface obtained by cutting out the YUV space with hue Hout). Exists on.

The corrected pixel value is defined by the corrected H component value Hout, the corrected C component value Cout, and the corrected L component value Lout obtained as described above. The current pixel value is defined by the current pixel H component value Hin, the current pixel C component value Cin, and the current pixel L component value Lin output from the LCH conversion circuit (22d).

<Color tone correction processing by CPU>
Next, the basic color tone correction process will be described with reference to FIGS. 15 and 16.

As shown in FIG. 15, first, the CPU (38) sets the reference gains xγ and yγ to the gains x1 and y1 (step St31). At this time, gain x1, x1, that is, reference gains xγ and yγ are set in each amplifier (221b, 222b) of the white balance adjustment circuit (22b). The reference gains xγ and yγ are gains for which the white balance is appropriately adjusted when a subject having a color temperature (reference temperature) of about 5100 K is imaged, and are determined at the manufacturing stage, so to speak, default values. Is.

Next, the CPU (38) outputs a through image processing command to the TG (16), the signal processing circuit (22), and the video encoder (28) in order to start the through image processing (step St32). Then, the CPU (38) performs color tone correction (step St33) until the shutter button (40) of the image processing device (10) is pressed (NO in step St34), and the shutter button (40) is pressed. If this is the case (YES in step St34), imaging / recording processing is performed (step St35). As a result, the compressed image data of the subject image captured in response to the operation of the shutter button (40) is recorded in the memory card (36) in a file format.

The color tone correction is performed according to FIG. First, the CPU (38) determines whether or not a vertical synchronization signal is generated (step St41). When the vertical synchronization signal is generated (YES in step St41), the CPU (38) determines the integrated values Ir (i) and Ig (i) of the R component, G component, and B component calculated by the integrating circuit (22c). , Ib (i) is taken in (step St42). Next, the CPU (38) calculates the optimum gain xs, ys using the integrated values Ir (i), Ig (i), and Ib (i) (step St43), and gains the calculated optimum gain xs, ys. It is held as x2 and y2 (step St44). The CPU (38) calculates the difference Δx between the gain x2 and the gain x1 and the difference Δy between the gain y2 and the gain y1 (step St45).

The calculated gain differences Δx and Δy are changes in the white balance of the subject image, and correspond to the movement amount W1 from the area 1 to the area 2 indicated by the white circles in FIGS. 6 and 17. Based on such gain differences Δx and Δy, the CPU (38) sets the target H component value, the target C component value, and the target L component value in the target value table (22n) into the H component (hue) and the C component (). Saturation) and L component (brightness) are changed in this order (step St46). The changed target H component value, target C component value, and target L component value move by the amount of movement T1 to the positions indicated by the white squares in FIG.

That is, as shown in FIG. 17, the moving direction from the area 1 to the area 2 in the white balance and the moving direction of each target component value are opposite to each other. The movement amount T1 of each target component value is an amount that cancels the movement amount W1 in the white balance (| W1 | = | T1 |).

After the processing of step St46 is completed, the CPU (38) holds the optimum gain x2 and y2 calculated in step St44 as gains x1 and x1 (step St47), and performs the processing after step St34 in FIG.

<Color adjustment processing in underwater imaging mode>
As described above, the image processing apparatus (10) according to the present embodiment can set the underwater imaging mode. When the imaging unit (11) captures an image of a subject underwater without firing a strobe while the underwater imaging mode is set, the image reproduces the actual color of the subject more faithfully as described below. Is processed.

Hereinafter, for convenience of explanation, the L correction circuit (22e), the C correction circuit (22f), the H correction circuit (22g), and the area discrimination circuit (22i) are collectively referred to as a color attribute adjustment circuit (22p).

FIG. 18 is a chromaticity diagram for explaining how the R component, the B component, and the G component of the image captured in water are displaced. In FIG. 18, the distribution of each color of the reference R component, B component, and G component is represented by a solid line.

As shown in FIG. 18, the color distribution of the image captured in water is affected by the influence of the water depth at the imaging location (position of the subject) in water and the water quality (specifically, the content of phytoplankton in water). It changes in response to.

Specifically, when the water depth is constant, the larger the amount of phytoplankton, the more the color distribution shifts upward on the vertical axis in FIG. 18, and conversely, the smaller the amount of phytoplankton, the more. The transition is entirely downward on the vertical axis. When the amount of phytoplankton is constant, the color distribution shifts to the right of the horizontal axis in FIG. 18 as the water depth is shallower, and conversely, the color distribution is as a whole to the left of the horizontal axis as the water depth is deeper. Transition to. Then, when the amount of phytoplankton is relatively large, the deeper the water depth, the stronger the green color (dotted line in FIG. 18), and when the amount of plankton is relatively small, the deeper the water depth, the stronger the bluish color (broken line in FIG. 18). ). This is because, of the light incident on the water from the ground, the R component is easily absorbed by the water and the B component is easily absorbed by the phytoplankton.

Therefore, the CPU (38) of the image processing device (10) according to the present embodiment first functions as a determination unit when imaging is performed in a state where the underwater imaging mode is selected, and the water depth at the position of the subject. And the state of water quality at that position. Next, the CPU (38) functions as a white balance correction coefficient adjusting unit, and white balance for correcting the gain x1 and y1 used for white balance adjustment of the image according to the determined water depth and water quality state. Determine the correction factor. In parallel with this, the CPU (38) uses a color attribute coefficient for each correction of the H component (hue), the C component (saturation), and the L component (brightness) according to the determined water depth and water quality state. (Specifically, the correction parameters for individually correcting the target H component value, the target C component value, and the target L component value described above) are determined.

FIG. 19 shows a diagram for explaining the concept of how to adjust the white balance when an image is taken in water. In the image captured in water, the G component and the B component are strengthened by the influence of the water depth and the water quality, so that the image taken on the coordinate plane defined by the U axis and the V axis is not the area 1 of FIGS. 6 and 17. The image before white balance adjustment may be located in the area 3 located in the third quadrant, the area 4 in the fourth quadrant, or the like. As an example, the image is located in area 3 when the amount of phytoplankton is high and in area 4 when the amount of phytoplankton is low. Further, as an example, in both the area 3 and the area 4, the position of the image approaches the intersection of the V-axis and the U-axis when the water depth is shallow, but is far from the intersection when the water depth is deep.

Regardless of which area the image before white balance adjustment exists in, the CPU (38) of the present embodiment finely adjusts the gain x1 and y1 by determining the white balance correction coefficient, and the signal processing circuit (22). Moves the image to the area 2 which is the intersection of the U-axis and the V-axis by adjusting the white balance using the gains x1 and y1. Further, the CPU (38) of the present embodiment appropriately determines the color attribute coefficient with the determination of the white balance correction coefficient, and the color attribute adjustment circuit (22p) uses the determined color attribute coefficient, for example, of an image. Further fine-tune adjustment processing of color tone such as slightly strengthening redness is performed.

By these processes, as shown in FIG. 19, as the image moves from area 3 or area 4 to area 2 by white balance adjustment, Mg (magenda), R (red), Ye (yellow), G The target component values of each of the colors (green), Cy (cyan), and B (blue) move in the direction opposite to the moving direction of the white balance adjustment. Specifically, when the movement by the white balance adjustment moves from the area 3 to the area 2, the movement direction of the target component value of each color is the direction from the area 2 to the area 3, and the movement amount is the white balance. It is an amount T1'that offsets the movement amount W1'in the above. When the movement by the white balance adjustment moves from the area 4 to the area 2, the movement direction of the target component value of each color is the direction from the area 2 to the area 4, and the movement amount is the movement amount W1 in the white balance. It is the amount T1 that offsets ".

By the above processing, in the present embodiment, it is possible to reproduce an actual color tone as if the image was captured by firing the strobe with respect to the image captured in water without firing the strobe.

Hereinafter, the flow of the control operation will be described in detail with reference to FIGS. 20 to 22.

First, it is assumed that the underwater imaging mode is set (YES of St61) and the image is taken without firing the strobe (YES of St62). The CPU (38) uses the integrated value Ir (i) of the R component, the Ig (i) of the G component, and the integrated value Ib (i) of the B component input from the integration circuit (22c) described above, and the ratio coefficient. “R / G” and “B / G” are obtained (step St63). The CPU (38) applies the obtained ratio coefficients “R / G” and “B / G” to the water depth-water quality table (22q) according to FIG. The content of phytoplankton in water) is calculated from the values in the table (22q) (step St64). That is, the CPU (38) determines the water depth at the position where the subject is imaged and the state of the water quality at the position based on the color distribution information of the image of the subject captured by the imaging unit (11).

Here, FIG. 21 is a conceptual diagram of a water depth-water quality table (22q) stored in the memory (22j). The water depth-water quality table (22q) takes the water depth vertically and the phytoplankton content in the water as the water quality condition horizontally, and the ratio coefficient "R / G" corresponding to each combination pattern of the water depth and the water quality condition. "" B / G "is represented in a matrix. “R / G” represents the ratio of the R component and the G component of the image, and “B / G” represents the ratio of the B component and the G component. The ratio coefficients "R / G" and "B / G" are defined in advance by theory, experiment, etc., on the condition of each combination pattern of the water depth and the state of water quality. The CPU (38) easily calculates data representing the water depth and the state of the water quality by applying the obtained ratio coefficients “R / G” and “B / G” to the water depth-water quality table (22q) of FIG. be able to.

Next, the CPU (38) applies the calculated data representing the water depth and the state of the water quality to the parameter table (22r) of FIG. 22 to configure the white balance adjustment circuit (22b) and the color attribute adjustment circuit (22q). Various parameters to be set in the various circuits (22e, 22f, 22g, 22i) to be set are selected (step St65).

Here, FIG. 22 is a conceptual diagram of the parameter table (22r) stored in the memory (22j). In the parameter table (22r), the white balance correction coefficient for correcting each gain x1 and y1 and the correction of the target H component value are shown for each combination pattern of the water depth and the water quality state (the content of phytoplankton in water). The parameters, the correction parameter of the target C component value, and the correction parameter of the target L component value are set in advance. In other words, this parameter table (22r) shows the fine white balance coefficient and color attribute coefficient for reproducing faithful color appearance regardless of the influence of color due to water depth and water quality on the image captured underwater. Can be said to have been prepared in advance. For example, the various parameters in the parameter table (22r) are set so that the deeper the water depth, the larger the correction value, or the larger the amount of phytoplankton, the larger the correction value. Can be done.

Both of the above two tables (22q, 22r) correspond to the white balance correction coefficient information and the color attribute coefficient information.

Next, the CPU (38) sets various parameters selected in step St65 in the color attribute adjusting circuit (22q) and the white balance adjusting circuit (22b) (step St66). Specifically, the white balance correction coefficient for correcting the gain x1 and y1 is set in the white balance adjustment circuit (22b). The color attribute coefficient is set in the color attribute adjustment circuit (22p).

The color attribute adjustment circuit (22q) and the white balance adjustment circuit (22b) in which various parameters are set in step St66 perform color adjustment on the target image based on the parameters (step St67). Specifically, the white balance adjustment circuit (22b) adjusts the white balance of the image using the white balance correction coefficients obtained from the two tables (22q and 22r) and the gains x1 and y1. The color attribute adjustment circuit (22p) uses the color attribute coefficient, the target H component value, the target C component value, and the target L component value obtained from the two tables (22q, 22r) to obtain the brightness, saturation, and image brightness. Correct the hue.

<Effect>
In the present embodiment, when the water depth and the state of water quality at the position where the subject is imaged are determined from the color distribution information of the image captured in water, the determination result is used to correct the white balance of the image. The white balance correction coefficient is determined, and the white balance adjustment is performed on the image using the coefficient. As a result, it is possible to obtain an image in which the actual color is reproduced as faithfully as possible, with the influence of not only the water depth but also the state of water quality reduced.

Further, in the present embodiment, the white balance correction coefficient is set in advance corresponding to the combination pattern of the water depth and the water quality state. Therefore, the white balance correction coefficient is quickly determined according to the actual water depth and the actual water quality condition, and the white balance control process is performed relatively quickly. In addition, the white balance correction coefficient can be finely adjusted according to the actual water depth and the actual water quality condition.

Further, in the present embodiment, the brightness, saturation, and hue of the image are tuned to be appropriate according to the determined water depth and water quality. Therefore, it is possible to obtain an image that more faithfully reproduces the actual color in water.

Further, in the present embodiment, color attribute coefficients for correcting the brightness, saturation, and hue of the image are set in advance according to the combination pattern of the water depth and the water quality state. Therefore, the color attribute coefficient is quickly determined according to the actual water depth and the actual water quality state, and the saturation, lightness, and saturation adjustment processing of the image is performed relatively quickly. It is also possible to finely adjust the color attribute coefficient according to the actual water depth and the actual water quality condition.

<< Other Embodiments >>
In the above embodiment, it has been explained that the white balance correction coefficient for correcting the gain x1 and y1 is determined according to the water depth and the water quality state, but the gain x1 and y1 itself are determined according to the water depth and the water quality state. It may be determined as a white balance correction coefficient.

In the above embodiment, it has been explained that the correction parameter of the target H component value, the correction parameter of the target C component value, and the correction parameter of the target L component value are determined as the color attribute coefficient according to the water depth and the state of the water quality. , The target H component value itself, the target C component value itself, and the target L component value itself may be determined as color attribute coefficients according to the state of water depth and water quality.

In the above embodiment, the case where the white balance correction coefficient information and the color attribute coefficient information are represented by the tables (22q, 22r) according to FIGS. 21 and 22 is illustrated, but they are represented by the tables of FIGS. 21 and 22. Not limited to things. For example, the white balance correction coefficient information and the color attribute coefficient information may be obtained by calculation each time using an equation.

In the above embodiment, the case where all the brightness, saturation and hue of the image are corrected according to the state of water depth and water quality is illustrated, but at least one of the brightness, saturation and hue of the image may be corrected. ..

In FIG. 20 of the above embodiment, a case where an image is taken in water without firing a strobe is illustrated. However, the present invention is not limited to the case where the strobe is imaged without being fired. Steps St63 to St67 in FIG. 20 are also applicable when the strobe is fired in water for imaging.

In FIG. 20 of the above embodiment, the underwater imaging mode may be set manually or automatically by using a sensor for determining whether or not it is underwater. In the case of automatic setting by the sensor, the sensor is mounted on the image processing device (10), and the CPU (38) automatically sets the underwater imaging mode if the detection result of the sensor is underwater ( Step St61 in FIG. 20).

The present invention is useful as an image processing device that faithfully reproduces the color appearance of an image captured in water.

10 Image processing equipment
11 Imaging unit
38 CPU (judgment unit, white balance correction coefficient adjustment unit)
22b White balance adjustment circuit (white balance control unit)
22j memory (1st storage, 2nd storage)
22p color attribute adjustment circuit (color attribute adjustment unit)

Claims (4)

An imaging unit that captures the subject underwater,
A storage unit that stores a water depth-water quality table that represents the state of water depth and water quality determined by the value of the R component, the value of the G component, and the value of the B component of the image.
A determination unit for determining the water depth and the state of the water quality by applying the R component value, the G component value, and the B component value of the image of the subject imaged by the imaging unit to the water depth-water quality table.
A white balance correction coefficient adjusting unit that determines a white balance correction coefficient for correcting the white balance of the image according to the water depth and the state of the water quality determined by the determination unit.
An image processing apparatus including a white balance adjusting unit that adjusts the white balance of the image by using the white balance correction coefficient.
In claim 1,
The storage unit further stores white balance correction coefficient information in which the white balance correction coefficient is preset according to the combination pattern of the water depth and the water quality state.
The white balance adjusting unit is an image processing apparatus characterized in that the white balance of the image is corrected by using the white balance correction coefficient information. In claim 1 or 2,
An image processing apparatus further comprising a color attribute adjusting unit that corrects at least one of the brightness, saturation, and hue of the image according to the state of the water depth and the water quality determined by the determination unit. In claim 3,
In the storage unit, a color attribute coefficient for correcting at least one of the brightness, saturation, and hue of the image corresponding to the combination pattern of the water depth and the water quality state is preset. Coefficient information is further stored,
The color attribute adjusting unit is an image processing apparatus characterized in that the color attribute coefficient is adjusted by using the color attribute coefficient information.

Images