KR102366254B1 - Image processing apparatus and method - Google Patents

Abstract

Step of receiving a captured image, determining block statistics representing the image characteristics of the block using a plurality of blocks obtained by dividing the captured image, applying the determined block statistics and the amount of infrared light included in the light source to the captured image Disclosed is an image processing method including determining a shading estimation coefficient to be used, and correcting shading of a captured image by using the determined shading estimation coefficient.

Description

Image processing apparatus and method {IMAGE PROCESSING APPARATUS AND METHOD}

Various embodiments relate to an image processing apparatus and method, and more particularly, to an image processing apparatus and an image processing method for correcting color shading of a captured image using various shading estimations.

The image processing apparatus may divide the captured image into a plurality of blocks and correct color shading of the captured image by using various statistical values of the blocks.

Various embodiments are intended to provide an image processing apparatus and method for correcting color shading of a captured image by using various shading estimations.

As a technical means for achieving the above technical problem, an embodiment of the present disclosure includes an input unit for receiving a captured image; and using a plurality of blocks obtained by dividing the captured image, block statistics indicating image characteristics of each of the plurality of blocks are determined, and the determined block statistics and the amount of infrared light included in the light source are used to apply to the captured image A data processing unit that determines a shading estimation coefficient to be performed and corrects shading of the captured image by using the determined shading estimation coefficient.

The data processing unit according to an embodiment may determine a shading estimation coefficient to be applied to the captured image by using at least one of a luminance of the captured image, a color temperature of the light source, and flatness of the captured image. there is.

The data processing unit according to an embodiment may determine the flatness of the captured image by adding up differences in characteristic values of blocks continuous from a peripheral portion to a central portion of the captured image.

The data processing unit according to an embodiment determines a block color evaluation value using the determined block statistics, determines a block weight using the determined block color evaluation value, and calculates the determined block statistics and the determined block weight. and determining a block evaluation value using the block evaluation value, and determining a shading estimation coefficient to be applied to the captured image by using the block evaluation value.

The data processing unit according to an embodiment may determine a histogram weight using the determined block color evaluation value, and determine the block weight using the histogram weight and the G-level weight.

The data processing unit according to an embodiment classifies the plurality of blocks into a plurality of groups, determines a group evaluation value using the determined block statistics and the classified group, and determines the determined block statistics and the determined group evaluation The effective group may be determined using the value, and a shading estimation coefficient to be applied to the captured image may be determined using the determined effective group.

The data processing unit may determine the group evaluation value by averaging an average value of the determined block evaluation values for each distance from the center of the captured image.

The data processing unit according to an embodiment may estimate an approximate straight line using the determined group evaluation value, determine a shading estimation coefficient to be applied to the captured image using the estimated approximate straight line, and determine the determined block evaluation value may be used to estimate a sample variance, and a shading estimation coefficient to be applied to the captured image may be determined using the estimated sample variance.

The data processing unit according to an embodiment may determine a shading estimation coefficient to have a value closest to 0 among the slopes of the negative number when the slope of the estimated approximate straight line is negative, and the slope of the estimated approximate straight line is negative. If not, the shading estimation coefficient to have the smallest value among the non-negative gradients may be determined.

The data processing unit according to an embodiment may estimate a variance using the determined group evaluation value, determine a shading estimation coefficient to be applied to the captured image using the estimated variance, and use the determined block evaluation value to estimate the sample variance, and determine a shading estimation coefficient to be applied to the captured image using the estimated sample variance.

In addition, another embodiment of the present disclosure includes the steps of receiving a captured image; determining block statistics indicating image characteristics of the blocks using a plurality of blocks obtained by dividing the captured image; determining a shading estimation coefficient to be applied to the captured image using the determined block statistics and an amount of infrared light included in a light source; and correcting shading of the captured image by using the determined shading estimation coefficient.

The determining of the shading estimation coefficient according to an embodiment may include determining a shading estimation coefficient to be applied to the captured image using at least one of luminance, light source color temperature, and flatness of the captured image. may include

The flatness of the captured image according to an embodiment may be determined by adding differences between characteristic values of blocks continuous from a peripheral portion to a central portion of the captured image to determine the flatness of the captured image.

The determining of the shading estimation coefficient according to an embodiment may include: determining a block color evaluation value using the determined block statistics; determining a block weight using the determined block color evaluation value; determining a block evaluation value using the determined block statistics and the determined block weight; and determining a shading estimation coefficient to be applied to the captured image by using the determined block evaluation value.

The determining of the block weight according to an embodiment may include determining a histogram weight using the determined block color evaluation value, and determining the block weight using the histogram weight and the G-level weight. .

The determining of the shading estimation coefficient according to an embodiment may include: classifying the plurality of blocks into a plurality of groups; determining a group evaluation value using the determined block statistics and the classified group; determining a valid group using the determined block statistics and the determined group evaluation value; and determining a shading estimation coefficient to be applied to the captured image by using the determined effective group.

The determining of the group evaluation value according to an embodiment may include averaging an average value of the determined block evaluation values for each distance from the center of the captured image to determine the group evaluation value.

The determining of the shading estimation coefficient according to an embodiment may include estimating an approximate straight line using the determined group evaluation value, and determining a shading estimation coefficient to be applied to the captured image using the estimated approximate straight line. ; The method may further include estimating a sample variance using the determined block evaluation value and determining a shading estimation coefficient to be applied to the captured image using the estimated sample variance.

The determining of the shading estimation coefficient according to an embodiment may include: estimating a variance using the determined group evaluation value, and determining a shading estimation coefficient to be applied to the captured image using the estimated variance; and estimating a sample variance using the determined block evaluation value, and determining a shading estimation coefficient using the estimated sample variance.

In addition, it is possible to provide a computer-readable recording medium in which a program for executing a method according to another embodiment of the present disclosure is recorded in a computer.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment.
2 is a block diagram illustrating a configuration of an image processing apparatus according to another exemplary embodiment.
3 is a block diagram illustrating a configuration of a shading estimator according to an exemplary embodiment.
4 is a flowchart illustrating a method of operating an image processing apparatus according to an exemplary embodiment.
5 is a flowchart illustrating a method of operating a shading estimator according to an exemplary embodiment.
6 is a diagram illustrating an example of an image for explaining an operation of a flat determiner according to an exemplary embodiment.
7 is a diagram illustrating an example of a pixel value for explaining an operation of a flat determiner according to an exemplary embodiment.
8 is a diagram illustrating an example of an image for explaining an operation of a flat determiner according to an embodiment.
9 is a flowchart illustrating an operation method of a shading estimation coefficient determiner according to an exemplary embodiment.
10 is a diagram illustrating an example of a group classified by a shading estimation coefficient determiner according to an embodiment.
11 is a diagram illustrating an example of a group classified by a shading estimation coefficient determiner according to another embodiment.
12 is a diagram illustrating an example of an image including a flat subject in which two or more different colors are mixed, according to an exemplary embodiment.
13 is a block diagram illustrating a configuration of a shading estimator according to another exemplary embodiment.
14 is a flowchart illustrating an operation method of a block color evaluation value determiner according to an exemplary embodiment.
15A and 15B are flowcharts illustrating an operation method of a block color evaluation value determiner, a block weight coefficient determiner, and a block evaluation value determiner according to an exemplary embodiment.
16 is a diagram illustrating an example of an image processed by the block weight factor determiner 212 according to an embodiment.
17 is a diagram illustrating an example of a histogram generated by a block weight factor determiner according to an embodiment.
18 is a diagram illustrating an example of an image processed by a block weight factor determiner according to an embodiment.
19 is a block diagram illustrating a configuration of a shading estimator according to another embodiment.
20 is a flowchart illustrating an operation method of a group classifying unit and a group evaluation value determining unit according to an exemplary embodiment.
21 is a diagram illustrating an example of a group classified by a group classification unit according to an embodiment.
22 is a flowchart illustrating an operation method of a valid group determiner and an approximate estimator according to an exemplary embodiment.
23 is a diagram illustrating an example of a group processed by a valid group determiner according to an embodiment.
24 is a diagram illustrating an example of a group processed by a valid group determiner according to another embodiment.
25 is a diagram for explaining a method of operating a valid group determiner according to an embodiment.
26 is a graph illustrating an approximate straight line processed by an approximate estimator according to an exemplary embodiment.
27 is a flowchart illustrating a method of operating a variance estimator according to an exemplary embodiment.
28 is a block diagram illustrating a configuration of a shading estimator according to another embodiment.
29 is a flowchart illustrating an operation method of a shading estimator according to another embodiment.
30 is a diagram for explaining a method of determining a valid group according to an embodiment.
31 is a diagram for explaining a method of determining a valid group according to another embodiment.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

In the entire specification, when a part "includes" a certain element, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment.

Referring to FIG. 1 , an image processing apparatus 100 according to an embodiment uses an input unit 300 for receiving a captured image and a plurality of blocks obtained by dividing the captured image, a block indicating image characteristics of each of the plurality of blocks. A data processing unit 310 that determines statistics, determines a shading estimation coefficient to be applied to the captured image using the determined block statistics and the amount of infrared light included in the light source, and corrects shading of the captured image using the determined shading estimation coefficient may include

The data processing unit 310 according to an embodiment determines the flatness of the captured image, and determines a shading estimation coefficient to be applied to the captured image by using at least one of the luminance of the captured image, the color temperature of the light source, and the determined flatness. may include

The data processing unit 310 according to another embodiment determines a block color evaluation value using the determined block statistics, determines a block weight using the determined block color evaluation value, and uses the determined block statistics and the determined block weight. It may include determining a block evaluation value and determining a shading estimation coefficient to be applied to the captured image by using the block evaluation value.

The data processing unit 310 according to another embodiment classifies a plurality of blocks into a plurality of groups, determines a group evaluation value using the determined block statistics and the classified group, and uses the determined block statistics and the determined group evaluation value to determine an effective group and determine a shading estimation coefficient to be applied to the captured image by using the determined effective group.

The data processing unit 310 according to another embodiment estimates an approximate straight line using the determined group evaluation value, determines a shading estimation coefficient to be applied to a captured image using the estimated approximate straight line, and uses the determined block evaluation value estimating the sample variance and determining a shading estimation coefficient to be applied to the captured image using the estimated sample variance.

The data processing unit 310 according to another embodiment estimates a variance using the determined group evaluation value, determines a shading estimation coefficient to be applied to a captured image using the estimated variance, and uses the determined block evaluation value to sample It may include estimating variance and determining a shading estimation coefficient to be applied to the captured image using the estimated sample variance.

2 is a block diagram illustrating a configuration of an image processing apparatus according to another exemplary embodiment.

Referring to FIG. 2 , an image processing apparatus 100 according to an exemplary embodiment includes a lens optical system 102 , an imaging device 104 , an analog front end (AFE) circuit 106 , and an image signal processing circuit ( 108), an image display unit 110, an image recording unit 112, a driver 114, a timing generator (TG) 200, a block statistics unit 118, a control unit 120, a coefficient storage unit 122, etc. can be The image processing apparatus 100 may be configured not only by a digital camera but also by an external device such as a PC, and may perform shading estimation and shading correction of an image in the external device.

The lens optical system 102 according to an embodiment includes a lens, an aperture, a shutter, and the like, and forms an image of a subject on the imaging surface of the imaging device 104 . The imaging device 104 is an image sensor such as a CCD or CMOS, and an infrared cut filter (not shown) is attached to the lens optical system 102 side of the imaging device 104 . The imaging device 104 photoelectrically converts the subject image to obtain a video signal (RGB color signal). The AFE circuit 106 A/D-converts the video signal acquired by the imaging device 104 and subjected to signal processing by the CDS circuit (not shown) into a digital signal.

The video signal processing circuit 108 performs demosaicing processing, edge enhancement processing, white balance (WB) correction processing, shading correction processing, gamma correction processing, and the like on the video signal output by the AFE circuit 106 . The image display unit 110 is a liquid crystal display (LCD) or the like, and displays an image signal that has undergone various processing in the image signal processing circuit 108 . The image recording unit 112 is a memory, and records image signals subjected to various processing in the image signal processing circuit 108 .

The driver 114 drives a lens, an iris, and a shutter of the lens optical system 102 . A timing generator (TG) 116 generates timing for driving the imaging element 104 . The block statistic unit 118 divides the captured image of the image signal that has become a digital signal in the AFE circuit 106 or a partial region of the captured image into a plurality of blocks, and determines a block statistical value for each block. The block statistics unit 118 according to an embodiment may include a block division unit and a statistical value determiner. The block statistics unit 118 may determine a value representing the image characteristics of each block, such as a sum of pixels for each RGB in each block or an average value for pixels for each RGB, as block statistical values. Also, the block statistic unit 118 may determine a value representing the image characteristic of each block other than the statistical value. According to an embodiment, the characteristic values of the blocks (R values, G values, and B values of the blocks) may be summed or average values of R, G, and B values of pixels within the block, but is not limited thereto.

The control unit 120 may control the white balance correction process of the image signal processing circuit 108 according to the block statistical value calculated by the block statistic unit 118 . The controller 120 according to an embodiment may include a shading estimator 124 and a shading corrector 126 .

The shading estimator 124 may estimate shading according to the block statistic value determined by the block statistic unit 118 and the shading estimation coefficient stored by the coefficient storage unit 122 , and may select a shading estimation coefficient suitable for an image. According to an embodiment, the shading estimation coefficient may have a coefficient according to a light source, according to an R value, a G value, and a B value, and according to a block, respectively. The shading estimation coefficient group may mean a set of a plurality of shading estimation coefficients set for each light source.

The shading correction unit 126 may determine a shading correction coefficient corresponding to the shading estimation coefficient determined by the shading estimator 124 from the coefficient storage unit 122 and correct shading of the captured image. According to an exemplary embodiment, the image signal processing circuit 108 may correct shading of the captured image by using the shading correction coefficient determined by the shading correction unit 126 . The shading correction unit 126 may determine a shading correction coefficient according to a light source, an R value, a G value, and a B value, and a shading correction coefficient, respectively, according to a block.

The coefficient storage unit 122 may store a pair of a shading estimation coefficient and a shading correction coefficient for each light source such as sunlight, a light bulb, or a fluorescent lamp. The coefficient storage unit 122 according to an embodiment may include an estimation coefficient storage unit and a correction coefficient storage unit. The shading estimation coefficient and the shading correction coefficient are in a one-to-one pair, but are not limited thereto. The shading estimation coefficient and the shading correction coefficient may be calculated for an external device such as a PC. The coefficient storage unit 122 may store a shading estimation coefficient group including a plurality of shading estimation coefficients corresponding to each light source, and may also store a shading estimation coefficient group including a plurality of shading estimation coefficients having different correction strengths. there is.

The shading estimation coefficient may be obtained using an image of a white chart photographed under each light source. For the white chart, a uniform diffuse reflection surface such as a standard white reflector having a spectral reflectance constant over the target wavelength region and not less than 90% may be used. For example, with respect to a light source such as a light bulb containing a lot of infrared light, a shading estimation coefficient for strongly correcting the R signal in the periphery of the image may be determined. For a light source, such as a fluorescent lamp, which does not separately include infrared light, a shading estimation coefficient for weakly correcting an R signal around an image may be determined. The shading estimation initial coefficient according to an embodiment may be an initial setting (default) shading estimation coefficient for most weakly correcting the R signal of the image periphery.

In addition, the image processing apparatus 100 according to an embodiment includes an image signal processing circuit 108 , a block statistics unit 118 , a control unit 120 , and a coefficient storage unit 122 , but this configuration is not limited to

In addition, each component included in the control unit 120 may be realized by executing a program under the control of an arithmetic device (not shown) included in the control unit 120 , which is a computer, for example. Specifically, the control unit 120 may be realized by loading the program stored in the storage unit (not shown) into the main memory device (not shown) and executing the program under the control of the arithmetic unit. In addition, each component is not limited to realization by software by a program, and can also be implemented by some combination of hardware, firmware, and software, etc.

The above-described program may be stored and supplied to a computer using various types of non-transitory computer readable media. Non-transitory computer-readable media includes various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives, etc.), magneto-optical recording media (eg, optical magnetic disks, etc.), CD-ROM (Read Only Memory, etc.) ), CD-R, CDR/W, semiconductor memory (eg, mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)), and the like.

Also, the program may be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium may supply a program to the computer through a wired communication path such as an electric wire and optical fiber, or a wireless communication path.

3 is a block diagram illustrating a configuration of a shading estimator according to an exemplary embodiment.

Referring to FIG. 3 , the shading estimator 124 may include a flat determiner 201 , a shading estimation coefficient determiner 202 , and the like, but is not limited thereto. For example, the shading estimator 124 may include means for obtaining the luminance (or brightness, brightness, etc.) of the subject or means for obtaining the color temperature of the subject.

The shading estimator 124 may determine whether the photographed subject is flat. According to an embodiment, the flat determiner 201 may determine whether the photographed subject is flat. The shading estimator 124 selects a shading estimation coefficient suitable for an image from a group of shading estimation coefficients according to whether the subject is flat or according to imaging conditions such as luminance (Bv) or light source color temperature (R Color Gain) of the subject. can According to an embodiment, the shading estimation coefficient determining unit 202 may determine the shading estimation coefficient according to whether the object is flat or according to imaging conditions such as the luminance (Bv) of the object or the color temperature of the light source (R Color Gain). A shading estimation coefficient suitable for an image may be selected from the group.

4 is a flowchart illustrating a method of operating an image processing apparatus according to an exemplary embodiment.

In S500, the image processing apparatus receives the captured image.

In S510, the image processing apparatus determines block statistics using a plurality of blocks obtained by dividing the captured image. According to an embodiment, the shading estimator 124 may determine block statistics using a plurality of blocks obtained by dividing a captured image.

In S520, the image processing apparatus determines a shading estimation coefficient using block statistics and the amount of infrared light included in the light source. According to an embodiment, the shading estimation unit 124 may determine the shading estimation coefficient by using block statistics and the amount of infrared light included in the light source.

In S530, the image processing apparatus corrects shading of the captured image by using the shading estimation coefficient. According to an embodiment, the shading estimator 124 may correct shading of the captured image by using a shading estimation coefficient.

5 is a flowchart illustrating a method of operating a shading estimator according to an exemplary embodiment.

The shading estimator 124 may include the operation of the flat determiner 201 in FIG. 5 . Also, the shading estimator 124 may include the operation of the shading estimation coefficient determiner 202 of FIG. 5 .

In step S1, the flat determining unit 201 determines whether the photographed subject is flat. The flat determiner 201 determines whether the photographed subject is flat by determining the degree of change in the pixel value by the subject in a partial area except for the change in the pixel value due to shading that has previously occurred in the image. Also, the flat determiner 201 may determine the luminance of the subject or the color temperature of the subject.

As a result of the determination in step S1, if the photographed subject is flat, in step S2, the shading estimation coefficient determiner 202 determines a shading estimation coefficient having the smallest evaluation value Ed from the first shading estimation coefficient group. According to an embodiment, the evaluation value Ed may be obtained by Equation 13 to be described later.

Table 1 below shows an example of a first shading estimation coefficient group including 6 types of shading estimation coefficients (or shading estimation tables) respectively corresponding to 6 types of light sources according to an embodiment.

Referring to Table 1, as the table number increases, the correction intensity of R becomes a strong shading estimation coefficient corresponding to a light source with a large amount of infrared light.

If it is determined in step S1 that the photographed subject is not flat, the shading estimation coefficient determiner 202 selects a second shading estimation coefficient group from the first shading estimation coefficient group in step S3. Specifically, the shading estimation coefficient determiner 202 may select at least one shading estimation coefficient from among the first shading estimation coefficient group to be the second shading estimation coefficient group.

In step S4, the shading estimation coefficient determining unit 202 selects a shading estimation coefficient having the smallest evaluation value Ed from the second shading estimation coefficient group.

In addition, when the shading estimation coefficient determining unit 202 selects the second shading estimation coefficient group, the shading estimation coefficient group may A suitable shading estimation coefficient can be selected.

Table 2 below shows an example of selecting the second shading estimation coefficient group according to the luminance (Bv) of the subject or the color temperature of the light source (R Color Gain) according to an exemplary embodiment.

Referring to Table 2, the second shading estimation coefficient group includes condition 1 and condition 2 for a preset light bulb color threshold (R Gain Limit Threshold), indoor threshold value (Indoor Bv Threshold) and outdoor threshold value (Outdoor Bv Threshold). is selected by applying According to an embodiment, when it is determined that the captured image is flat (ie, a flat subject) or when it is determined that the captured image is an indoor image (ie, Bv < indoor threshold) according to the subject luminance (Bv), the color temperature of the light source A second shading estimation coefficient group is selected from the first shading estimation coefficient group according to (R Color Gain). On the other hand, when it is determined that the captured image is not flat (ie, a non-flat object), a second shading estimation coefficient group is selected from the first shading estimation coefficient group according to the luminance Bv of the object.

Specifically, in the case of a flat subject or subject luminance < indoor threshold in condition 1, and light source color temperature < bulb color threshold in condition 2, shading estimation coefficients of all light sources among the first shading estimation coefficient group in Table 1 (i.e., , Table numbers 0 to 5) are selected as the second shading estimation coefficient group. In addition, when a flat object or subject luminance < indoor threshold value in condition 1, and light source color temperature ≥ bulb color threshold in condition 2, shading estimation coefficients other than the light bulb color system in Table 1 (i.e., Table Nos. 0 to 3) ) is selected as the second shading estimation coefficient group.

In addition, when the subject is not flat under condition 1 and indoor threshold ≤ subject luminance ≤ outdoor threshold under condition 2, the shading estimation coefficients of the outdoor light sources in Table 1 (that is, Table Nos. 1 to 3) are the second It is selected as a group of shading estimation coefficients. In addition, in the case of a non-flat subject under condition 1 and subject brightness > outdoor threshold under condition 2, the shading estimation coefficients of the outdoor light sources in Table 1 (i.e., Table Nos. 1 and 2) are the second shading estimation coefficient group. is chosen

6 is a diagram illustrating an example of an image for explaining an operation of a flat determiner according to an exemplary embodiment.

The shading estimator 124 may include the operation of the flat determiner 201 in FIG. 6 .

Referring to FIG. 6 , the flat determiner 201 according to an exemplary embodiment may determine the image flatness by adding up differences in characteristic values of blocks that are continuous from a peripheral portion to a central portion of a captured image.

The flat determiner 201 according to an embodiment may determine whether the subject is flat by using a G value of a block positioned on a diagonal of the image. Referring to FIG. 6 , the image is divided into 4 lines so that a block on a diagonal of the image becomes a block to be determined. For example, from the upper left block 301 to the center block 302 on the L2C line extending from the upper left to the center of the image, and the lower right block from the center block 303 on the C2R line extending from the center to the lower right of the image. Up to 304, from the upper right block 305 to the center block 306 on the R2C line extending from the upper right to the center of the image, left from the center block 307 on the C2L line extending from the center to the lower left of the image Up to the lower block 308 can be used as a judgment target block. Accordingly, the line dividing the image in FIG. 6 is a line extending from the edge of the image to the center and a line extending from the center diagonally, but is not limited thereto.

The flat determiner 201 according to an exemplary embodiment may determine whether the subject is flat by using Equations 1 to 3.

upG and dwG according to an embodiment may be expressed as in Equation (1).

In Equation 1, deltaG is a difference value between Gi+1, which is the G value of the i+1th block, and Gi, which is the G value of the i-th block. upG is the sum of deltaG when deltaG is greater than 0, and dwG is the sum of deltaG when deltaG is less than 0.

On the other hand, Equation 1 can be used also for from the upper right block 305 to the center block 306 and from the center block 307 to the lower left block 308 .

Also, the flat determiner 201 may determine the line evaluation value El1 between the L2C line and the C2R line and the line evaluation value El2 between the R2C line and the C2L line by using the result of Equation 1 .

E11 and E12 according to an embodiment may be expressed as Equation (2).

In Equation 2, L2C is the line from the upper left block 301 to the center block 302, L2C_upG is the sum of deltaG when deltaG is greater than 0, and L2C_dwG is the sum of deltaG when deltaG is less than 0. In addition, C2R is the line from the central block 303 to the lower right block 304, C2R_upG is the sum of deltaG when deltaG is greater than 0, and C2R_dwG is the sum of deltaG when deltaG is less than 0. In addition, R2C is the line from the upper right block 305 to the center block 306, R2C_upG is the sum of deltaG when deltaG is greater than 0, and R2C_dwG is the sum of deltaG when deltaG is less than 0. In addition, C2L is the line from the center block 307 to the lower left block 308, C2L_upG is the sum of deltaG when deltaG is greater than 0, and C2L_dwG is the sum of deltaG when deltaG is less than 0.

Also, the flat determination unit 201 may determine the flat evaluation value Ef by using the result of Equation (2).

The flat evaluation value Ef according to an embodiment may be expressed as Equation (3).

The flat determination unit 201 uses the result of Equation 3 to determine the subject as a flat subject when the flat evaluation value Ef is less than a predetermined threshold value (ie, a flat threshold value), and the flat evaluation value ( When Ef) is greater than or equal to a predetermined threshold (ie, a flat threshold), the subject is determined as a non-flat subject.

7 is a diagram illustrating an example of a pixel value for explaining an operation of a flat determiner according to an exemplary embodiment.

Referring to FIG. 7 , with respect to the G value of the block on the diagonal of the image, from the upper left block 301 to the center block 302, deltaG is greater than 0, while from the center block 303 to the lower right block 304 ), deltaG is less than 0. 7 and Equation 1, the sum of upG and dwG will be close to zero. That is, if the subject is flat, the sum of upG and dwG will be close to zero.

8 is a diagram illustrating an example of an image for explaining an operation of a flat determiner according to an embodiment.

The shading estimator 124 may include the operation of the flat determiner 201 in FIG. 8 .

Referring to FIG. 8 , in the case of FIG. 8A , since the flat evaluation value Ef is less than the flat threshold value, the flat determination unit 201 determines that the subject is flat, and in the case of FIG. 8B , the flat evaluation value Ef Since it is equal to or greater than this flat threshold, the flat determination unit 201 determines that the subject is not flat.

9 is a flowchart illustrating an operation method of a shading estimation coefficient determiner according to an exemplary embodiment.

The shading estimator 124 may include the operation of the shading estimation coefficient determiner 202 of FIG. 9 .

The shading estimation coefficient determiner 202 according to an embodiment selects a shading estimation coefficient from the first to second shading estimation coefficient groups, but is not limited thereto.

In step S010, the shading estimation coefficient determining unit 202 determines the R value, G value, and B value of the block for all blocks by using the block statistical value obtained from the image that the block statistic unit 118 is subjected to shading correction. The average values (Rav, Gav and Bav) are determined respectively.

In step S020, the shading estimation coefficient determining unit 202 determines the normalization coefficients (Rg, Bg, and Gg) of the R value, the G value, and the B value, respectively.

The normalization coefficients (Rg, Bg, and Gg) of the R value, the G value, and the B value according to an embodiment may be expressed as in Equation (4).

In Equation 4, the normalization coefficients (Rg, Bg, and Gg) of the R value, the G value, and the B value are coefficients for matching the RGB balance of the entire image, and are a gain for normalizing the image.

In step S030, the shading estimation coefficient determiner 202 calculates Rc[t](x,y), Gc[t](x,y), and Bc[t](x,y).

Rc[t](x,y), Gc[t](x,y), and Bc[t](x,y) according to an embodiment may be expressed as Equation 5.

In Equation 5, Rc[t](x,y), Gc[t](x,y), and Bc[t](x,y) are block statistics R(x) of R values, G values, and B values. ,y), shading estimation coefficients of R, G, and B values for G(x,y) and B(x,y) (Cr[t](x,y), Cg[t](x,y) ) and Cb[t](x,y)) and the normalization coefficients (Rg, Gg, and Bg) of the R, G, and B values, respectively.

According to an embodiment, the shading estimation coefficient may use an initial shading estimation coefficient in S030. The shading estimation initial coefficient according to an embodiment may be an initial setting (default) shading estimation coefficient for most weakly correcting the R signal of the image periphery. As the shading estimation coefficients fit the image, Rc[t](x,y), Gc[t](x,y), Bc[t](x,y) are the same and close to 1.0, while Rc[t] If (x,y), Gc[t](x,y), and Bc[t](x,y) are displayed as an image, the image may be a single gray color.

In step S040, the shading estimation coefficient determiner 202 calculates the R signal ratio Rr[t](x,y) representing the block characteristic value.

The R signal ratio (Rr[t](x,y)) according to an embodiment may be expressed as Equation (6).

Meanwhile, according to an embodiment, the denominator in Equation 6 may be the sum of Rc[t](x,y), Gc[t](x,y), and Bc[t](x,y). In addition, Rr[t](x,y) may add the R signal ratio or use the B signal ratio instead of the R signal ratio.

Referring to Equation 6, since the R signal ratio is obtained for the RGB image to which the standardization coefficient is applied, the more the shading estimation coefficient is suitable for the image, the more R The signal ratio is close to 1.0. On the other hand, as the shading estimation coefficient is not suitable for the image, the R signal ratio may be a value farther from 1.0.

In step S060, the shading estimation coefficient determiner 202 calculates the difference Rd(x, y) between the R signal ratios of the current block and its neighboring blocks using the spatial filter m, and uses this to calculate the block weight Wb( x, y)) is calculated.

The spatial filter m according to an embodiment may be expressed as in Equation (7).

Rd(x, y) according to an embodiment may be expressed as Equation (8).

The block weight Wb(x, y) according to an embodiment may be calculated as in Equation 9.

In Equation (9), Rdmax is the maximum value of the difference (Rd(x, y)) of the R signal ratios. Referring to Equation 9, when the complexity of the photographed subject and the complexity of a part of the image are reflected in the block weight, when the image change is small and smooth, that is, when the similarity between blocks is large, the block weight increases and the image change When it is large, that is, when the similarity between blocks is small, the block weight may be small. In addition, Wb(x, y) may be added to the R signal ratio or a B signal ratio may be used instead of the R signal ratio.

In step S080, the shading estimation coefficient determiner 202 calculates the R signal ratio GpRr[j] representing the group characteristic value.

An R signal ratio (GpRr[j]) representing a group characteristic value according to an embodiment may be expressed as in Equation (10).

The R signal ratio (GpRr[j]) representing the group characteristic value may be determined by weighted averaging the R signal ratios (Rr(i)) of the blocks included in the group using Wb(x, y). Referring to Equation 10, by using the block weight (Wb(x, y)) according to the degree of similarity between the current block and the neighboring blocks, the characteristic value of a block with a small change in image or a region constituting the block is more important. Group characteristics can be calculated. For example, in an image in which green leaves, red flowers, sky, or buildings are photographed, the characteristic value of a block or area having the sky or a building may be more important than the characteristic value of a block or area adjacent to the green leaf and red flower.

In step S100, the shading estimation coefficient determiner 202 selects a reference group. From the group in the center of the image, the group is sequentially searched toward the outer group, and one group is determined as the reference group.

The reference group according to an embodiment may be expressed as Equation (11).

According to an embodiment, a group satisfying Equation 11 first among R signal ratios GpRr[j] representing a group characteristic value may be selected as a reference group. The reference group thresholds Th1 and Th2 may be set to, for example, 0.9 and 1.1 so that 1.0 is included therebetween. Since the reference group is selected after the normalization coefficient is applied, the reference group is not limited to the case of an achromatic object and may be selected even in the case of a chromatic object.

In step S110, the shading estimation coefficient determining unit 202 determines whether a reference group exists.

As a result of the determination in step S110, if a reference group exists, the shading estimation coefficient determining unit 202 calculates a group evaluation value (Dg[j+k]) in step S120. According to an embodiment, the group evaluation value may be calculated for each group outside the reference group selected in step S100. For example, when group 1 is selected as the reference group in FIG. 10 , the shading estimation coefficient determiner 202 calculates a group evaluation value for each group of groups 2 to 5, which are groups outside of group 1 .

The group evaluation value (Dg[j+k]) according to an embodiment may be calculated as in Equation (12).

Referring to Equation 12, each group evaluation value (Dg[j+k]) is the R signal ratio (GpRr[j]) indicating the group characteristic value of the reference group and the R signal ratio indicating the group characteristic value of the outer group (GpRr[j + k]) ) may be determined by multiplying the weight (Wg[j+k]) of the outer group by a difference.

In addition, according to an exemplary embodiment, each group evaluation value (Dg[j+k]) does not multiply by the preset weight of the outer group (Wg[j+k]), and the R signal ratio (GpRr[ j]) and the R signal ratio (GpRr[j+k]) representing the group characteristic value of the outer group may be determined only by the difference.

Table 3 below shows an example of the weight (Wg) of a group according to an embodiment.

The weight (Wg) of the group at the periphery of the image is large. Accordingly, it is possible to evaluate with higher precision whether the shading estimation coefficient is suitable for shading correction in the periphery of the image.

In step S140, the shading estimation coefficient determining unit 202 takes the sum of the absolute values of the group evaluation values (Dg[j+k]) and determines whether the shading estimation coefficient used in step S030 is appropriate. Ed) is calculated.

The evaluation value Ed of the shading estimation coefficient according to an embodiment may be expressed as Equation (13).

The evaluation value (Ed) of the shading estimation coefficient is an index for determining the suitability of the shading estimation coefficient used in step S030 for the image to be shading correction as extracting the characteristic of the R signal ratio that is attenuated from the center of the image to the periphery. can be

Also, according to an exemplary embodiment, the evaluation value Ed of the shading estimation coefficient may be obtained as a statistical value of the group evaluation value different from Equation 13, such as the variance of the group evaluation value.

In step S150, the shading estimation coefficient determiner 202 evaluates the shading estimation coefficients for all shading estimation coefficients (that is, shading estimation coefficients included in the first shading estimation coefficient group or the second shading estimation coefficient group) (Ed) to determine whether the

As a result of the determination in step S150, if evaluation values (Ed) of shading estimation coefficients for all shading estimation coefficients are not calculated, the flow returns to step S030 and shading estimation coefficients for which evaluation values (Ed) of shading estimation coefficients are not calculated For , the processing from step S030 to step S150 is repeated.

According to an embodiment, when the evaluation value Ed of the shading estimation coefficient other than the initial shading estimation coefficient is calculated, the selection of the reference group in step S100 and whether the reference group in step S110 exist are not performed. Then, the reference group selected when calculating the evaluation value Ed of the shading estimation initial coefficient can be selected as the reference group when calculating the evaluation value Ed of the shading estimation coefficient other than the shading estimation initial coefficient.

In step S160 , the shading estimation coefficient determiner 202 selects a shading estimation coefficient that has the smallest evaluation value Ed of the shading estimation coefficient among all shading estimation coefficients including the shading estimation initial coefficient.

In addition, when there is no reference group as a result of the determination in step S110 , the shading estimation coefficient determiner 202 may select an initial shading estimation coefficient as the shading estimation coefficient.

According to an embodiment, the shading correction coefficient may use a different value as well as the same value as the shading estimation coefficient.

Also, according to an embodiment, in order to evaluate the shading estimation coefficient, the shading estimation coefficient may be evaluated using not only the RGB color space but also another color space such as HSV. Specifically, when calculating the block weight, which is the degree of similarity between blocks, the RGB values may be HSV-converted and the block weight may be calculated according to changes in luminance or chroma between blocks.

In addition, according to an embodiment, the shading estimation coefficient determining unit 202 performs from the reference group selection process to determining the evaluation value of the shading estimation coefficient (steps S100 to S150), but the shading estimation coefficient determination unit 202 may divide an image to be subjected to shading correction into a plurality of regions, and perform from a reference group selection process to determining an evaluation value of a shading estimation coefficient for each region.

10 is a diagram illustrating an example of a group classified by a shading estimation coefficient determiner according to an embodiment.

The shading estimator 124 may include the operation of the shading estimation coefficient determiner 202 of FIG. 10 .

The shading estimation coefficient determining unit 202 classifies each block obtained by the block statistic unit 118 by dividing the image region in a grid pattern into a plurality of groups according to a distance from the image center. Referring to FIG. 10 , from the center of the image 400 to the periphery of the image, each block of the image is classified into six groups, from group 0 to group 5. Then, in order to select a reference group, a search is started from group 0 shown in FIG. 10, and up to group 3, which is considered not to be affected by shading, can be searched.

11 is a diagram illustrating an example of a group classified by a shading estimation coefficient determiner according to another embodiment.

The shading estimator 124 may include the operation of the shading estimation coefficient determiner 202 of FIG. 11 .

The shading estimation coefficient determiner 202 divides the image 401 into four regions I to IV, and classifies each block in the regions I to IV into a plurality of groups according to a distance from the center of the image. Referring to FIG. 11 , in the region I, each block is classified into five groups, from group I-1 to group I-5. Then, the shading estimation coefficient determining unit 202 performs from the reference group selection process to the determination of the evaluation value of the shading estimation coefficient in each of the regions I to IV. Accordingly, an evaluation value of each shading estimation coefficient may be determined for each region I to IV, and a shading estimation coefficient suitable for a subject may be selected for each region I to IV.

12 is a diagram illustrating an example of an image including a flat subject in which two or more different colors are mixed according to an exemplary embodiment;

12 , an image 410 includes a flat blue subject on a gray background. In this case, the shading estimator of FIG. 3 may not select an appropriate shading estimation coefficient for the image 410 of FIG. 12 . For example, in an uncomplicated monotonous scene in which flat subjects of different colors such as blue and red are mixed, the block characteristic similarity of neighboring blocks increases, so that the subject is not determined as a complex subject. However, in this case, since flat portions of different colors have the same weight, a shading estimation error may occur under the influence of the subject color.

13 is a block diagram illustrating a configuration of a shading estimator according to another exemplary embodiment.

According to another embodiment, the shading estimator may reduce an error in selecting a shading estimation coefficient in an image in which a plurality of colors that are not locally complex and monotonous are mixed. Specifically, the shading estimator may estimate a shading correction coefficient suitable for an image by extracting and evaluating a color block occupying a large area in the image.

Referring to FIG. 13 , the shading estimator 124 includes a block color evaluation value determiner 211 , a block weight coefficient determiner 212 , a block evaluation value determiner 213 , and a shading estimation coefficient determiner 214 , etc. may include, but is not limited thereto. For example, the shading estimator 124 may use the block color evaluation value determiner 211 and the block weight coefficient determiner 212 as one block.

The shading estimator 124 may include the operation of the block color evaluation value determiner 211 . Also, the shading estimator 124 may include the operation of the block weight factor determiner 212 . Also, the shading estimator 124 may include the operation of the block evaluation value determiner 213 . Also, the shading estimator 124 may include the operation of the shading estimation coefficient determiner 214 .

The block color evaluation value determining unit 211 determines the block color evaluation value Hb by using the block statistical value and an average gain. The block weight coefficient determining unit 212 calculates a histogram weight from the block color evaluation value Hb, and determines the block weight Wb using the histogram weight and the G-level weight. The block evaluation value determining unit 213 determines the block evaluation value Eb by using the block statistical value and the block weight Wb. The shading estimation coefficient determining unit 214 determines a group evaluation value by using the block evaluation value Eb, and determines a shading estimation coefficient according to a change amount of the group evaluation value.

14 is a flowchart illustrating an operation method of a block color evaluation value determiner according to an exemplary embodiment.

The shading estimator 124 may include the operation of the block color evaluation value determiner 211 of FIG. 14 .

The block color evaluation value determiner 211 according to an embodiment may determine a color block occupying a large area in the image.

In step S201, the block color evaluation value determining unit 211 multiplies the shading estimation initial coefficients by the block statistical values (Ri, Gi, and Bi) of the R, G, and B values before shading correction, and obtains the R values, G values and B values. Determine the block statistics of values: Rn(N, M), Gn(N, M), and Bn(N, M). In step S201, the block color evaluation value determiner 211 may reduce shading by multiplying the block statistical value before shading correction by the default shading estimation coefficient (ie, shading estimation initial coefficient) having the weakest correction intensity.

Block statistical values Rn(N, M), Gn(N, M), Bn(N, M)) of the R value, the G value, and the B value according to an embodiment may be expressed as Equation 14.

In Equation 14, Nr, Ng, and Nb are shading estimation initial coefficients of R, G, and B values, respectively.

In step S202, the block color evaluation value determining unit 211 calculates the block statistical values (Rn(N, M), Gn(N, M), Bn(N, M)) of all blocks for each of R, G and B. add

In steps S204 to S206, the block color evaluation value determining unit 211 determines the normalization gain (AverageGainR) of the R value, and in steps S207 to S209, the block color evaluation value determining unit 211 determines the normalization gain of the B value. (AverageGainB) is determined. According to an exemplary embodiment, the block color evaluation value determiner 211 may determine the normalization gain of the G value to be 1.0 and the normalization gain of the R value (AverageGainR) and the normalization gain of the B value (AverageGainB).

In steps S204 to S206, the block color evaluation value determining unit 211 determines a normalization gain (AverageGainR) of the R value.

In step S204, the block color evaluation value determining unit 211 determines whether the sum of the R values and the block statistical values Rn(N, M) is zero.

As a result of the determination in step S204, if the sum of the block statistical values Rn(N, M) of R values is not 0, in step S205, the block color evaluation value determining unit 211 determines the normalization gain (AverageGainR) of the R values. It is decided by the sum of G/R sum values. Meanwhile, in order to prevent overflow, the normalization gain (AverageGainR) of the R value is limited to the upper limit of the normalization gain (AverageGainLimitR) of the R value.

As a result of the determination in step S204, if the sum of the block statistical values Rn(N, M) of the R values is 0, in step S206, the block color evaluation value determining unit 211 sets the R value normalization gain (AverageGainR) to 1.0 to decide

In steps S207 to S209, the block color evaluation value determining unit 211 determines a normalization gain (AverageGainB) of the B value.

In step S207, the block color evaluation value determining unit 211 determines whether the sum of the B values and the block statistical values Bn(N, M) is zero.

As a result of the determination in step S207, if the sum of the block statistical values (Bn(N, M)) of the B values is not 0, in step S208, the block color evaluation value determining unit 211 determines the normalization gain (AverageGainb) of the B values. It is decided by the sum of G/sum of B. On the other hand, to prevent overflow, the normalization gain of the B value (AverageGainB) is limited to the upper limit of the normalization gain of the B value (AverageGainLimitB).

As a result of the determination in step S207, if the sum of the block statistical values (Bn(N, M)) of the B values is 0, in step S209, the block color evaluation value determining unit 211 sets the B value normalization gain (AverageGainB) to 1.0 to decide

According to an embodiment, the normalization gain of the R value (AverageGainR) and the normalization gain of the B value (AverageGainB) may be expressed as in Equation 15.

15A and 15B are flowcharts illustrating an operation method of a block color evaluation value determiner, a block weight coefficient determiner, and a block evaluation value determiner according to an exemplary embodiment.

The shading estimator 124 may include the operation of the block color evaluation value determiner 211 in FIGS. 15A and 15B . Also, the shading estimator 124 may include the operation of the block weight factor determiner 212 in FIGS. 15A and 15B . Also, the shading estimator 124 may include the operation of the block evaluation value determiner 213 in FIGS. 15A and 15B .

In step S210, the block weight factor determining unit 212 determines the G-level weight. The G-level weight according to an embodiment may use a block statistical value (Gn(N, M)) of the G-value. The block weight coefficient determining unit 212 determines the G-level weight of the block as 1 if the block statistical value (Gn(N, M)) of the G-value is within a range of a preset upper limit and a lower limit, and determines the G-level weight of the block as 1 If the value of , the G-level weight of the block can be determined to be 0.

16 is a diagram illustrating an example of an image processed by the block weight factor determiner 212 according to an embodiment.

The shading estimator 124 may include the operation of the block weight coefficient determiner 212 in FIG. 16 .

Referring to FIG. 16 , the block weight factor determining unit 212 may exclude a block near a dark portion or saturation from an evaluation target. Also, when one of the R value, the G value, and the B value of an arbitrary block is out of a preset range, the block weight factor determiner 212 may exclude the block from the evaluation target block.

In step S211, the block color evaluation value determining unit 211 determines the block color evaluation value Hb(N, M).

The block color evaluation value Hb(N, M) according to an embodiment may be expressed as Equation (16). A block color evaluation value (Hb(N, M)) can be obtained for all blocks.

Referring to Equation 16, the block color evaluation value Hb (N , M)) can be determined. In addition to the R value, the block color evaluation value (Hb(N, M)) can also be determined for the G value and the B value. According to an embodiment, the block statistical value (Bn(N, M)) obtained in Equation 14 is referred to as a first block statistical value, and a value obtained by multiplying the first block statistical value by the normalization gain (AverageGainB) of the B value If it is the second block statistical value Bng(N,M), the block color evaluation value Hb(N, M) may be determined using the first block statistical value and the second block statistical value. According to an embodiment, the block color evaluation value determining unit 211 determines the ratio of the second block statistical value to the R value, the ratio of the second block statistical value to the G value, and the B value of the second block statistical value. The block color evaluation value Hb(N, M) may be determined using at least one of the ratios. The ratio to the R value may be three times the ratio of R to all RGB (3 R/(R+G+B)), or it may be the ratio of R to all RGB (R/(R+G+B)).

In step S212, the block weight coefficient determining unit 212 determines a histogram of the block color evaluation value Hb.

In steps S213 to S218, the block weight coefficient determiner 212 may determine a histogram weight and exclude blocks separated from the mode.

In step S213, the block weight factor determining unit 212 determines the mode of the histogram of Hb.

In step S214, the block weight factor determining unit 212 determines whether the block color evaluation value Hb for each block is within the range of the histogram mode of Hb.

As a result of the determination in step S214, if the block color evaluation value Hb is within the range of the histogram mode of Hb, the block weight coefficient determining unit 212 determines the histogram weight of Hb to be 1 in step S215.

In step S216, the block weight coefficient determining unit 212 determines the block weight Bw by multiplying the G-level weight and the histogram weight of Hb.

If it is determined in step S214 that the block color evaluation value Hb is out of the range of the histogram mode of Hb, the block weight coefficient determining unit 212 determines the histogram weight of Hb to be 0 in step S217.

17 is a diagram illustrating an example of a histogram generated by a block weight factor determiner according to an embodiment.

17A is a diagram illustrating an example of a histogram generated by a block weight factor determining unit. Referring to FIG. 17A , 0.65 to 1.35 were divided by 0.05 intervals. Fig. 17B is a diagram showing an example in which the operations of steps S215 and S217 are shown in an image.

18 is a diagram illustrating an example of an image processed by a block weight factor determiner according to an embodiment.

The shading estimator 124 may include the operation of the block weight coefficient determiner 212 of FIG. 18 .

18 and step 216 , the block weight factor determining unit 212 determines the block weight Bw by multiplying the G-level weight and the histogram weight of Hb.

In step S219, the block evaluation value determining unit 213 multiplies the block statistical values (Ri, Gi, and Bi) of the R, G, and B values before shading correction by the shading estimation coefficient, which is one of the shading estimation candidates, to obtain the R value, G Determine the block statistics (Rc, Gc and Bc) of the values and B values.

Block statistical values Rc, Gc, and Bc according to an embodiment may be expressed as in Equation 17.

In Equation 17, Cr[t] and Cg[t] are shading estimation coefficients that are one of shading estimation candidates, and t is a table number.

In step S221, the block evaluation value determining unit 213 determines the block evaluation value Eb(N, M) by multiplying the block statistical values Rc, Gc and Bc by the block weight Bw(N,M).

The block evaluation value Eb(N, M) according to an embodiment may be expressed as Equation (18).

After determining the block evaluation values (Eb) of all blocks, the shading estimation coefficient determining unit 214 determines that, for a group in which an image region is divided according to a distance from the image center, the block evaluation value (Eb) in the group is 0 It is determined as the group evaluation value of the relevant group by averaging the exclusions. In addition, a change amount of the group evaluation value from the central group to the peripheral group may be determined as the evaluation value of the shading estimation coefficient.

19 is a block diagram illustrating a configuration of a shading estimator according to another embodiment.

According to another embodiment, the shading estimator determines the validity of a block with respect to a block statistical value to which a shading estimation coefficient for each light source is applied, determines the validity of a group consisting of a set of blocks, a judging unit that determines the continuity of the valid group toward the group, interpolates and validates the invalid group according to the group continuity judgment result, extracts a group of subject areas of a similar color, and extracts a group of the valid group from the image center of the valid group By selecting a shading correction coefficient suitable for correction of the captured image according to the slope of the approximate straight line of the R ratio over the peripheral portion, it is possible to reduce misjudgment by the subject.

Referring to FIG. 19 , the shading estimator 124 includes a block color evaluation value determiner 211 , a block weight coefficient determiner 212 , and a block evaluation value determiner included in the shading estimator 124 of FIG. 13 . In addition to 213 and shading estimation coefficient determining unit 214 , group classification unit 221 , group evaluation value determination unit 222 , effective group determination unit 223 , approximate estimation unit (or approximate straight line estimation unit, straight line) It further includes an approximation unit 224 and a variance estimation unit 225, but is not limited thereto. For example, according to the approximation of the approximate estimator 224, the shading estimation coefficient determiner 214 calculates the shading estimation coefficient. You can also choose

The shading estimator 124 may include the operation of the group classifier 221 . Also, the shading estimator 124 may include the operation of the group evaluation value determiner 222 . Also, the shading estimator 124 may include the operation of the effective group determiner 223 . Also, the shading estimator 124 may include the operation of the approximate estimator 224 . Also, the shading estimator 124 may include the operation of the variance estimator 225 .

The group classification unit 221 classifies a plurality of blocks obtained by dividing the captured image into a plurality of groups according to a distance from the center of the image. The group evaluation value determining unit 222 determines the group evaluation value according to the block statistical value. In addition, the group evaluation value determining unit 222 may include a valid block determining unit that determines whether a block is valid or invalid according to the block statistical value, and determines the group evaluation value according to the color-specific sum of the statistical values for the valid blocks in the group. can decide The valid group determining unit 223 determines whether the group is valid or invalid according to the group evaluation value or the number of valid blocks in the group. The approximation estimator 224 determines, according to the group evaluation value, an approximation equation for approximating the correction by the shading estimation coefficient, and selects a shading estimation coefficient according to the approximation equation. Also, the approximation estimator 224 may linearly approximate the color R ratio for the effective group from the central portion to the peripheral portion of the captured image. The variance estimation unit 225 determines the sample variance of the block evaluation values, and selects a shading estimation coefficient having a minimum variance value.

According to an exemplary embodiment, the block color evaluation value determiner 211 of FIG. 19 , the block weight factor determiner 212 , and the block evaluation value determiner 213 of FIG. 19 each include the block color evaluation value determiner 211 of FIG. 13 . ), the block weight coefficient determiner 212 and the block evaluation value determiner 213 may operate in the same manner.

20 is a flowchart illustrating an operation method of a group classifying unit and a group evaluation value determining unit according to an exemplary embodiment.

The shading estimator 124 may include the operation of the group classifier 221 in FIG. 20 . Also, the shading estimator 124 may include the operation of the group evaluation value determiner 222 of FIG. 20 .

In step S301, the group classification unit 221 divides the plurality of blocks obtained by dividing the captured image into quadrants Q[0] to Q[3], and groups G[0] to G according to the distance from the center. Classified as [g].

21 is a diagram illustrating an example of a group classified by a group classification unit according to an embodiment. The shading estimator 124 may include the operation of the group classifier 221 in FIG. 21 .

In step S302, the group evaluation value determiner 222 multiplies the block detection data by the shading estimation coefficient (Small Table) of the estimation candidate. According to an embodiment, the group evaluation value determiner 222 multiplies the block statistical values (Ri, Gi, and Bi) of the R, G, and B values before shading correction by the shading estimation coefficient, which is one of the shading estimation candidates. It is possible to determine the block statistics (Rc, Gc, and Bc) of the R, G and B values.

In step S303, the group evaluation value determining unit 222 determines the block evaluation value Eb. The group evaluation value determiner 222 may determine the block evaluation value Eb by using Equations 17 and 18.

In step S304, the group evaluation value determining unit 222 determines the group sum value (Eg[g]).

The group sum value Eg[g] according to an embodiment may be expressed as Equation 19.

According to an embodiment, the group evaluation value determiner 222 may use the group sum value (Eg) as the total of the block statistical values of the valid blocks among the block statistical values to which the shading estimation coefficient is sequentially applied, included in the group. .

In step S305, the group evaluation value determining unit 222 determines the number of valid blocks (Nb[g]).

The number of effective blocks (Nb[g]) according to an embodiment may be expressed as in Equation 20.

Referring to Equation 20, the group evaluation value determining unit 222 adds up the block weights Bw(N, M) for each group G[0] to G[g], and the number of effective blocks Nb[ g]) can be determined.

The group evaluation value determining unit 222 performs steps S301 to S305 for all candidates of the shading estimation coefficients.

22A and 22B are flowcharts illustrating operating methods of a valid group determiner and an approximate estimator according to an exemplary embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIGS. 22A and 22B . In addition, the shading estimator 124 may include the operation of the approximate estimator 224 in FIGS. 22A and 22B .

In step S310, the effective group determination unit 223 determines the number of effective blocks (Nb[g]) for each group (G[0] to G[g]). On the other hand, the effective group determination unit 223 sets the number of valid blocks Nb[g] to zero when the number of valid blocks Nb[g] is smaller than a predetermined threshold (that is, the threshold for the number of valid blocks). .

In step S311, the effective group determination unit 223 determines the average value Ag[g] of the block evaluation values Eb for each group G[0] to G[g].

The average value Ag[g] of the block evaluation value Eb according to an embodiment may be expressed as Equation 21.

The effective group determination unit 223 sets the average value Ag[g] of the block evaluation values Eb to 0 when the number of valid blocks Nb[g] is smaller than the threshold value of the number of valid blocks in the group. According to an embodiment, in the case of a group in which the number of effective blocks (Nb[g]) is less than the threshold value of the number of effective blocks, the shading estimator may exclude the corresponding group from estimation of the shading estimation coefficient.

In step S312, the effective group determiner 223 averages the average value (Ag[g]) of the block evaluation value Eb for each distance (D[0] to D[d]) from the center of the image, and the group evaluation value Ad [d]) is determined.

The group evaluation value Ad[d] according to an embodiment may be expressed as in Equation 22.

23 is a diagram illustrating an example of a group processed by a valid group determiner according to an embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIG. 23 .

Referring to FIG. 23 , an example of a group having a distance D[0] to D[6] is shown.

In step S313 , the effective group determination unit 223 calculates a difference value Diff between the group evaluation value Ad and the adjacent distance group for each distance group.

For all candidates of the shading estimation coefficient, the valid group determiner 223 performs steps S310 to S313.

In step S314, the effective group determiner 223 determines whether the shading estimation coefficient used in steps S310 to S313 is an initial shading estimation coefficient.

As a result of the determination in step S314, if it is the shading estimation initial coefficient, in step S315, the effective group determiner 223 uses the values of the group evaluation values ((Ad)[d]) to determine the distances D[0] to D [d]) determines whether the group is valid or invalid (Ed=1 if valid, Ed=0 if invalid).

According to an exemplary embodiment, the group evaluation value (Ad)[d]) is 0, or the difference value Diff of the group evaluation value Ad with the adjacent distance group is greater than a predetermined upper limit or less than a predetermined lower limit. A small group may be declared invalid.

24 is a diagram illustrating an example of a group processed by a valid group determiner according to another embodiment. The shading estimator 124 may include the operation of the effective group determiner 223 in FIGS. 24A and 24B . Referring to FIG. 24A , it is an example of a case where D[2] to D[6] are valid and D[0] to D[1] are invalid. Referring to FIG. 24B , it is an example of a case where D[0] to D[3] and D[6] are valid and D[4] and D[5] are invalid.

According to an embodiment, the valid group determiner 223 validates the group determined to be invalid according to the group continuity determiner determining the continuity of the effective group among the groups from the center to the periphery of the captured image and the group continuity determination result. It may include an invalid group validator.

In step S316, the valid group determining unit 223 determines that there are two or more consecutive valid groups (ie, Ed=1) on both sides of a single invalid group (ie, Ed=0), and also includes a single invalid group and a single invalid group. When the difference value Diff of the group evaluation value Ad from both sides of the effective group is less than or equal to a predetermined threshold value (ie, invalid distance group interpolation threshold), a single invalid group may be changed to a valid group (that is, , interpolating Ed=0 to 1).

25 is a diagram for explaining a method of operating a valid group determiner according to an embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIG. 25 .

Referring to FIGS. 25A to 24C, FIG. 25A is an example of an interpolation target because the difference value between the group evaluation values Ad of both sides of D[2] is less than or equal to a predetermined threshold, and FIG. 25B shows that both sides of D[1] are Since it is not continuous, it is an example not to be interpolated, and FIG. 25C is an example not to be interpolated because D[2] to D[3] are not Ed=0 alone.

In step S317, the valid group determining unit 223 calculates whether the number of consecutive valid groups (ie, effective distance) Ne is equal to or greater than a predetermined threshold (ie, valid group consecutive number threshold).

If it is determined in step S317 that the number of consecutive valid groups Ne is less than a predetermined threshold, in step S318, the effective group determination unit 223 sets the shading estimation coefficient determination method as variance estimation. According to an embodiment, when the continuous number Ne of effective groups is small, since there is a high possibility that the object is a complex object that is difficult to estimate, the effective group determiner 223 may determine the shading estimation coefficient determination method as variance estimation.

On the other hand, if it is determined in step S317 that the number of consecutive valid groups Ne is equal to or greater than a predetermined threshold value, or if it is determined in step S314 that the shading estimation initial coefficient is not, the effective group determination unit 223 performs shading estimation Let the coefficient determination method be approximate straight line estimation.

For all candidates of the shading estimation coefficient, the group evaluation value determining unit 222 performs steps S314 to S319.

In step S320, the approximation estimator 224 estimates an approximate straight line for the group evaluation value Ad of the effective group by using the least-squares method for all candidates of the shading estimation coefficient. According to an embodiment, the effective distance obtained by using the initial shading estimation coefficient in steps S315 and S316 may be applied to all candidates of the shading estimation coefficient. For example, when the effective distances of the shading estimation initial coefficients are D[2] to D[4], the effective distances of all shading estimation coefficients may be D[2] to D[4].

In step S321, the approximation estimator 224 determines whether there is a table number of the shading estimation coefficient in which the slope of the approximate straight line estimated in step S320 is negative.

As a result of the determination in step S321, if there is a table number of the shading estimation coefficient in which the slope of the approximate straight line is negative, in step S322, the approximate estimation unit 224 selects a table number to have a value closest to 0 among the slopes of the negative number. It is determined by the table number of the shading estimation coefficient.

As a result of the determination in step S321, if there is no table number of the shading estimation coefficient in which the slope of the approximate straight line is negative, in step S323, the approximate estimation unit 224 converts the table number to have the smallest value to the table number of the shading estimation coefficient. to be decided by

According to an embodiment, the approximation estimator 224 may select a shading estimation coefficient in which the approximated straight line approaches the horizontal or the shading estimation coefficient closest to the slope of the preset reference. In addition, when approximating the ratio of R values, the approximation estimator 224 may use the ratio of the R values to all RGB (R/(R+G+B)), and three times the ratio of the R values to all RGB ( 3R/(R+G+B)) can also be used.

In step S324, the approximate estimator 224 subtracts the previously estimated table number from the currently estimated table number. According to an embodiment, the approximate estimator 224 determines whether to update the estimation result by using hysteresis so that the estimation result is stabilized.

As a result of obtaining the difference in step S324, if the value obtained by subtracting the previously estimated table number from the current estimated table number is 1, in step S325, the approximation estimator 224 determines that Slope > SlopeHys (slope of hysteresis) x ( -1) to determine whether it is

As a result of obtaining the difference in step S324, if the value obtained by subtracting the previously estimated table number from the currently estimated table number is (-1), in step S326, the approximate estimator 224 determines that Slope < SlopeHys (slope of hysteresis). ) to determine whether

As a result of the determination in step S325, if Slope>SlopeHys(slope of hysteresis)x(-1), in step S327, the approximate estimator 224 maintains the estimation result as the previously estimated table number.

As a result of obtaining the difference in step S324, if the value obtained by subtracting the previously estimated table number from the currently estimated table number is not ±1, as a result of the determination in step S325, Slope > SlopeHys (slope of hysteresis) x (-1) Otherwise, or if it is determined in step S326 that Slope is not SlopeHys (slope of hysteresis), in step S328, the approximate estimator 224 updates the estimation result with the currently estimated table number.

As a result of the determination in step S326, if Slope < SlopeHys (slope of hysteresis), in step S329 the approximate estimator 224 maintains the estimation result as a previously estimated table number.

26 is a graph illustrating an approximate straight line processed by an approximate estimator according to an exemplary embodiment.

The shading estimator 124 may include the operation of the approximate estimator 224 in FIG. 26 .

Referring to FIG. 26 , there is an approximate straight line for all candidates of the shading estimation coefficient, and the shading estimation coefficient ( FIG. 26( a ) ) that is not over-corrected and makes the correction result close to the horizontal among the approximate straight lines is used as the estimation result. On the other hand, when all approximate straight lines are overcorrected, the shading estimation coefficient that weakens the correction degree may be used as the estimation result.

27 is a flowchart illustrating a method of operating a variance estimator according to an exemplary embodiment.

The shading estimator 124 may include the operation of the variance estimator 225 in FIG. 27 .

The variance estimating unit 225 according to an embodiment of the present invention may generate a sample from the block evaluation average value Ab and each block evaluation value Eb, which are average values of the block evaluation values Eb of the entire image, in the case of a complex subject for which approximate straight line estimation is difficult. The variance can be calculated and the table number (t) that minimizes the variance can be used as the estimation result.

In step S331, the variance estimation unit 225 determines a block evaluation average value Ab of valid blocks.

According to an embodiment, the block evaluation average value Ab of valid blocks may be expressed as in Equation 23.

In Equation 23, Nb is the number of valid blocks (block weight (Bw)=1).

In step S332, the sample variance (Varp) of the valid block is determined.

In step S333, the minimum value of the sample variance (Varp) of the valid block and the corresponding table number (t) are stored.

A sample variance (Varp) of a valid block according to an embodiment may be expressed as Equation (24).

In step S334, the variance estimation unit 225 determines whether steps S331 to S333 have been performed for all candidates of the shading estimation coefficients.

In step S335, the variance estimation unit 225 may use a table number t that minimizes the sample variance Varp of a valid block as an estimation result.

An estimation result for step S335 according to an embodiment may be expressed as Equation (25).

According to another embodiment, the shading estimator classifies a block image by group according to a distance from a block in the center of the image with respect to a block image to which a candidate of a shading estimation coefficient is applied, and determines whether a group is used for estimation. A color (R) ratio of the group around the image may be approximated by a straight line from the group in the center of the image. A shading correction table corresponding to one-to-one shading estimation coefficients may be selected according to a slope of an approximate straight line with respect to a shading estimation table corresponding to a plurality of light sources. The shading estimator according to another embodiment determines the table evaluation value by evaluating the continuity of the color R ratio change from the central region of the image to the peripheral region. The estimation error can be reduced.

28 is a block diagram illustrating a configuration of a shading estimator according to another embodiment.

According to another embodiment, the shading estimation unit increases the number of effective groups, that is, the number of evaluation target groups by changing the method for determining effective groups, and calculates the variance for each successive effective group, and adds the obtained variance. It can be set as the evaluation value of a coefficient.

Referring to FIG. 28 , the shading estimator 124 includes a group variance estimator 226 and a block instead of the approximate estimator 224 and the variance estimator 225 included in the shading estimator 124 of FIG. 19 . It includes a variance estimation unit 227, but is not limited thereto. The shading estimator 124 may estimate shading according to either group variance or block variance.

The shading estimator 124 may include the operation of the group variance estimator 226 .

The group variance estimator 226 according to an embodiment obtains a variance value of the group evaluation value Ad[d] for each successive effective group (ie, a group continuation region), and calculates the sum of the variance values of the shading estimation coefficient. It is set as an evaluation value, and the shading estimation coefficient which makes the evaluation value the smallest can be determined.

The shading estimator 124 may include the operation of the block variance estimator 227 . Also, the shading estimator 124 may include the operation of the group variance estimator 226 .

The block variance estimation unit 227 according to an embodiment may perform effective block variance estimation processing instead of the processing of the group variance estimation unit 226 when the number of group continuous regions is less than a predetermined threshold value.

29 is a flowchart illustrating an operation method of a shading estimator according to another embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIG. 29 .

According to an embodiment, the block color evaluation value determining unit 211 , the block weight coefficient determining unit 212 , the block evaluation value determining unit 213 , the group classifying unit 221 , and the group evaluation value determining unit of FIG. 222 and the effective group determining unit 223 are the block color evaluation value determining unit 211, block weight coefficient determining unit 212, block evaluation value determining unit 213, and group classification unit 221 of FIG. 19, respectively. , may operate in the same manner as the group evaluation value determiner 222 and the effective group determiner 223 .

In step S401, the effective group determination unit 223 determines the number of effective blocks Nb[g] for each group G[0] to G[g]. On the other hand, the effective group determination unit 223 sets the number of valid blocks Nb[g] to zero when the number of valid blocks Nb[g] is smaller than a predetermined threshold (that is, the threshold for the number of valid blocks). . According to an embodiment, the operation of step S401 may be the same as the operation of step S310.

In step S402, the effective group determination unit 223 determines the average value Ag[g] of the block evaluation values Eb for each group G[0] to G[g]. According to an embodiment, the operation of step S402 may be the same as that of step S311.

The average value Ag[g] of the block evaluation value Eb according to an exemplary embodiment may be obtained using Equation 21 of step S311. At this time, when the number of valid blocks Nb[g] is 0, the effective group determination unit 223 sets the average value Ag[g] of the block evaluation values Eb to 0. That is, the valid group determining unit 223 may invalidate the group when the number of valid blocks in the group is small.

In step S403, the effective group determining unit 223 averages the average value (Ag[g]) of the block evaluation value Eb for each distance (D[0] to D[d]) from the center of the image, and the group evaluation value Ad [d]) is determined. According to an embodiment, the operation of step S403 may be the same as that of step S312.

The group evaluation value Ad[d] according to an embodiment may be obtained using Equation 22 of step S312.

For all candidates of the shading estimation coefficients, the effective group determination unit 223 executes steps S401 to S403.

In step S404, the effective group determination unit 223 determines whether the shading estimation coefficient used in steps S401 to S403 is an initial shading estimation coefficient.

As a result of the determination in step S404, if it is the shading estimation initial coefficient, in step S405, the effective group determining unit 223 for each distance group, the difference value (Diff) of the group evaluation value (Ad[d]) with the adjacent distance group to calculate

In step S406, the effective group determination unit 223 determines the effective group and the group continuation area.

30 is a diagram for explaining a method of determining a valid group according to an embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIG. 30 .

30A and 30B, the horizontal direction of FIGS. 30A and 30B shows a case in which the image is divided into seven groups of distances D[0] to D[6], and the vertical direction of FIGS. 30A and 30B is The group evaluation value Ad[d] for each distance D[0] to D[6] is shown.

According to an embodiment, the difference value Diff between the group evaluation value Ad[d] of the current group and the group evaluation value Ad[d] for both groups of the current group is a predetermined threshold value (that is, , adjacent group difference determination threshold), the valid group determination unit 223 determines the current group as an invalid group.

Referring to FIG. 30A, the group evaluation values (Ad[d]) of D[0] to D[2] and the group evaluation values (Ad[d]) of D[3] to D[6] are almost constant, respectively, and , the difference between these distances is almost zero. Also, the difference between D[2] and D[3] is greater than the adjacent group difference judgment threshold. In this case, the effective group determining unit 223 determines all of the groups D[0] to D[6] as valid groups.

30B, the group evaluation values (Ad[d]) of D[0] to D[2] and the group evaluation values (Ad[d]) of D[4] to D[6] are almost constant, respectively , the difference between these distances is almost zero. In addition, the difference value between D[2] and D[3] and the difference value between D[3] and D[4] are respectively greater than the adjacent group difference judgment threshold value. In this case, the valid group determination unit 223 determines each group of D[0] to D[2] and D[4] to D[6] as a valid group, while D[3] is determined to be an invalid group do.

According to an embodiment, in the case of a group in which only one adjacent group exists, such as D[0] or D[6], the difference value (Diff) of the group evaluation value (Ad[d]) with the adjacent group is adjacent It may be determined whether the group is a valid group by comparing it with a group difference determination threshold.

31 is a diagram for explaining a method of determining a valid group according to another embodiment.

The shading estimator 124 may include the operation of the effective group determiner 223 in FIG. 31 .

Referring to FIG. 31 , similarly to FIGS. 30A and 30B , the horizontal direction of FIG. 31 represents a case in which the image is divided into seven groups of distances D[0] to D[6], and the vertical direction of FIG. 31 is each distance The group evaluation value (Ad[d]) for each (D[0] to D[6]) is shown.

The Diff array stores 0 or 1 according to the difference value (Diff) between the group evaluation value (Ad[d]) of the current group and the group evaluation value (Ad[d]) with the adjacent group. According to an embodiment, the Diff arrangement is such that the difference value (Diff) between the group evaluation value (Ad[d]) of the current group and the group evaluation value (Ad[d]) with the adjacent group is equal to or greater than the adjacent group difference determination threshold value The interval is set to 0, and the interval in which the difference value (Diff) between the group evaluation value (Ad[d]) of the current group and the group evaluation value (Ad[d]) with the adjacent group is within the adjacent group difference judgment threshold is 1 to store this value. In this case, the Diff array has a length of (the number of adjacent groups)+1, and the first and last components of the Diff array are set to 0.

And, the Ed array determines whether each group is valid. According to one embodiment, the Ed array stores 1 if at least one of the adjacent components of the Diff array is 1. In addition, the effective group determination unit 223 determines that the group continuation region (Series) is when two or more 1s are consecutive in the Ed array, that is, when two or more valid groups are consecutive.

Referring to FIG. 31 , the Diff array stores 0 in (*D0), (D1D2), (D2D3), and (D7*) components. And, Ed array stores 0 in D2 component. Accordingly, the valid group determining unit 223 determines that D[2] is an invalid group, and determines that the other D[0], D[1], D[3] to D[6] are valid groups. Meanwhile, the effective group determination unit 223 determines D[0] and D[1] and D[3] to D[6] as group continuation regions, respectively.

In step S407, the effective group determination unit 223 determines whether the number Ns of group continuous areas is equal to or greater than a predetermined threshold (ie, continuous area threshold).

Referring to FIG. 31 , the number Ns of group continuation regions is two.

If it is determined in step S407 that the number Ns of group continuous regions is equal to or greater than the continuous region threshold, or if the determination in step S404 is not the initial coefficient of shading estimation, in step S408 the group variance estimation unit 226 returns the group evaluation value Determine the average value (AveAd) of (Ad[d]).

The average value AveAd of the group evaluation values Ad[d] according to an embodiment may be expressed as Equation 26.

In the formula (26), Ne is the number of groups in the group continuation area. According to an embodiment, the group continuation region obtained in steps S405 and S406 may be commonly used when obtaining a variance value according to the shading estimation initial coefficient or a variance value according to a shading estimation coefficient other than the shading estimation initial coefficient.

In step S409, the group variance estimation unit 226 determines a variance value SeriesVarp[n] of the group evaluation value Ad[d].

A variance value SeriesVarp[n] of the group evaluation value Ad according to an embodiment may be expressed as Equation 27.

In step S410, the group variance estimation unit 226 determines a table evaluation value Et[t].

The table evaluation value Et[t] according to an embodiment may be expressed as Equation (28).

For all candidates of the shading estimation coefficients, the group variance estimation unit 226 performs steps S408 to S410.

In step S411, the group variance estimation unit 226 determines a shading estimation coefficient that has the smallest table evaluation value Et[t] among all the shading estimation coefficients.

The shading estimation coefficient for step S411 according to an embodiment may be expressed as Equation 29.

On the other hand, if it is determined in step S407 that the number Ns of group continuous regions is smaller than the continuous region threshold, the block variance estimation unit 227 in step S412 uses the estimation method as variance estimation. According to an embodiment, the operation of the block variance estimator 227 may be the same as the operation of the variance estimator 225 of FIG. 27 .

The image display method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also included in the scope of the present invention. belongs to

100: image processing device
300: input unit
310: data processing unit

Claims (20)

an input unit for receiving a captured image; and
Block statistics representing image characteristics of each of the plurality of blocks are determined using a plurality of blocks obtained by dividing the captured image, and applied to the captured image using the determined block statistics and the amount of infrared light included in the light source. a data processing unit that determines a shading estimation coefficient and corrects shading of the captured image by using the determined shading estimation coefficient; including,
The data processing unit,
A block color evaluation value is determined using the determined block statistics, a block weight is determined using the determined block color evaluation value, and a block evaluation value is determined using the determined block statistics and the determined block weight. and determining a shading estimation coefficient to be applied to the captured image by using the block evaluation value. According to claim 1,
The data processing unit,
and determining a shading estimation coefficient to be applied to the captured image by using at least one of a luminance of the captured image, a color temperature of the light source, and flatness of the captured image. 3. The method of claim 2,
The data processing unit,
The image processing apparatus of claim 1, wherein the flatness of the captured image is determined by summing differences in characteristic values of blocks continuous from a peripheral portion to a central portion of the captured image. delete According to claim 1,
The data processing unit,
An image processing apparatus for determining a histogram weight using the determined block color evaluation value, and determining the block weight using the histogram weight and G-level weight. According to claim 1,
The data processing unit,
classifying the plurality of blocks into a plurality of groups, determining a group evaluation value using the determined block statistics and the classified group, determining a valid group using the determined block statistics and the determined group evaluation value, and determining a shading estimation coefficient to be applied to the captured image by using the determined effective group. 7. The method of claim 6,
The data processing unit,
and averaging an average value of the determined block evaluation values for each distance from the center of the captured image to determine the group evaluation value. 7. The method of claim 6,
The data processing unit,
estimating an approximate straight line using the determined group evaluation value, determining a shading estimation coefficient to be applied to the captured image using the estimated approximate straight line, estimating a sample variance using the determined block evaluation value, and An image processing apparatus for determining a shading estimation coefficient to be applied to the captured image by using the estimated sample variance. 9. The method of claim 8,
The data processing unit,
If the slope of the estimated approximate straight line is negative, a shading estimation coefficient is determined to have a value closest to 0 among the slopes of the negative number, and if the slope of the estimated approximate straight line is not negative, it has the smallest value among the slopes. An image processing apparatus that determines a shading estimation coefficient to be used. 7. The method of claim 6,
The data processing unit,
Estimating variance using the determined group evaluation value, determining a shading estimation coefficient to be applied to the captured image using the estimated variance, estimating a sample variance using the determined block evaluation value, and An image processing apparatus for determining a shading estimation coefficient to be applied to the captured image by using a sample variance. receiving a captured image;
determining block statistics representing image characteristics of the plurality of blocks using a plurality of blocks obtained by dividing the captured image;
determining a shading estimation coefficient to be applied to the captured image using the determined block statistics and an amount of infrared light included in a light source; and
correcting shading of the captured image using the determined shading estimation coefficient;
The step of determining the shading estimation coefficient comprises:
determining a block color evaluation value using the determined block statistics;
determining a block weight using the determined block color evaluation value;
determining a block evaluation value using the determined block statistics and the determined block weight; and
and determining a shading estimation coefficient to be applied to the captured image by using the determined block evaluation value. 12. The method of claim 11,
The step of determining the shading estimation coefficient comprises:
and determining a shading estimation coefficient to be applied to the captured image by using at least one of a luminance of the captured image, a light source color temperature, and a flatness of the captured image. 13. The method of claim 12,
The flatness of the captured image is,
An image processing method of determining flatness of the captured image by summing differences in characteristic values of blocks continuous from a peripheral portion to a central portion of the captured image. delete 12. The method of claim 11,
Determining the block weight includes:
and determining a histogram weight using the determined block color evaluation value, and determining the block weight using the histogram weight and the G-level weight. 12. The method of claim 11,
The step of determining the shading estimation coefficient comprises:
classifying the plurality of blocks into a plurality of groups;
determining a group evaluation value using the determined block statistics and the classified group;
determining a valid group using the determined block statistics and the determined group evaluation value; and
and determining a shading estimation coefficient to be applied to the captured image by using the determined effective group. 17. The method of claim 16,
The step of determining the group evaluation value,
An image processing method for determining the group evaluation value by averaging an average value of the determined block evaluation values for each distance from the center of the captured image. 17. The method of claim 16,
The step of determining the shading estimation coefficient comprises:
estimating an approximate straight line using the determined group evaluation value, and determining a shading estimation coefficient to be applied to the captured image using the estimated approximate straight line; and
The method further comprising the step of estimating a sample variance using the determined block evaluation value, and determining a shading estimation coefficient to be applied to the captured image using the estimated sample variance. 17. The method of claim 16,
The step of determining the shading estimation coefficient comprises:
estimating a variance using the determined group evaluation value, and determining a shading estimation coefficient to be applied to the captured image using the estimated variance; and
The method further comprising the step of estimating a sample variance using the determined block evaluation value, and determining a shading estimation coefficient using the estimated sample variance. 19. The method of claim 18,
The step of determining the shading estimation coefficient comprises:
determining a shading estimation coefficient to have a value closest to 0 among the slopes of the negative number when the slope of the estimated approximate straight line is negative; and
and determining a shading estimation coefficient to have a smallest value among the slopes if the slope of the estimated approximate straight line is not negative.

Images