JP4769332B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method capable of automatically adjusting the contrast of a digital image obtained by a digital camera or the like to obtain a clearer image at high speed.

  A color image taken with a digital camera is actually taken due to the influence of the SN level that represents the noise ratio in the analog value obtained by the CCD element, which is an image sensor, and the conversion accuracy when the analog value is converted into a digital value. Since the natural image is limited to a range narrower than the dynamic range of the pixel density, there is a tendency that information in details with shadows is lost. This tendency is particularly significant when trying to photograph a sample in which a bright area and a dark area are mixed in the image. As an improvement, firstly, a method of performing contrast enhancement so as to expand the range of the luminance or the like of the digital image from a higher luminance image portion to a lower luminance image portion can be considered. As a conventional method of contrast enhancement, a histogram showing the distribution of the luminance values of all the pixels that make up the original image is created, and the luminance value of the pixels in the original image is set to a new luminance using the histogram's cumulative curve as the luminance conversion curve. There is a histogram equalization technique that converts to a value and enhances the contrast of the image. In this method, since the luminance of the pixels in the entire original image region is converted to a new luminance using the same luminance conversion curve, a portion where contrast is lowered may occur. For this reason, when it is desired to perform contrast enhancement over the entire image, it is necessary to perform contrast enhancement processing suitable for the region.

  As a further improvement of this technique, many local histogram equalization techniques that divide an image into a plurality of rectangular areas and apply the above-described histogram equalization technique for each area have been proposed (for example, see Patent Document 1). The configuration diagram is as shown in FIG. FIG. 22 shows a contrast improving unit, an image data dividing unit 221 that divides an image into rectangles, a histogram creating unit 222 that creates a histogram for each rectangle, and a contrast expanding unit 223 that performs contrast expansion for each rectangle. It becomes more. However, when this method is used, problems have been pointed out, such as the occurrence of a rectangular region in which the contrast is too emphasized, or the contrast being discontinuous at the boundary between adjacent rectangular regions.

  On the other hand, without using a histogram, as a solution to such a problem, the shutter time and aperture of the digital camera are changed for each field, and a bright part and a dark part are imaged separately. A method has also been proposed in which a halftone density is realized by synthesizing with a single image so as to approach the dynamic range of the pixel density degree of a natural image actually taken. As an example, there is an example referred to in Patent Document 2, and the configuration diagram of the apparatus is as shown in FIG. In FIG. 23, reference numeral 230 denotes an image sensor for performing a photoelectric effect on a subject, 231 denotes a memory for recording an image signal, 232 denotes multiplication means for multiplying the signal level by a constant, and 233 and 238 add weights according to the level of the image signal. Level weighting means, 234 is an adding means for adding signals, 235 is a speed converting means for converting the speed of the image signal, 236 is a level compressing means for compressing the level of the image signal, and 237 is a timing control for controlling the timing of each block. Means. The inventive device performs weighted synthesis according to the signal level of two or more images having different charge accumulation periods in the image sensor, converts the resultant synthesized image output into the speed of a standard television signal, Since the present invention relates to a television imaging apparatus that compresses to a reference level, it has speed conversion means, level compression means, and the like. For this reason, when applied to a digital camera, speed conversion means and level compression means are not necessary components. However, in the case of the method by image synthesis obtained in a plurality of charge accumulation periods as in the present invention, contrast discontinuity does not easily occur in the synthesized image, but since at least two images are taken continuously, the principle Cannot take the same image. For this reason, when these images are combined, there is a possibility that an image in which the details of the combined image are blurred or shifted is generated although it is affected by the shutter speed. Also, if the density range when shooting a bright part and the density range when shooting a dark part do not cover the entire density range in the image, there is a risk of discontinuity between the two intermediate density ranges. is there.

  By the way, when humans observe details and colors in such shadowed areas, human vision can perceive the wide density dynamics and colors of images without causing the above problems. Can do. The concept of central vision / peripheral vision Retinex centered on human vision is “An Alternative Technique for the Computation of the Designator in the Retinex Theory of Color Vision” by Edwin Land, National Academy of Science, Volume 84. Pp. 3078 to pp. 3080 (1986). In this, the Retinex concept of human vision is described by an inverse square function with a central visual field having a diameter of 2 to 4 basic units and a peripheral visual field having a diameter approximately 200 to 250 times the central visual field. ing. Then, the spatial average of the signal intensity in each of the central visual field and the peripheral visual field is defined as related to the perceived intensity. In recent years, methods for improving the color and lightness expression in the dark part as described above have been proposed in accordance with these principles. This example is described in Patent Document 3, and FIG. 24 shows its configuration. Although a gray scale image is described here as an example, it can be extended to a color image. The pixel value I (i, j) in (i, j) of the image is adjusted by the processor 241 and the filter 242, and the filtering process is performed.

  For each pixel, the processor 241 calculates an adjusted pixel value I ′ (i, j) as in the following equation (Equation 1). Here, F (x, y) is a peripheral visual field function representing the peripheral visual field, and “*” indicates a convolution calculation process.

  Then, the normalization coefficient K is determined so that F (x, y) satisfies the condition of the equation (Equation 2), whereby the second term of the equation (Equation 1) is the pixel value in the peripheral visual field. Corresponds to the average value. That is, the equation (Equation 1) corresponds to a logarithmic conversion of the ratio of the pixel value of each pixel to the average pixel value in a large area. The peripheral visual field function F (i, j) is designed so that the contribution ratio increases as it approaches the target pixel from the correspondence with the human visual model, and a Gaussian function such as Expression (3) is applied. . Here, c is a constant for controlling the adjusted pixel value I ′ (i, j) of each pixel value I (i, j).

  As described above, in the conventional invention, the target pixel value with respect to the average pixel value in the peripheral visual field is calculated as the adjusted pixel value I ′ (i, j), and this value is used by the display 243. Filter processing for generating Retinex output R (i, j) is performed by 242. Reference numeral 242 converts I ′ (i, j) from a logarithmic region to a pixel value region of R (i, j) handled by the display 243, and is the same for all pixels for the sake of simplification of processing. A process that applies an offset and gain conversion function is used.

  However, this method is greatly influenced by c that controls the peripheral visual field function. For example, if c is a large value, the peripheral visual field that contributes to the target pixel is increased, so that only color compensation in a large shadow is possible, whereas if c is a small value, only the vicinity of the target pixel is detected. Only an improvement in small shadow areas can be seen. Appropriate c needs to be taken into account according to the dynamic range of pixel values in the image handled in this way, and there is a problem of image dependency. As an improvement, a method of providing a plurality of peripheral visual field areas has been proposed in the same patent, but it is not clear how many peripheral visual field areas are prepared. In addition, in order to improve the improvement accuracy, there is a problem that the processing time becomes enormous by preparing many small peripheral areas from large peripheral areas.

  In addition, when the pixel value variation is extremely small within the maximum peripheral field of view set by a plurality of constants c, the adjusted pixel value I ′ (i, j) is I (i, j ) Regardless of (). In such a case, I ′ (i, j) at the target pixel with small pixel value variation is often located in the vicinity of the average of I ′ (i, j) in the entire input image, and offset and gain conversion at 242 Regardless of the function, there is a tendency to tend toward the center of the actual pixel value. In particular, in a uniformly wide image area having a highlight luminance, there is a problem that the luminance after adjustment is easily adjusted in a decreasing direction and visually deteriorates. On the other hand, color noise and compression noise generated at the time of photographing may appear due to excessive emphasis in a low luminance and wide area such as a night scene.

  As a technique for solving this problem, it was shown that these can be improved by deriving an adaptive composite image of the input image and the emphasized image from the prior application. On the other hand, flattening of the image, particularly a decrease in density difference at the contour portion, occurred.

JP 2000-285230 A JP-A-6-141229 Special Table 2000-511315

  In this way, the conventional contrast improvement technology based on the human visual model was able to obtain a clearer contrast-converted image than other methods. There is a problem that a lot of empirical knowledge is included in the design of the filter processing when converting into actual pixel values to be finally handled. As a countermeasure, a multi-resolution model with a plurality of peripheral fields of view has been proposed, but the processing time becomes enormous.

  Further, it is necessary to flatten the synthesized image generated by the image synthesis, particularly to further emphasize the contour portion.

  The present invention has been made in view of the above points, and provides an image processing apparatus and an image processing method capable of improving the contrast of a captured image more clearly and at high speed using only the captured image having a bright and dark part. The purpose is to do.

  In order to solve the above problems, the first image processing apparatus and the image processing method of the present invention obtain the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the surrounding area. When converting the obtained contrast improvement amount into an actual pixel value, by controlling according to the luminance of the target pixel, the flattening of the image generated in the composite image of the input image and the emphasized image, or the density difference in the contour portion The reduction can be improved.

  In order to solve the above problems, the second image processing apparatus and the image processing method of the present invention provide the luminance of the target pixel when obtaining the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the surrounding area. Accordingly, the comparison density which is the average pixel value in the peripheral area is controlled. In this way, the difference between the density of the target pixel and the surrounding comparative density can be enhanced, and the flattening of the image generated in the composite image of the input image and the emphasized image and the reduction of the density difference in the contour portion are improved. It is something that can be done.

  In order to solve the above problem, the third image processing apparatus and image processing method of the present invention obtains the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the surrounding area. When the obtained contrast improvement amount is converted into an actual pixel value, the contrast improvement amount is corrected according to the difference between the density of the target pixel and the density of the surrounding pixels, so that a composite image of the input image and the emphasized image is obtained. The flattening of the image and the reduction of the density difference in the contour portion can be improved.

  In order to solve the above-described problem, the fourth image processing apparatus and image processing method of the present invention first determines the vertical position of a pixel to be used for contrast improvement when obtaining an average pixel value in a peripheral region of the target pixel. Then, for the determined pixel, an addition value of pixel density in the vertical direction is obtained for each horizontal pixel position. Then, the comparison density for the target pixel is calculated by using the value of the horizontal pixel position included in the region set as the surrounding pixel of the target pixel from the obtained addition value at each horizontal pixel position, thereby simplifying the processing. Is to realize.

  In order to solve the above problems, the fifth image processing apparatus and image processing method of the present invention first determines the vertical and horizontal positions of pixels used for contrast improvement when obtaining the average pixel value in the peripheral region of the target pixel. decide. Then, an added value of the selected pixel density in the vertical direction is obtained for the determined horizontal pixel position. When the selected horizontal pixel position is included in the area set as the surrounding pixels of the target pixel, the added value at the included horizontal pixel position is used. When the selected horizontal pixel position is not included in the area set as the surrounding pixel of the target pixel, the added value at the adjacent selected horizontal pixel position is used. Based on this procedure, the process can be simplified by calculating the comparison density for the target pixel at high speed.

  In order to solve the above problem, the sixth image processing apparatus and image processing method of the present invention obtains the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the peripheral region. Then, the contrast improvement amount obtained in the same manner as in the first image processing apparatus and method of the present invention is converted into an actual pixel value, and a synthesis process with the input image is performed. Then, compared with the luminance / color component of the target pixel in the input image and the luminance / color component of the composite image, the luminance is not reduced, and the value proportional to the color component of the input image is used. By suppressing the color misregistration in the image, the image quality of the finally output image is improved.

  As described above, the first image processing apparatus and the image processing method of the present invention obtain the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the surrounding area. When converting the obtained contrast improvement amount into an actual pixel value, by controlling according to the luminance of the target pixel, the flattening of the image generated in the composite image of the input image and the emphasized image, or the density difference in the contour portion The reduction can be improved.

  As described above, according to the second image processing apparatus and image processing method of the present invention, when the contrast improvement amount is obtained by comparing the pixel value of the target pixel and the average pixel value in the surrounding area, the second image processing apparatus and the image processing method according to the luminance of the target pixel. Thus, the comparison density which is an average pixel value in the peripheral region is controlled. In this way, the difference between the density of the target pixel and the surrounding comparative density can be enhanced, and the flattening of the image generated in the composite image of the input image and the emphasized image and the reduction of the density difference in the contour portion are improved. It is something that can be done.

  As described above, the third image processing apparatus and the image processing method of the present invention obtain the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the peripheral area. When the obtained contrast improvement amount is converted into an actual pixel value, the contrast improvement amount is corrected according to the difference between the density of the target pixel and the density of the surrounding pixels, so that a composite image of the input image and the emphasized image is obtained. The flattening of the image and the reduction of the density difference in the contour portion can be improved.

  As described above, in the fourth image processing apparatus and image processing method of the present invention, when determining the average pixel value in the peripheral region of the target pixel, the vertical position of the pixel used for improving the contrast is first determined and determined. For the obtained pixel, an added value of pixel density in the vertical direction is obtained for each horizontal pixel position. Then, the comparison density for the target pixel is calculated by using the value of the horizontal pixel position included in the region set as the surrounding pixel of the target pixel from the obtained addition value at each horizontal pixel position, thereby simplifying the processing. Is to realize.

  As described above, in the fifth image processing apparatus and image processing method of the present invention, when obtaining the average pixel value in the peripheral region of the target pixel, first, the vertical and horizontal position of the pixel used for the contrast improvement is determined. . Then, an added value of the selected pixel density in the vertical direction is obtained for the determined horizontal pixel position. When the selected horizontal pixel position is included in the area set as the surrounding pixels of the target pixel, the added value at the included horizontal pixel position is used. When the selected horizontal pixel position is not included in the area set as the surrounding pixel of the target pixel, the added value at the adjacent selected horizontal pixel position is used. Based on this procedure, the process can be simplified by calculating the comparison density for the target pixel at high speed.

  As described above, the sixth image processing apparatus and image processing method of the present invention obtain the contrast improvement amount by comparing the pixel value of the target pixel and the average pixel value in the surrounding area. Then, the contrast improvement amount obtained in the same manner as in the first image processing apparatus and method of the present invention is converted into an actual pixel value, and a synthesis process with the input image is performed. Then, compared with the luminance / color component of the target pixel in the input image and the luminance / color component of the composite image, the luminance is not reduced, and the value proportional to the color component of the input image is used. By suppressing the color misregistration in the image, the image quality of the finally output image is improved.

1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment of the present invention. The figure showing the structure of the contrast improvement means in the image processing apparatus which is the 1st Embodiment of this invention 1 is a block diagram showing the configuration of contrast improving means at multiple resolutions in the image processing apparatus according to the first embodiment of the present invention. 1 is a block diagram showing the configuration of image composition means in an image processing apparatus according to a first embodiment of the present invention. 1 is a block diagram showing a configuration of enhancement amount calculation means in an image processing apparatus according to a first embodiment of the present invention. The block diagram showing the structure of the emphasis amount calculation means in the image processing apparatus which is the 2nd Embodiment of this invention The block diagram showing the structure of the emphasis amount calculation means in the image processing apparatus which is the 3rd Embodiment of this invention. Schematic diagram showing examples of pre-processing means and post-processing means in the image processing apparatus according to the first, second, third, fourth, and fifth embodiments of the present invention. FIG. 1 is an overall flowchart of the image processing method according to the first embodiment of the present invention. Flowchart of contrast improvement processing and image composition processing in the image processing method according to the first embodiment of the present invention Flowchart of contrast improvement processing in the image processing method according to the second embodiment of the present invention Flowchart of contrast improvement processing in the image processing method according to the second embodiment of the present invention Conceptual diagram of a human visual model used for contrast improvement processing in the image processing method according to any one of the first to sixth embodiments of the present invention. The block diagram showing the structure of the contrast improvement means in the image processing apparatus which is the 4th Embodiment of this invention The block diagram showing the structure of the contrast improvement means in the multi-resolution in the image processing apparatus which is the 4th Embodiment of this invention Flowchart of contrast improvement processing in the image processing method according to the fourth embodiment of the present invention FIG. 7 is a block diagram showing the configuration of contrast improving means in an image processing apparatus according to a fifth embodiment of the present invention. The figure showing the outline | summary of the contrast improvement process in the image processing method which is the 5th Embodiment of this invention The figure showing the structure of the post-processing means in the image processing apparatus which is the 6th Embodiment of this invention Flowchart of contrast improvement processing and post-processing means in the image processing method according to the sixth embodiment of the present invention The figure showing the outline | summary of the post-process in the image processing method which is the 6th Embodiment of this invention The block diagram showing the structure of the image processing apparatus in a prior art example The block diagram showing the structure of the image processing apparatus in a prior art example The block diagram showing the structure of the image processing apparatus in a prior art example

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an overall configuration diagram of an image processing apparatus according to the first to sixth embodiments of the present invention, and FIG. 2 is a contrast improvement means in the image treatment apparatus according to the first to third embodiments of the present invention. FIG. 4 shows a configuration diagram of image synthesizing means in the image processing apparatus according to the first to sixth embodiments.

  FIG. 5 is a block diagram of enhancement amount deriving improvement means in the image processing apparatus according to the first embodiment of the present invention.

  FIG. 6 is a block diagram of enhancement amount deriving improvement means in the image processing apparatus according to the second embodiment of the present invention.

  FIG. 7 is a block diagram of enhancement amount deriving improvement means in the image processing apparatus according to the third embodiment of the present invention.

  FIG. 14 is a block diagram of the contrast improving means in the image processing apparatus according to the fourth embodiment of the present invention. FIG. 17 is a block diagram of the contrast improving means in the image processing apparatus according to the fifth embodiment of the present invention. Represents the figure.

  FIG. 19 is a configuration diagram of post-processing means in the image processing apparatus according to the sixth embodiment of the present invention.

  FIG. 9 is a diagram showing an overall flowchart in the image processing method according to the first to sixth embodiments of the present invention.

  FIG. 10 is a diagram showing a flowchart of contrast improvement processing and image composition processing in the image processing method according to the first embodiment of the present invention.

  FIG. 11 is a diagram showing a flowchart of contrast improvement processing in the image processing method according to the second embodiment of the present invention.

  FIG. 12 shows a flowchart of contrast improvement processing in the image processing method according to the third embodiment of the present invention, and FIG. 20 shows contrast improvement processing and post-processing in the image processing method according to the sixth embodiment of the present invention. It is a figure showing the flowchart of a means.

  Note that, in each drawing of the configuration diagram, the same number is assigned to the same part. In addition, hereinafter, pixel units are all used as the unit of the pixel position (i, j).

(First embodiment)
First, an image processing apparatus and an image processing method according to the first embodiment of the present invention will be described. FIG. 1 is a diagram showing the overall configuration. 1 is an input image, 2 is an output image to be finally output, and 3 is an image after enhancement processing obtained by the contrast improving means 12. 10 is an image input means such as a CCD element for obtaining one input image, 11 is a preprocessing means for performing preprocessing such as inverse gamma conversion on the input image, and 12 is obtained by 11. Contrast improvement processing means for performing contrast improvement processing for enhancing details of the digital input image, 13 is an image composition means for synthesizing the digital input image and the enhanced image obtained by the 12 contrast improvement means, and 14 is 13. Post-processing means for performing post-processing such as gamma conversion applied to the input image on the obtained composite image. Reference numeral 15 denotes a desired device (printer, display, etc.) using the composite image obtained in 13 as an image after final processing. Etc.) is output. The contrast improving means is configured as shown in FIG. 2. In FIG. 2, 20 is a comparison pixel determining means for determining peripheral pixels to be compared from the peripheral area of the target pixel Pij, and 21 is a color 3 in the target pixel Pij. By comparing the component value VPij (r (i, j), g (i, j), b (i, j)) with the comparison target pixel selected from the Pij surrounding area of the width c, the contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is calculated as an enhancement amount deriving unit. Reference numeral 22 denotes a contrast improvement amount VRPij obtained by the enhancement amount deriving unit 21 as an actual pixel value. Is a conversion reference value calculation means for obtaining a conversion reference value VTc (rtc (i, j), gtc (i, j), btc (i, j)) at the time of conversion to 23. The pixel value conversion unit converts the contrast improvement amount VRPij obtained by the enhancement amount deriving unit 21 into a pixel value after contrast improvement in the actual pixel Pij. The enhancement amount deriving unit 21 includes a surrounding average unit 50 and an improvement amount calculating unit 51 as shown in FIG.

  Further, as shown in FIG. 4, the image synthesizing means 13 is based on the luminance information in the input image 1 and the coupling coefficient ws (s = 1, 3; where w1 is a coupling coefficient applied to the input image 1 and w3 represents a coupling coefficient applied to the enhanced image 3), and a coupling coefficient w0 obtained by the coupling coefficient derivation means 40. , W1 to generate a weighted average image 41 of the input image 1 and the weighted average image 3 obtained by the contrast improving means, and the weighted average composite image 41 obtained by 41, the input image 1, and the weighted image It comprises output value determining means 42 for comparing the image 3 and determining the pixel value of the output image.

  The operation of the image processing apparatus according to the first embodiment configured as described above will be described with reference to FIGS.

  The color image 1 is digitally input via the image input means 10. In the case of a color image, the component data of red, green g, and blue b can be obtained with the accuracy of the input means (a value from 0 to 255 for 8 bits). 10 normalizes this value from 0.0 to 1.0.

  Preprocessing as shown in FIG. 8 is performed on the digital input image 1. In recent years, image data that has undergone such contrast improvement has been photographed with a digital camera (DSC). Normally, when image data captured by DSC is displayed on a monitor on a personal computer or the like, a gamma function having a convex shape as shown in the upper left of FIG. 8 is used to make the input image clearer. Conversion processing is being performed. In the generally used sRGB space, 2.2 is used as the gamma γ in this gamma conversion. Even when output is performed by a printer, image processing for output is executed on the assumption that this gamma function is preliminarily applied to the input image. Therefore, in order to accurately extract image data obtained by a photographing element such as a CCD, a conversion process using an inverse gamma function is required to restore the effect of γ = 2.2. A downward convex gamma conversion as shown in the figure is performed.

  Next, the contrast improving means 12 performs a contrast improving process for improving the contrast in the dark part of the input image on the digital input image 1 subjected to inverse gamma conversion. The contrast improving means 12 performs as shown in FIG. In human vision, as schematically shown in FIG. 13, the pixel information (color, contrast, etc.) of Pij is not recognized only by the pixel value perceived with respect to the target pixel Pij, but the target pixel Pij The pixel information of Pij is perceived by adjusting the pixel value of the target pixel value Pij based on the relative relationship with the information of surrounding pixels. This is called the Retinex concept introduced by Edwin Land, as explained in the previous example. Even in a scene where there is a change in the intensity of the pixel value, the color of the object can be accurately recognized. Even in the present invention, this concept is used to clarify the color and detail information in a dark part such as a shadow.

  First, a pixel value VPij (r (i, j), g (i, j), b (i, j))) of the target pixel Pij is regarded as a central visual field in Land, and an area belonging to a rectangular area of c pixels around it Is regarded as peripheral vision. Then, the weighted average pixel value VAPij (Ar (i, j), Ag (i, j), Ab (i, j)) of the pixel value in the peripheral visual field is obtained, and the relative relationship between VAPij and VPij is determined. The connected contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is calculated. As this VAPij, the pixel value VPij (r (i, j), g (i, j), b (i, j) in the peripheral visual field of the c pixel in the second term of the formula (Equation 1) as in the conventional example. ) And expressions (Equation 2) and (Equation 3) can be defined by the convolution integral value of the peripheral visual field function F (x, y) defined by the Gaussian function, and the contrast improvement amount VRPij is also expressed by the equation ( Although it is possible to define Equation 1) independently for the three components, in the present invention, the definition is made in consideration of simplification and speeding up of processing. As an example, VAPij is an average value of pixel values VPij (r (i, j), g (i, j), b (i, j)) in the peripheral visual field of c pixels as shown in Equation (4). The contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is also defined by the weighted average pixel value of the pixel value VPij for each component as shown in Equation (5) It can be defined as the ratio to VAPij.

  Further, the average value of the luminance y (i, j) in the peripheral visual field of the c pixel is defined as three components of VAPij as in the equations (Equation 6) and (Equation 7), and the contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) can also be defined as a ratio of each component value in the pixel Pij to the weighted average pixel value VAPij as shown in Expression (7).

  By doing so, it is considered that the balance of the obtained contrast improvement amounts can be maintained better than the calculation of the contrast improvement amount independently for each component as in the equations (4) and (5). Therefore, in the present invention, the definitions based on the equations (Equation 6) and (Equation 7) are adopted.

  The contrast improvement amount VRPij for the target pixel Pij as described above is performed on all the images in the input image. This processing is performed by the surrounding average means 50 and the improvement amount calculation means 51.

  Next, it is necessary to convert this contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) into an actual pixel value. As this processing, there is a method of converting each component of VRPij into a value within the range of 0.0 to 1.0 and converting it to an actual pixel value extracted from a necessary part. Therefore, the conversion reference value VTc (trc, tgc, tbc) is calculated, and the conversion reference value is multiplied by each component of VRPij to obtain the pixel value in the pixel Pij of the enhanced image 3. In this case, it may be converted from 0 to 255, which is normally handled 8-bit integer data, but here the conversion reference value and contrast improvement amount are first converted to real data within a normalized range of 0.0 to 1.0. The data is converted using VRPij, and then converted to 8-bit integer data from 0 to 255. Therefore, the value that each component of VTc can take is in the range of 0.0 to 1.0. Although the conversion reference value calculation means 22 determines VTc, it can be set to the average of the pixel values that can be uniformly taken as VTc = (0.5, 0.5, 0.5). However, it seems that it is possible to suppress a sudden change from the input image 1 by converting the brightness so that the brightness is higher and the brightness is lower. That is, by controlling the value of VTc according to the luminance y (i, j) at the target pixel Pij of the input image 1 in this way, it is possible to suppress a sudden increase in the luminance level of the shadow portion. Therefore, VTc (trc, tgc, tbc) is controlled according to y (i, j) as described above. An example of this is given by equation (Equation 8), but it is also conceivable to use a nonlinear function instead of such a linear function.

  If the slope in the equation (Equation 8) is excessively large, the contrast improvement in the dark part is excessively suppressed, and the characteristic that enhances the contrast by emphasizing the part having a variation difference may be impaired. . Also, since the range that VTc can take is 0.0 to 1.0, it is necessary to set the equation (Equation 8) based on these conditions.

  In response to the result of VRPij of the enhancement amount deriving unit 21 and the result of VTc of the conversion reference value calculation unit 22, the pixel value conversion unit 23 obtains a pixel value at the pixel Pij of the enhanced image 3.

  On the other hand, as a problem of the contrast improvement method based on the Retinex theory, a decrease in the luminance level occurs in a highlight portion having a uniform color in a very large area described in the conventional example. Similarly, in a shadow portion having a uniform color in a very large area, the luminance level in the emphasized image rapidly rises, so that the color noise in the shadow portion generated at the time of input by a CCD or the like is conspicuous. In order to solve these problems in the previous application (Japanese Patent Application No. 2002-33658), the inventors of the present patent application adaptively used the input image 1 and the enhanced image 3 obtained by the visual model. By combining, the reduction or increase in luminance level inherent in the input image 1 is suppressed. Although this means is also used in the present invention, the conversion reference value VTc for converting the VRPij from the above-described enhancement amount deriving means 21 into an actual pixel value is controlled according to the luminance y (i, j) of the target pixel Pij. By combining with the processing to be performed, it is possible to improve the appearance of the dark part while maintaining the atmosphere of the input image 1 more. The image synthesizing means 15 is configured as shown in FIG. 4. First, at 40, the coupling coefficient w1 applied to the input image 1 and the coupling coefficient w3 applied to the enhanced image 3 are controlled. In 41, a weighted average composite image 4 using w1 and w3 is generated. Finally, at 42, the luminance y (i, j) and the color differences cr (i, j), cb (i, j) of the target pixel Pij of the input image 1 are obtained, and the luminance wy ( i, j), color difference wcr (i, j), wcb (i, j) are obtained. If y (i, j) ≧ wy (i, j), then wy (i, j) = y (i, j) and this wy (i, j), wcr (i, j), The final output image 2 is generated by converting wcb (i, j) to VWPij (wr (i, j), wg (i, j), wb (i, j)) in the RGB space.

  At the time of 40, the pixel value VRPij of the input image 1 is compared with the pixel value VRPij of the emphasized image 3. If all three components are larger than VRPij, w1 = 1.0 and w3 = 0.0. The pixel value VWPij of the composite image is determined with w1 = w3 = 0.5. However, as described in the previous application (Japanese Patent Application No. 2002-33658), a method of controlling with edge information in the corresponding pixel of the input image Also, a method of controlling as in Expression (9) using the luminance y (i, j) and predetermined threshold values TH and THComb is also possible.

  In the case of equation (Equation 9), the processing overhead due to the exp function can be considered, but the overhead of this processing can be suppressed by making a table of the exp function corresponding to the preset accuracy and referring to the table. This formula (Equation 9) gives priority to the emphasized image 3 as much as possible in the dark portion, but based on the ratio of the luminance of the input image and the luminance of the emphasized image, if the improvement is rapid, the w3 is suppressed in the dark portion. It is devised to suppress noise enhancement.

  By performing the configuration and processing as described above, the image processing apparatus and the image processing method according to the first embodiment of the present invention effectively reduce the luminance in the highlight portion by effectively utilizing the advantages of the visual model of the conventional example. In addition, it is possible to easily and accurately perform contrast improvement that suppresses noise enhancement against a sharp increase in luminance in the shadow portion. Further, unlike the prior application and the conventional method of Jobson et al., It is not necessary to extract a region that seems to be more effective than the contrast improvement amount obtained in 21, which has the advantage of reducing the processing amount.

  Further, in the present invention, the contrast improving means 12 is constituted by one peripheral region c as shown in FIG. 2, but it seems that the size of the peripheral region c is caused by the size of the dark part to be improved. It is. Usually, in the case of natural images, there can be shadows with various sizes, so a multi-resolution model with multiple peripheral area sizes c [s] such as Jobson et al. Is. When the present invention is applied to such a multi-resolution model, the twelve contrast improving means are configured as shown in FIG.

  First, the peripheral range initialization means 30 sets c0 of a plurality of peripheral visual field region sizes c [s] (s = 0, 1,..., Cnum-1) prepared in advance as c = c0. Set. In this case, c [s] may be prepared in ascending order from the minimum size region, or may be prepared in descending order from the maximum size, but it is better to align the size change directions. Here, assuming that the images are prepared in descending order from the maximum size, it is assumed that the detail improvement in the input image is performed while sequentially reducing the peripheral visual field region. 20 and 21, the contrast improvement amount VRPij [s] (Rr # s (i, j), Rg # s () of the pixel value in the peripheral visual field is compared with the peripheral visual field in the rectangular area of c = ck that is currently set. i, j), Rb # s (i, j)) is calculated. Then, the end determination unit 31 determines whether or not the calculation of the contrast improvement amount in the peripheral visual field of the pixel Pij at 21 is completed for all c [s]. If it is determined that the process has not been completed, the process moves to the peripheral range changing means 32, the currently set peripheral visual field region c is changed to the next candidate, and the processes in steps 20 and 21 are performed again. On the other hand, if the end determination is made at 31, the Pij contrast improvement amount VRPij [s] (Rr # s (i, j), Rg # s (i, j), Rb) for each peripheral visual field region c [s]. #s (i, j)) Find the weighted average value and set the value as the Pij contrast improvement VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) . For simplification, the average value of VRPij [s] by each c [s] is adopted as the contrast improvement amount VRPij of Pij. In addition to this, it is also possible to compare the value VRPij [s] based on c [s] for each component and set the maximum value as the contrast improvement amount VRPij of Pij. In such a case, the processing time is increased by the comparison process rather than the addition process, but the pixel value fluctuation of the image generated as the emphasized image 3 becomes moderate, and there is an advantage that a sharp rise at the edge portion can be suppressed.

  The subsequent processing is the same as in the case of one peripheral range, and is therefore omitted.

  These processes are similarly performed by software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like in accordance with the image processing method according to the first embodiment of the present invention. Can be realized.

  The same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the first embodiment of the present invention.

(Second Embodiment)
Next, an image processing apparatus and an image processing method according to the second embodiment of the present invention will be described.

  The overall configuration diagram is configured as shown in FIG. 1 as in the first embodiment of the present invention, and the contrast improving means is as shown in FIG. When c [s] is provided, the configuration is as shown in FIG. Similarly, the image composition means 13 is as shown in FIG. The enhancement amount deriving means 21 in FIG. 2 or FIG. 3 is configured as shown in FIG.

  The enhancement amount deriving means 21 is a weighted weighted sum VAPij (Ar () of the comparison target pixels selected from around the target pixel Pij (r (i, j), g (i, j), b (i, j)). i, j), Ag (i, j), Ab (i, j)), and an edge information detecting means 60 for obtaining edge information e (i, j) of luminance in Pij; Correction coefficient calculation means 61 for determining a coefficient VDAPij (dcr (i, j), dcg (i, j), dcb (i, j)) for correcting 50 VAPij based on e (i, j) of Comparison amount correction means 62 for correcting VAPij based on the correction coefficient VDAPij obtained in 61, VAPij representing the local feature of Pij obtained in 62, and pixel Pij (r (i, j), g ( (I, j), b (i, j)) based on the ratio of contrast enhancement VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) Consists of means 51.

  The operation of the image processing apparatus according to the second embodiment configured as described above will be described.

  The color image 1 is digitally input via the image input means 10. In the case of a color image, the component data of red, green g, and blue b can be obtained with the accuracy of the input means (a value from 0 to 255 for 8 bits). 10 normalizes this value from 0.0 to 1.0. Next, the processing means 11 performs pre-processing for returning the gamma conversion of the input image to the original linearity for the digital input image 1. The contrast improving means 12 performs a contrast improving process for improving the contrast in the dark portion on the input image 1 after the preprocessing. The contrast improving means 12 is performed as shown in FIG. 2 or FIG. First, a comparison target pixel used for comparison density calculation is selected from surrounding pixels in the surrounding range c or c [s] of the target pixel, and a contrast enhancement amount VRPij (Rr (i, j) is calculated from the comparison target pixel and the target pixel. , Rg (i, j), Rb (i, j)). Then, this value is converted into a pixel value, which is actual 8-bit integer data, using the conversion reference value VTc (trc, tgc, tbc). Then, the image composition means 13 shown in FIG. 4 performs adaptive composition of the obtained enhanced image 3 and the input image 1, and the gamma conversion originally added to the input image on the obtained composite image 2 is performed again. When the post-processing unit 14 performs, the image output unit 15 generates a final output image to be output to a device device such as a printer or a display.

  The enhancement amount deriving means 21 first regards the pixel value VPij of the pixel Pij in the input image as the central visual field in Land, and regards the area belonging to the rectangular area of c pixels around it as the peripheral visual field. Then, based on the comparison target pixel selected by the comparison pixel determining means 20, a weighted weighted sum VAPij (Ar (i, j), Ag (i, j), Ab (i, j) within the surrounding range is obtained. ) Is calculated. At the same time, a correction coefficient VDAPij (dcr (i, j), dcg (i, j), dcb (i, j)) for correcting this value is obtained. In the first embodiment of the present invention, the flattening phenomenon near the edge of the output image 2 that occurs in the composite image 3 by controlling the conversion reference value VTc when converting the contrast improvement amount into the actual pixel value. Although it was suppressed, in order to cancel the flattening near this contour, instead of controlling the conversion reference value VTc, it is a uniform central value (here, it is handled in the range from 0.0 to 1.0, 0.5) It is also possible to control the weighted weighted sum VAPij (Ar (i, j), Ag (i, j), Ab (i, j)) in the surrounding range Aij obtained at 50 by setting It seems to be effective. Therefore, in the present invention, using the information of the target pixel Pij in the input image, a correction coefficient VDAPij (dcr (i, j), dcg (i, j), dcb (i, j)) for correcting VAPij is obtained, By multiplying this value by 61 for VAPij for each component, the degree of enhancement near the contour is increased. As the correction coefficient control, here, the degree of edge information is linearly expressed as in Expression (10) based on the edge information e (i, j) of the luminance y (i, j) of the target pixel Pij. Therefore, the correction coefficient was suppressed. However, this control method is not uniquely determined, and control using a nonlinear function is also possible. In Equation (Equation 10), ave_e represents the average of e (i, j) in the image, but 0.5 can be set for simplification. In the equation (Equation 10), ttt = dcr (i, j) = dcg (i, j) = dcb (i, j), and the weighted sum after correction is expressed as a weighted sum VAPij ′ = (Ar ′ (i, j) , Ag ′ (i, j), Ab ′ (i, j)).

  However, as in the first embodiment of the present invention, it is possible to control the correction coefficient linearly based on the luminance y (i, j) of the target pixel Pij in the input image. The correction coefficient is decreased as the luminance increases.

  By adopting such means, it is possible to improve the contrast in the dark part so as to preserve the sharpness of the emphasized image 3 and the composite image 2 of the image 3 and the input image 1 to some extent.

  Similarly to the first embodiment of the present invention, a configuration with multiple resolutions as shown in FIG. 3 is also possible. By doing so, the pixel value of the input image can be changed while taking advantage of the image sharpness preservation of the present invention. The contrast of the input image can be automatically improved without being greatly affected by the size of the dark part such as the dynamic range and shadow, leading to efficient image contrast adjustment.

  These processes are similarly performed by software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like in accordance with the image processing method according to the second embodiment of the present invention. Can be realized.

  Also, the same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the second embodiment of the present invention.

(Third embodiment)
Next, an image processing apparatus and an image processing method according to a third embodiment of the present invention will be described.

  The overall configuration diagram is configured as shown in FIG. 1 as in the first embodiment of the present invention, and the contrast improving means is as shown in FIG. When c [s] is provided, the configuration is as shown in FIG. Similarly, the image composition means 13 is as shown in FIG. The enhancement amount deriving means 21 in FIG. 2 or FIG. 3 is configured as shown in FIG.

  The enhancement amount deriving means 21 is a weighted weighted sum VAPij (Ar () of the comparison target pixels selected from around the target pixel Pij (r (i, j), g (i, j), b (i, j)). i, j), Ag (i, j), Ab (i, j)), and the difference between the luminance at Pij and the luminance of the comparison target pixel Qij selected from the surrounding area. Luminance difference averaging means 70 for obtaining weighted weighted sum Ady (i, j) of absolute values, and contrast enhancement amount VRPij (Rr (i, j), Rg obtained in 51 based on Ady (i, j) (i, j), Rb (i, j)) emphasis component VBRPij (BRr (i, j), BRg (i, j), BRb (i, j)) , 71 based on VBRPij (BRr (i, j), BRg (i, j), BRb (i, j)) obtained in 71, the contrast enhancement amount VRPij (Rr (i, j), Rg (i, j ), Rb (i, j)).

  The operation of the image processing apparatus according to the third embodiment configured as described above will be described.

  The color image 1 is digitally input via the image input means 10. In the case of a color image, the component data of red, green g, and blue b can be obtained with the accuracy of the input means (a value from 0 to 255 for 8 bits). 10 normalizes this value from 0.0 to 1.0. Next, the processing means 11 performs pre-processing for returning the gamma conversion of the input image to the original linearity for the digital input image 1. The contrast improving means 12 performs a contrast improving process for improving the contrast in the dark portion on the input image 1 after the preprocessing. The contrast improving means 12 is performed as shown in FIG. 2 or FIG. First, a comparison target pixel used for comparison density calculation is selected from surrounding pixels in the surrounding range c or c [s] of the target pixel, and a contrast enhancement amount VRPij (Rr (i, j) is calculated from the comparison target pixel and the target pixel. , Rg (i, j), Rb (i, j)). Then, this value is converted into a pixel value, which is actual 8-bit integer data, using the conversion reference value VTc (trc, tgc, tbc). Then, the image composition means 13 shown in FIG. 4 performs adaptive composition of the obtained enhanced image 3 and the input image 1, and the gamma conversion originally added to the input image on the obtained composite image 2 is performed again. When the post-processing unit 14 performs, the image output unit 15 generates a final output image to be output to a device device such as a printer or a display.

  The enhancement amount deriving means 21 first regards the pixel value VPij of the pixel Pij in the input image as the central visual field in Land, and regards the area belonging to the rectangular area of c pixels around it as the peripheral visual field. Then, based on the comparison target pixel selected by the comparison pixel determining means 20, a weighted weighted sum VAPij (Ar (i, j), Ag (i, j), Ab (i, j) within the surrounding range is obtained. ) Is calculated.

  In the first embodiment of the present invention, the flattening phenomenon near the edge of the output image 2 that occurs in the composite image 3 by controlling the conversion reference value VTc when converting the contrast improvement amount into the actual pixel value. It was suppressed. In the second embodiment of the present invention, the weight in the surrounding range is not controlled based on the edge information e (i, j) of the luminance y (i, j) of Pij, instead of controlling the conversion reference value VTc. By controlling the weighted sum VAPij, the flattening phenomenon near the edge in the composite image 3 is suppressed.

  In the third embodiment, first, the luminance difference average calculating means 70 uses the luminance y (i, j) in the target pixel Pij and the luminance ys (i) of Q (i, j) selected as a comparison pixel from the peripheral range. , j) is obtained a weighted weighted sum Ady (i, j) of the absolute value of the difference amount. Based on this value, the emphasis component calculating means 71 calculates the emphasis component VBRPij (BRr (i, j), BRg (i, j), BRb (i, j)) according to the equation (Equation 11). .

  This expression regards Ady (i, j) as the displacement enhancement amount in Pij, and from 1.0 of VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) obtained in 51 The amount of component to be emphasized is calculated by this Ady (i, j) and a preset enhancement coefficient. For example, no emphasis component is generated where the luminance does not change uniformly. However, for a target pixel having a contrast improvement amount greater than 1.0, a component that enhances the pixel value of the enhanced image is calculated. Conversely, for a target pixel having a contrast improvement amount smaller than 1.0, A component that suppresses the pixel value is obtained. Therefore, the enhanced image 3 has a sharper feeling where the change in the pixel value near the edge or the like is larger, and it is possible to realize the sharpness of the image in the finally obtained composite image 2.

  As described above, in the third embodiment of the present invention, the degree of sharpness of the emphasized image 3 and the composite image 2 of the image 3 and the input image 1 is reduced to some extent by incorporating such means for calculating the enhancement component. It is possible to improve the contrast in the dark part so as to preserve.

  Similarly to the first embodiment of the present invention, a configuration with multiple resolutions as shown in FIG. 3 is also possible. By doing so, the pixel value of the input image can be changed while taking advantage of the image sharpness preservation of the present invention. The contrast of the input image can be automatically improved without being greatly affected by the size of the dark part such as the dynamic range and shadow, leading to efficient image contrast adjustment.

  These processes are similarly performed by software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like according to the image processing method according to the third embodiment of the present invention. Can be realized.

  The same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the third embodiment of the present invention.

(Fourth embodiment)
Next, an image processing apparatus and an image processing method according to a fourth embodiment of the present invention will be described.

  The overall configuration diagram is configured as shown in FIG. 1 as in the first embodiment of the present invention, and the contrast improving means is configured as shown in FIG. 14 when having one peripheral range c. Similarly, the image composition means 13 is as shown in FIG. The contrast improving unit 12 includes a comparison pixel determining unit 20 that determines a peripheral pixel to be compared from a peripheral region of the target pixel Pij, and a vertical direction for the selected comparison pixel with respect to each horizontal pixel position i. The vertical direction addition means 140 for obtaining the addition value N [i] of the peripheral area, and the weighted sum VAPij (Ar (i, j), Ag (i, j), Ab (i, j)), a simple ambient averaging means 141, VAPij representing a local feature of Pij obtained in 141, and a pixel Pij (r (i, j ), g (i, j), b (i, j)) to calculate the contrast enhancement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) Improvement amount calculating means 51 for converting the contrast improvement amount VRPij obtained in 51 into an actual pixel value, VTc (rtc (i, j), gtc (i, j), btc (i, j)) for the conversion reference value calculation means 22 and the conversion reference value of 22 to improve the correction amount Composed of the pixel value converting means 23 for converting the pixel values after contrast improvement in the actual pixel Pij contrast improvement amount VRPij obtained in stage 72.

  The flow of the contrast improvement process in the image processing apparatus according to the fourth embodiment configured as shown in FIG. 14 will be described.

  The color image 1 is digitally input via the image input means 10. In the case of a color image, the component data of red, green g, and blue b can be obtained with the accuracy of the input means (a value from 0 to 255 for 8 bits). 10 normalizes this value from 0.0 to 1.0. Next, the processing means 11 performs pre-processing for returning the gamma conversion of the input image to the original linearity for the digital input image 1. The contrast improving means 12 performs a contrast improving process for improving the contrast in the dark portion on the input image 1 after the preprocessing. The contrast improving means 12 first selects a comparison target pixel in the peripheral area c. In this case, for example, as shown in the left diagram of FIG. 16, the comparison target pixel Qij is determined by thinning out in the vertical direction at a certain interval. Next, the vertical direction adding means 140 obtains an addition value N [i] of luminance values in the vertical direction at each horizontal pixel position i as shown in the left diagram of FIG. Then, the simple surrounding average means 141 obtains a weighted sum S of N [k] included in the peripheral region c of the target pixel Pij from this N [i] and divides it by the number of comparison pixels m included in the peripheral range. Thus, the average comparison target luminance Asy (i, j) is calculated. Then, the contrast improvement amount VRPij is obtained by comparing this value with the pixel values (r (i.j), g (i, j), b (i, j)) of the target pixel Pij. Furthermore, when the horizontal pixel position i moves to i + 1 as shown in the right diagram of FIG. 16, the first N [0] is first reduced from the weighted sum S obtained by Pij as shown in FIG. . Then, by adding N [7] which is newly included in the peripheral range, the sum S of luminances of the comparison pixels in the peripheral range of Pi + 1j is obtained. Similarly, by dividing this value by the number m of comparison target pixels, an average comparison target brightness Asy (i + 1, j) is obtained. In this way, the average comparison target luminance can be easily obtained by adding and subtracting the previously prepared addition value N [k] in the vertical direction. This is the target pixel Pij (r (i, j), g (i, j), b (i) of the contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)). , j)) is the comparison target concentration VAPij. By doing so, it is possible to perform a filtering process when obtaining VAPij in the peripheral range at high speed.

  When performing contrast improvement by Retinex processing, the reduction of the processing amount for obtaining VAPij has been a major point. In particular, when multi-resolution processing with multiple peripheral range areas c [k] is performed as shown in Fig. 15, the amount of processing becomes very large. However, as in the present invention, the weighted weighted sum can be simplified to a simple average in the peripheral range, and the processing amount can be reduced by devising the weighted sum calculation procedure in the peripheral range. .

  When this contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is converted into an actual pixel value, it is the same as in the first embodiment of the present invention. In addition, by controlling the value of the conversion reference value VTc as shown in Expression (8) according to the luminance y (i, j) at the target pixel Pij of the input image 1, a sudden increase in the luminance level of the shadow portion is suppressed. . At this time, it is also possible to add processing for controlling the average comparison density VAPij in the second embodiment of the present invention and controlling the emphasis component VBRPij as in the third embodiment.

  Further, in the present invention, the contrast improving means 12 is constituted by one peripheral region c as shown in FIG. 14, but it seems that the size of the peripheral region c is caused by the size of the dark part to be improved. It is. Usually, in the case of natural images, there can be shadows with various sizes, so a multi-resolution model with multiple peripheral area sizes c [s] such as Jobson et al. Is. When the present invention is applied to such a multi-resolution model, the twelve contrast improving means have the configuration shown in FIG.

  In such a case, the pixel value fluctuation of the image generated as the emphasized image 3 becomes moderate, and there is an advantage that a sharp rise in the edge portion can be suppressed.

  As described above, the fourth image processing apparatus and the image processing method of the present invention devise and simplify the process of calculating the average comparison pixel density VAPij around the target pixel with the largest processing amount in the first embodiment of the present invention. As a result, the processing time can be shortened while maintaining the contrast improvement effect while preserving edges. In particular, this speed-up effect becomes greater as the peripheral area c [k] has a plurality of peripheral ranges.

  Further, these processes are the same in software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like according to the color image processing method according to the fourth embodiment of the present invention. Can be realized.

  Further, the same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the fourth embodiment of the present invention.

(Fifth embodiment)
An image processing apparatus and an image processing method according to a fifth embodiment of the present invention will be described.

  The overall configuration diagram is configured as shown in FIG. 1 as in the first embodiment of the present invention, and the contrast improving means is configured as shown in FIG. 17 when it has one peripheral range c. Similarly, the image composition means 13 is as shown in FIG. The contrast improving unit 12 is a comparison pixel determining unit 20 that determines a peripheral pixel to be compared from the peripheral region of the target pixel Pij, and the selected comparison pixel is targeted for the selected i at the horizontal pixel position. The thinning-out vertical direction addition means 170 for obtaining the addition value N [i] in the vertical direction, and the weighted sum VAPij () of the peripheral range from the addition value N [i] of the horizontal pixel position i included in the peripheral range of the target pixel Pij. Ar (i, j), Ag (i, j), Ab (i, j)), a simple ambient averaging means 141, VAPij representing a local feature of Pij obtained by 141, and pixel Pij (r Based on the ratio of (i, j), g (i, j), b (i, j)), the contrast enhancement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j )) And a conversion reference value VTc (rtc (i, j), gtc (i, j), when converting the contrast improvement amount VRPij obtained in 51 into an actual pixel value. btc (i, j)), a conversion reference value calculation means 22, and 22 conversion bases Composed of the pixel value converting means 23 for converting the pixel values after contrast enhancement in value and based on improvement amount correction means 72 in the actual pixel Pij contrast improvement amount VRPij obtained.

  Compared with the fourth embodiment of the present invention, the present invention does not perform the addition process in the vertical direction only for the selected comparison target pixel for all the horizontal pixel positions i, The horizontal pixel position is also thinned out at a predetermined interval in advance by the comparison pixel determination means. Then, addition processing in the vertical direction of the selected horizontal pixel position i is performed.

  FIG. 18 shows an outline of the processing, and it is assumed that the pixel is thinned out to the horizontal pixel position as shown in the left diagram of FIG. In that case, in the thinning-out vertical direction addition processing 170, the vertical direction addition value N [0] at i = 0 is distributed to the thinned vertical direction addition value N [1] = N [0]. Similarly, interpolation is performed such that N [3] = N [2] and N [5] = N [4]. In this way, the amount of processing is reduced without performing addition processing in the vertical direction for all horizontal pixel positions. Further, the processing for the subsequent horizontal pixel position i + 1 is as shown in the right diagram of FIG. 17, and the leading N [0] is obtained from the weighted sum S of N [k] included in the peripheral region c of the target pixel Pij. Subtracted. Thereafter, N [7] that is newly included in the peripheral range is added, but the vertical direction addition processing at the horizontal pixel position of i = 7 is excluded by the comparison pixel determining means 20, so N [7] was interpolated by N [6] at i = 6 selected by the comparison pixel determining means 20 and closest to i = 7. Then, the interpolated N [7] is added to the weighted sum S. Thereafter, the average comparison target luminance Asy (i, j) is calculated by dividing by the number of comparison pixels m included in the peripheral range. By doing so, the calculation procedure of the weighted weighted sum can be reduced as compared with the fourth embodiment of the present invention, and the processing amount can be further reduced.

  In the case where the contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is converted into an actual pixel value, the same as in the first embodiment of the present invention. In this way, by controlling the value of the conversion reference value VTc as shown in Equation (8) according to the luminance y (i, j) at the target pixel Pij of the input image 1, the sudden increase in the luminance level of the shadow portion is suppressed. However, it is also possible to add processing for controlling the average comparison density VAPij in the second embodiment of the present invention and controlling the emphasis component VBRPij as in the third embodiment.

  Further, in the present invention, the contrast improving means 12 is constituted by one peripheral region c as shown in FIG. 17, but it seems that the size of the peripheral region c is caused by the size of the dark part to be improved. It is. Usually, in the case of natural images, there can be shadows with various sizes, so a multi-resolution model with multiple peripheral area sizes c [s] such as Jobson et al. Similar to the first embodiment of the present invention.

  Further, these processes are also performed by software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like according to the image processing method according to the fifth embodiment of the present invention. Can be realized.

  Also, the same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the fifth embodiment of the present invention.

(Sixth embodiment)
An image processing apparatus and an image processing method according to the sixth embodiment of the present invention will be described.

  The overall configuration diagram is configured as shown in FIG. 1 as in the first embodiment of the present invention. The contrast improving means is assembled with the same structure as any one of the first to fifth embodiments of the present invention. The image composition means 13 is as shown in FIG. 4 as in the first embodiment of the present invention. The point of the present invention is that the post-processing means is configured as shown in FIG. 19. In FIG. 19, 190 is an input luminance / color difference calculating means for calculating the luminance and color difference of the input image, and 191 is a composition. Luminance ratio calculation means for comparing the luminance wy (i, j) in Pij of the image 3 with the luminance y (i, j) obtained in 190 and calculating the ratio Ratio (i, j) therebetween, 192 A luminance adjusting means for adjusting the luminance wy (i, j) of the composite image 3 based on the obtained luminance ratio. 193 is between the adjusted luminance wy (i, j) and y (i, j). Color difference component correcting means for correcting the color difference of the input image from Ratio (i, j). 194 is a pixel value VWPij (wr (i, j), wg (i, j), wb (i, j)) are image regenerating means for recalculating, and 195 is a gamma converting means for performing predetermined gamma conversion.

  Contrary to the processing of the image processing method according to the sixth embodiment of the present invention configured as described above, the contrast improvement processing and post-processing means are as shown in FIG. The contrast improvement processing is performed in the same manner as in the first embodiment of the present invention. However, in the case of the first embodiment, the contrast improvement amount VRPij (Rr (i, j), Rg (i, j), Rb (i, j)) is calculated for each component according to the equations (Equation 6) and (Equation 7), but in the present invention, as in the equations (Equation 6) and (Equation 11), The improved luminance yRP (i, j) indicated by the ratio of the average comparison luminance Ay (i, j) and the luminance y (i, j) of the target pixel Pij is calculated.

  Then, yRP (i, j) is converted into luminance yRP (i, j) possessed by an actual pixel value based on yTc (i, j) obtained by the conversion reference value calculation means 22. Next, the combined luminance image 2 of the luminance yRP (i, j) of the enhanced image 3 and the luminance y (i, j) of the input image 1 is generated by the image synthesizing unit 13. In other words, in the first to fifth embodiments of the present invention, each of the three components has been handled, but here, only the luminance is processed up to the image composition. This is because the color balance of the input image 1 is maintained to some extent when the component ratios of the average comparison luminance VAPij and the input image VPij are used as in the equations (Equation 6) and (Equation 7) of the first embodiment of the present invention. However, the color balance of the input image 1 may be lost as shown in the left diagram of FIG. Therefore, in the present invention, as shown in the right diagram of FIG. 21, the color difference of the input image based on the improvement ratio Ratio (i, j) between the improved luminance wy (i, j) and y (i, j) after synthesis. Generate improved color difference wcr (i, j), wcb (i, j) by correcting cr (i, j), cb (i, j), and calculate pixel value VWPij of composite image based on these values By doing so, the color balance is maintained.

  However, the ratio (i, j) between the improved brightness wy (i, j) and y (i, j) after synthesis is directly multiplied by the color difference cr (i, j), cb (i, j) of the input image. Since the color differences wcr (i, j) and wcb (i, j) obtained in this way may be saturated, this saturation determination is performed. If wcr (i, j) is saturated, the ratio dRatio (i, j) = wcr (i, j) / cr (i, j) with respect to this color difference is changed to the improved luminance after synthesis. Let the improvement ratio Ratio (i, j) depending on the luminance of the input image. Based on this value, the luminance adjustment means 192 performs a process of calculating the improved luminance dwy (i, j) after synthesis. Next, 193 again uses the Ratio (i, j) to correct the color difference cr (i, j), cb (i, j) of the input image and combine the color difference dwcr (i, j), d wcb (i, j) is generated. Thereafter, the pixel value VWPij (wr (i, j), wg (i, j), wb (i, j)) in Pij of the composite image 3 is obtained by conversion into an actual pixel value, and the gamma conversion means 195 inputs it. By performing the gamma conversion processing originally added to the image, the output image 2 that is finally output by the image output means 15 is generated.

  As described above, in the present invention, in order to maintain the color balance in the input image, only the luminance is improved during the contrast improvement process, and the color difference of the input image is corrected based on the improvement ratio indicating the degree of improvement at that time. Thus, the color balance of the composite image is improved. In particular, it is possible to improve the contrast while maintaining the color balance of the input image by considering the saturation state of the color difference after correction, and it is possible to improve the color balance. The feature is that there is no need to consider.

  Further, these processes are also performed by software processing using a central processing unit (CPU) and a digital signal processor (DSP) used in a computer or the like according to the image processing method according to the sixth embodiment of the present invention. Can be realized.

  Also, the same can be realized by using a semiconductor chip such as an LSI chip generated according to the image processing method according to the sixth embodiment of the present invention.

DESCRIPTION OF SYMBOLS 1 Input image 2 Output image 3 Enhanced image 10 Image input means 11 Pre-processing means 12 Contrast improvement means 13 Image composition means 14 Post-processing means 15 Image output means 20 Comparison pixel determination means 21 Enhancement amount derivation means 22 Conversion reference value calculation means 23 Pixel value conversion means 30 Peripheral range initialization means 31 Completion determination means 32 Peripheral range change means 40 Coupling coefficient derivation means 41 Weighted average synthesis means 42 Output value determination means 50 Perimeter average means 51 Improvement amount calculation means 60 Edge information detection means 61 Correction Coefficient calculation means 62 Comparison amount correction means 70 Luminance difference averaging means 71 Enhanced component calculation means 72 Improvement amount correction means 140 Vertical direction addition means 141 Simple ambient averaging means 170 Thinning out vertical direction addition means 190 Input luminance / color difference calculation means 191 Brightness ratio calculation Means 192 Brightness adjustment means 193 Color Difference component correction means 194 Image regeneration means 195 Gamma conversion means 220 Image data input means 221 Image data division means 222 Histogram creation means 223 Contrast expansion means 224 Image data output means 230 CCD
231 Memory 232 Multiplication means 233 Level weight addition means H
234 Addition means 235 Speed conversion means 236 Level compression means 237 Timing control means 238 Level weight addition means L
239 Image composition unit 240 Digital imaging device 241 Processor 242 Filter 243 Display

Claims (4)

A surrounding average derivation step for deriving a weighted weighted sum of comparison target pixels belonging to a peripheral region around the target pixel in the input image;
An edge information detection step for deriving edge information in the target pixel;
A correction coefficient deriving step for deriving a correction coefficient for correcting the value of the weighted sum based on the edge information;
A correction step of correcting the value of the weighted sum based on the correction coefficient;
A contrast improvement step of deriving a contrast improvement amount of the target pixel by comparing the corrected weighted sum value and the pixel value of the target pixel;
Using the average value and the deviation amount with respect to the contrast improvement amount, the contrast improvement amount of each pixel is normalized to a value from a predetermined minimum value to a predetermined maximum value, and the normalized contrast improvement amount is the pixel value of the enhanced image And an image synthesis step for synthesizing the emphasized image and the input image with a predetermined priority,
An image processing method comprising: In the correction coefficient deriving step, in order to suppress over-emphasis when the target pixel is near the contour, the contrast improvement amount becomes a large value when the target pixel is in a flat region, and the target pixel is near the contour. In some cases, the contrast improvement amount is small.
For the edge information of the target pixel, set a correction coefficient that increases linearly and monotonously so that the higher the edge information of the target pixel, the higher the value, and the lower the edge information of the target pixel, the lower the value.
The image processing method according to claim 1, wherein the correction step corrects the value of the weighted sum by multiplying the value of each component of the weighted sum by the correction coefficient. A surrounding average deriving means for deriving a weighted weighted sum value of comparison target pixels belonging to a peripheral region around the target pixel in the input image;
Edge information detection means for deriving edge information in the target pixel;
Correction coefficient deriving means for deriving a correction coefficient for correcting the value of the weighted sum based on the edge information;
Correction means for correcting the value of the weighted sum based on the correction coefficient;
Contrast improvement means for deriving a contrast improvement amount of the target pixel by comparing the corrected weighted sum value and the pixel value of the target pixel;
Using the average value and the deviation amount with respect to the contrast improvement amount, the contrast improvement amount of each pixel is normalized to a value from a predetermined minimum value to a predetermined maximum value, and the normalized contrast improvement amount is the pixel value of the enhanced image Image combining means for combining the emphasized image and the input image with a predetermined priority;
An image processing apparatus comprising: A program for causing a computer to execute the image processing method according to claim 1.

Images