JP2005348081A - Program capable of improving contrast of shading portion, and improvement method of contrast of shading portion - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve contrast of a shading portion in a brightness image, and to reduce discontinuity of brightness. <P>SOLUTION: A shading region detector 200 detects a shading region in a brightness image, and a shading region brightness converter 300 converts brightness so that the contrast in the shading region is improved. A gradation processor 400 performs gradation process on a group of near border pixels in the border adjoining region in the shading region adjacent to the border between the non-shading region containing a non-shading portion other than the shading region and the shading region, so that a group of further improved brightness for the group of near border pixels is determined not to cause discontinuity of brightness between the border adjoining region and its near region. The brightness of the group of near border pixels is changed to the group of brightness determined by the shading portion brightness converter, resulting in the group of brightness that has been further improved. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention improves the contrast of an image of a relatively large shadow portion generated by a building such as a building and is included in a captured image such as an image obtained by photographing the ground surface from an artificial satellite or an aircraft, etc. The present invention relates to a program capable of improving the contrast of a shadow portion, and a method for improving the contrast of a shadow portion, which makes an image easy to see, that is, an image with improved visibility.

  In recent years, an image obtained by photographing the ground surface from a high place by an artificial satellite or an aircraft (hereinafter sometimes referred to as an artificial satellite) (hereinafter, this image may be referred to as a geographic image). The use of geographic image data regarding is becoming widespread. For example, generation of a map or grasping of land use status in an urban area. However, in urban areas, the shadows of high-rise buildings such as high-rise buildings (hereinafter sometimes referred to as shadows) and shadows (hereinafter sometimes referred to as shadows) are very large as a whole. There are cases where the image is dark and the contrast of the image is very small and the visibility is poor.

  Hereinafter, the shadow portion and the shadow portion may be collectively referred to as a shadow portion. Here, the shaded part is a part of the object such as a building where sunlight is blocked by other parts of the object and no longer receives sunlight. For example, it is a wall surface of a building that is not irradiated with sunlight. A shadow part is a black part projected on the ground surface etc. by the object irradiated with sunlight, such as a building.

  Thus, if the density of the shadow portion is low and the contrast of the image is small, it becomes difficult to clearly identify an object in the shadow portion, which causes a problem of visibility of the shadow portion, which is a geographical image. This is one of the problems when using data. Therefore, research for improving the visibility in the shaded area has been actively conducted. As an example, Patent Document 1 proposes a method of deleting a shadow portion from a geographic image. Specifically, in this proposal, the shooting date and time information included in the satellite image data and the height information of an object such as a building are used to irradiate the object with sunlight, the irradiation angle, and the existence of the object. Is a technique for calculating the width and length of a shadow generated by determining the presence position of a shadow part of an object such as a building based on the calculated information, and deleting the shadow part from the geographic image based on the determination Proposed.

As for other images different from geographic images, when the contrast of the image obtained by photographing the object is low, such as when the illumination on the object is bad or when the object is photographed with backlight, Several methods are known as methods for improving contrast. For example, a technique for making a density histogram uniform or a technique for converting density linearly or nonlinearly is known (see, for example, Patent Documents 2, 3, 4 and Non-Patent Document 1). Also known is a method of removing noise from an image to make the image clear (see, for example, Non-Patent Document 2).
Japanese Patent Laid-Open No. 10-269347 JP 2002-209115 A Japanese Patent Laid-Open No. 2001-118062 JP 2002-140700 A Sakai Yukiichi, “Introduction to Digital Image Processing”, CQ Publishing Co., Ltd., October 1, 2002, p. 38-49 Nobuyuki Yagi, 5 others, “Practical image processing learned in C language”, Ohm Co., Ltd., December 8, 1999, p. 54-63, 84-97

  Since the technique disclosed in Patent Document 1 proposes to remove only the shadow portion, there is no image information regarding the object on the ground on which the shadow portion is projected, and the object cannot be identified at all. Therefore, it can be applied only to the usage where the shadow portion may be deleted. Furthermore, in patent document 1, the problem of the low visibility of a shadow part is not solved at all.

  In general, in order to improve the visibility of a shadow portion in a monochrome grayscale image including a shadow portion, it is conceivable to use uniformization of a density histogram or linear transformation or non-linear transformation known per se. However, in the experiment by the inventor, it has been found that even when such a contrast improvement measure is applied to the entire black-and-white image having both the shadow portion and the non-shadow portion, the contrast of the image in the shadow portion is not improved so much. Therefore, the present inventor has come to consider that it is preferable to perform contrast improvement processing only on the shadow portion in order to improve the contrast of the shadow portion, as distinguished from the non-shadow portion. However, as a result of experiments by the present inventor, even if the already known processing such as uniformity of density histogram or linear transformation or non-linear transformation, which is a conventional contrast improvement measure, is applied only to the shaded portion, the image obtained is It has been found that the contrast is too high or too low to provide an appropriate contrast. Therefore, the present inventor has come to consider that a new contrast improving method suitable for improving the contrast of a specific image portion where a low density image called a shadow portion is most desirable.

  Further, as a result of the inventor's investigation, it has been found that when the contrast improvement process is performed only on the shadow part as distinguished from the non-shadow part, the image of the shadow part after the contrast improvement may become unnatural. In particular, it has been found that the density of an image in an area having a density close to the threshold for binarization used for detecting a shadow portion may be discontinuous with respect to an adjacent area. That is, it has been found that density discontinuity occurs at the boundary between the region detected as the shadow portion and the region that is not. Further, it has been found that this problem occurs regardless of the type of contrast improvement method applied to the shadow portion. For example, it has been found that this may occur even when conventional density histogram equalization or linear conversion is performed only on the shaded portion.

  It has been found that the visibility problem of the shadow image and the discontinuity of the density are problems that occur when the image is a monochrome grayscale image or a color image. Therefore, in this specification, brightness is used as an image value representing the brightness of both images. The brightness is the density itself when the image is a monochrome grayscale image. When the image is a color image, the lightness determined from the color image as the image value of each pixel of the image and used in combination with the hue and the saturation can be used as the lightness in the present invention. The brightness of an image is called a value that is large or high in a bright image portion, and is called a value that is small or low in a dark image portion. The same applies when discussing density instead of image brightness.

  Therefore, an object of the present invention is to generate a brightness area close to a threshold for binarization and a neighboring area, which occurs when brightness contrast improvement processing is performed on a shadow area in distinction from a non-shadow area. It is to provide a method for improving the contrast of a shadow part and a program capable of improving the contrast of a shadow part, which can reduce discontinuity of brightness occurring between them and unnaturalness of an image caused thereby.

  Another object of the present invention is to improve the shadow contrast more effectively than the conventional contrast improvement method, and further, a brightness region close to the threshold for binarization, which occurs after the shadow contrast improvement, To provide a method for improving the contrast of a shadow part and a program capable of improving the contrast of a shadow part, which can reduce the lightness discontinuity occurring between the adjacent areas.

  In order to achieve the above object, the present invention provides a value for improving contrast of brightness of a plurality of pixels in a boundary adjacent region adjacent to a region detected as a non-shadow portion among regions detected as a shadow portion. Without changing suddenly, gradation processing is applied to the brightness of the pixels in the border adjacent area, and as the brightness of the pixels in the border adjacent area is moved away from the boundary, the original brightness conversion is performed. The brightness is changed from the brightness close to the previous brightness to the brightness close to the brightness after the brightness conversion.

  More specifically, the program capable of improving the contrast of the shadow portion according to claim 1 includes a shadow portion region detecting means for detecting a shadow portion region where the shadow portion in the brightness image exists, and the detected shadow portion. The brightness for improving the contrast of the shaded area is determined according to a predetermined brightness conversion rule with respect to the brightness of a plurality of pixels in the partial area, and the brightness of the plurality of pixels is converted to the determined brightness. A group of pixels in the vicinity of the boundary in the boundary adjacent region located adjacent to the boundary between the shaded region brightness conversion means and the non-shadowed region where the non-shadowed portion exists in the shaded region in the brightness image. The computer is made to function as gradation processing means for executing gradation processing on the computer. Further, the gradation processing determines a group of further improved brightness values for the group of boundary neighboring pixels, and sets the brightness values of the group of boundary neighboring pixels to a group of groups determined by the shadow area lightness conversion means. It is the process which changes to the improved brightness of the said group instead of the brightness. Further, the improved brightness of the group is obtained by converting the brightness of the shaded area of each boundary neighboring pixel as the position of each of the neighboring pixels of the group moves away from the boundary toward the inside of the shaded area. The lightness close to the lightness before being determined by the means changes from the lightness close to the lightness before the lightness to the lightness close to the lightness after being determined by the shade area lightness conversion means of each of the pixels near the boundary.

  Here, the lightness is an image value representing the brightness of the image, and is the density itself when the image is a monochrome grayscale image. When the image is a color image, the lightness that is determined from the color image as the image value of each pixel of the image and is used in combination with hue and saturation can be used as the lightness. According to the above invention, after determining the brightness for improving the contrast with respect to the shadow area of the image, the brightness of the group of pixels near the boundary in the boundary adjacent area in the shadow area is determined by gradation processing. As the respective positions of the pixels in the vicinity of the boundary move away from the boundary toward the inside of the shaded area, the values near the brightness before the brightness is determined by the shaded area lightness conversion means of the pixels in the vicinity of the boundary. The value is changed so as to change to a value close to the lightness after being determined by the shaded area lightness conversion means of each boundary vicinity pixel. Therefore, the brightness of the pixels in the shaded area near the boundary between the shaded area and the non-shadowed area is close to the brightness before the brightness is determined by the shaded area brightness conversion means. The brightness of this pixel is close to the brightness determined by the shadow area brightness conversion means. Therefore, the brightness does not change abruptly in the boundary adjacent region of the shadow region. In this way, the lightness discontinuity is reduced between the portion close to the binarization threshold and the vicinity thereof. The above invention has an advantage that can be applied to a color image.

  The program capable of improving the contrast of the shadow portion according to claim 2 is a computer as boundary adjacent region dividing means according to claim 1, wherein the boundary adjacent region is divided into a plurality of gradation regions depending on a distance from the boundary. Is made to function further. Further, the gradation processing includes, for each of a plurality of pixels in each gradation area, the brightness of each pixel before the brightness is determined by the shadow area brightness conversion means, and the shadow area brightness conversion means. For each pixel in the gradation area, using the brightness of each pixel determined by and the weight for the gradation area determined in advance depending on the distance from the boundary to the gradation area. The further improved brightness is determined. As a result, the brightness of the pixel in the border adjacent region is determined based on the lightness close to the brightness of the pixel before the lightness is determined by the shadow region lightness conversion means in accordance with the position of the pixel in the shadow portion. The gradual change to the lightness close to the lightness of the pixel determined by the region lightness conversion means can be realized by a simple process.

According to a third aspect of the present invention, there is provided a program capable of improving the contrast of the shadow portion according to the first or second aspect, and a histogram detection means for detecting a lightness histogram relating to a plurality of pixels in the detected shadow portion region. The computer is further caused to function as target histogram determination means for determining a target brightness histogram representing an improved contrast related to the shaded area from the brightness histogram.
Further, the predetermined lightness conversion rule executed by the shadow area lightness conversion means is that the improved lightness histogram related to the shadow area is substantially equal to the target lightness histogram. Is a conversion rule for converting the brightness of a plurality of pixels. Further, the target histogram determining means determines a central region of the lightness histogram to be expanded in a range from a predetermined maximum lightness to a minimum lightness for the lightness image based on the lightness histogram. Determining means; central area enlarging means for enlarging the central area of the brightness histogram into an enlarged histogram distributed in a range from the maximum brightness to the minimum brightness; and smaller than the determined central area of the brightness histogram A histogram lower region for lightness and a histogram upper region for lightness greater than the central region are combined to convert to a combined histogram in which pixels are distributed in the lightness range from the minimum lightness to the maximum lightness, and generated by the conversion The combined histogram is converted by the central area converting means. And upper and lower region growing means for generating the target intensity histogram by combining with the expanded histogram generated, in which comprises a.

  As a result, the target histogram after improvement of the shadow area can be determined based on the brightness histogram of the shadow area, and the brightness contrast of the shadow area can be improved to become this target histogram. Furthermore, the central area of the lightness histogram of the shadow area before lightness conversion can be expanded so that it is distributed from the minimum lightness to the maximum lightness that the image can have. The area and the upper area of the histogram are combined to convert to a combined histogram in which pixels are distributed in the range from the minimum brightness to the maximum brightness, and the histogram is combined with the enlarged histogram to obtain the target histogram. The brightness distribution range can be expanded from the minimum brightness to the maximum brightness without removing the information of the pixels having the brightness belonging to the original lower histogram area and the upper histogram area, and the shadow area can be expanded. It becomes possible to improve brightness contrast. Furthermore, the effect described above with respect to at least one of claims 1 to 3 is also produced with respect to the shaded portion with improved contrast.

  According to a fourth aspect of the present invention, there is provided the program capable of improving the contrast of the shadow portion according to any one of the first to third aspects. The shaded area brightness conversion means determines brightness for improving the contrast of the shaded area included in each of the plurality of blocks. Thereby, the contrast of a shadow part area | region can be improved according to a block for every block.

According to the present invention, a portion close to the binarization threshold and its vicinity, which are generated when the contrast improvement processing is performed on the detected shadow portion region, distinguished from the non-shadow portion region detected from the brightness image, Lightness discontinuities that occur during
Furthermore, in a desirable aspect of the present invention, the contrast of the shadow portion can be improved more effectively than the conventional contrast improvement method, and further, the portion near the binarization threshold value and the vicinity thereof caused by the improvement of the contrast of the shadow portion. Lightness discontinuities that occur in between can be reduced.

Embodiments of a program capable of improving the contrast of a shadow portion and a method for improving the contrast of a shadow portion according to the present invention will be described below in detail with reference to the drawings.
FIG. 1 is a schematic block diagram of one embodiment of an apparatus capable of improving the contrast of a shaded portion of an image according to the present invention. Reference numeral 1 denotes the whole of one embodiment of the apparatus. The apparatus 1 includes, for example, a processing device 10 realized by a personal computer or a workstation, and a RAM (Random Access Memory) (not shown) used as a main memory and an auxiliary storage device (not shown) such as a magnetic disk storage device. And the input / output device 30. The input / output device 30 includes an input device 31 including a pointing device such as a keyboard and a mouse, and an output device such as a display device 32 such as a CRT display device or a printer 33. The input device 31 is used for inputting parameters and starting commands. The display device 32 or the printer 33 is used to display or print an image for which the contrast of a shadow portion should be improved or an image generated from the image with an improved contrast of the shadow portion. When data is stored in the storage device 20, it is determined in advance for each piece of data whether the data is stored in a RAM (not shown) or an auxiliary storage device (not shown).

  The program capable of improving the contrast of the shadow portion is stored in the storage device 20 and is executed by being incorporated in the processing device 10, but for the sake of easy understanding in the figure, the program 40 and a plurality of modules constituting the program 40 are processed. It is described inside the block showing the device 10. The program 40 capable of improving the contrast of the shadow portion includes a color format conversion unit 100, a shadow region detection unit 200, a shadow region brightness conversion unit 300, a gradation processing unit 400, and a color format reverse conversion unit 500. including. The shadow area brightness conversion unit 300 includes a shadow area histogram detection unit 600, a target histogram generation unit 700, and a brightness conversion unit 800. The gradation processing unit 400 includes modules called a gradation region determination unit 900 and a gradation processing execution unit 950.

  When each module in the program 40 is executed, the processing apparatus 10 has a functional block for converting color image information, a functional block for detecting a shadow area, and a shadow so as to improve the contrast of the shadow area. Function to execute a gradation process to prevent a sudden change in brightness on the border adjacent area in the shadow area located at the boundary between the function block that converts the brightness of the partial area and the non-shadow area The computer is operated as a plurality of functional blocks called blocks and functional blocks for inversely converting color image information. Therefore, a plurality of functional blocks corresponding to each module are realized by these modules of the processing apparatus 10 and the program 40. Therefore, the processing device 10, the storage device 20, the input / output device 30, the color format conversion unit 100, the shadow region detection unit 200, the shadow region brightness conversion unit 300, the gradation processing unit 400, and the color format inverse conversion. The part 500 relates to the present invention. One embodiment of the apparatus capable of improving the contrast of the shadow portion of the image is realized.

  The program 40 realizes one embodiment of a program according to the present invention that can improve the contrast of a shadow portion, and is recorded on a recording medium or stored in the storage device 20 via a network and is processed. 10 is executed. The program 40 can be recorded on a recording medium or sold via a network. The procedure in which the processing device 10 executes the program 40 to improve the contrast of the shadow portion realizes one embodiment of the method for improving the contrast of the shadow portion according to the present invention.

  The storage device 20 stores in advance color image data 21 that is a target for improving the contrast of the shadow portion, and the color image data 21 is processed by the program 40 to obtain brightness data (hereinafter referred to as improvement target brightness data). 22), saturation data 22A, hue data 22B, binary image data 23A, shadow area data 23B, actual histogram data 24A, target histogram data 24B, brightness conversion data 25, and contrast improvement. Post-brightness data 26, gradation area data 27, post-gradation lightness data 28, and improved color image data 29 are generated and stored in the storage device 20. In the present embodiment, it is assumed that the color image data 21 is color multi-valued image data unless otherwise specified. More specifically, in the present embodiment, as an example of an image including a shadow portion, a color geographic image obtained by photographing the ground surface from an altitude flying object such as an artificial satellite or an aircraft is used.

  The outline of the processing of the program 40 is as follows. The color image data 21 representing the geographic image is converted to generate improvement target lightness data 22, saturation data 22A, and hue data 22B. From the improvement target lightness data 22, a shadow part region including a shadow part in the image is detected, and shadow part region data 23B representing the shadow part region is generated. Actual histogram data 24A relating to the brightness of a plurality of pixels included in the shaded area indicated by the generated shaded area data 23B is detected, and target histogram data 24B with improved contrast is obtained from the actual histogram data 24A. decide. Lightness conversion data 25 for converting the lightness of the plurality of pixels in the shadow area is generated depending on the detected actual histogram data 24A and the determined target histogram data 24B. Based on the lightness conversion data 25, the lightness of the plurality of pixels in the shaded area is converted to generate post-contrast lightness data 26.

  A plurality of gradation areas to be subjected to gradation processing are determined, which are located in a boundary adjacent area adjacent to the boundary with the non-shadow area in the shadow area indicated by the shadow area data 23B, and each gradation area is determined. The gradation area data 27 for designating is generated. For example, the plurality of gradation areas are determined to be located at different distances from the boundary toward the inside of the shadow area in the boundary adjacent area. In accordance with the weight determined in advance corresponding to the plurality of gradation areas specified by the gradation area data 27, the brightness of the border adjacent area in the brightness data after contrast improvement 26 is converted, and the brightness after gradation processing is converted. Data 28 is generated. The post-gradation lightness data 28, the saturation data 22A, and the hue data 22B are combined to generate improved color image data 29.

  FIG. 2 is a diagram illustrating an example of a geographic image indicated by the improvement target brightness data 21. This geographic image is an actual geographic image obtained by photographing the ground with an artificial satellite. In this image, there are a plurality of buildings, roads exist at a plurality of locations, and a plurality of automobiles exist on the road. In the figure, sunlight is directly radiated where it is not a shaded part, and the brightness is very high. On the other hand, a shaded part with a low brightness exists in many places on the wall surface of the building or on the road. The contrast of lightness is so small that the inside of the shaded part is hardly discernable. As a whole image, the brightness distribution is very wide, the shadow portion is in a very low contrast state, and the visibility of the shadow portion is low.

  FIG. 30 is a diagram for explaining the linear conversion of the brightness histogram according to the prior art and its problems. In the linear conversion, the brightness distribution range [a + 1, b−1] of the brightness histogram is converted into an enlarged brightness distribution range [a ′, b ′] as shown in FIG. This is a technique for enlarging contrast. The lightness i (a + 1 ≦ i ≦ b−1) of an arbitrary pixel is converted into lightness i ′ (a ′ ≦ i ′ ≦ b ′). For example, as shown in the brightness histogram of FIG. 4B, when the brightness range of the image is substantially within the brightness distribution range [a + 1, b−1], a ′ = 0, b ′ = MAX ( In the case of (maximum brightness), the converted brightness range [a ′, b ′] is the maximum brightness distribution range [0, MAX] as shown in the histogram of FIG. Contrast is improved.

  FIG. 31 is a diagram schematically illustrating a lightness histogram of an image including a shadow portion and a bright non-shaded portion and a lightness histogram of an image obtained as a result of performing linear conversion on the image according to the related art. FIG. 6A is a diagram schematically showing a brightness histogram for an image (for example, FIG. 2) including a shadow portion and a bright non-shadow portion, and FIG. It is a figure which shows the example of the brightness histogram obtained as a result. As can be seen from FIG. 2A, in the image illustrated in FIG. 2, the minimum brightness value a + 1 of the shadow portion is close to the minimum brightness value 0 that the image can take and the maximum brightness value of the non-shadow portion is obtained. The value b-1 is also close to the maximum brightness value MAX that the image can take. Even if such an image is subjected to linear transformation, the brightness distribution range of the brightness histogram obtained is larger than the original brightness distribution range shown in FIG. 31A, as shown in FIG. It is not enlarged so much. Therefore, for the image as shown in FIG. 2, it is difficult to improve the contrast of the shadow portion by the conventional linear conversion.

Further, in order to expand the dynamic range, the brightness distribution range [a + 1, b−1] to be linearly converted is limited to a part of the brightness range in which the pixels in FIG. It is also conceivable to convert the lightness of only the pixels within the range to the maximum lightness range [0, MAX]. In this case, the lightness pixel whose lightness i is i ≦ a + 1 has a lightness after processing. It becomes 0 and the phenomenon of crushing occurs. In addition, a pixel of brightness that satisfies b-1 ≦ i has a brightness of MAX after processing, and a phenomenon of overexposure occurs. There is a problem that information on the part is lost when overexposure occurs.
On the other hand, with the conventional uniformity of the brightness histogram, the obtained image has too high brightness contrast, and a lot of noise is generated, resulting in an image that is difficult to see.

  In the present invention, in order to improve the visibility of a shadow part with a low contrast included in the image as shown in FIG. 2, the shadow part area where the shadow part exists is detected, and the brightness of the pixels in the shadow part area is detected. Is converted so as to improve the contrast of the shadow portion. In the present embodiment, a method is adopted in which the entire image is divided into two areas, a shadow area and a non-shadow area, and a process for improving the contrast is performed on the shadow area. However, depending on the image, the brightness distribution of the pixels in the shaded area may vary greatly depending on the location. In that case, the shadow area may be divided into a plurality of blocks, and processing for improving the contrast of the shadow area in each block may be performed for each block.

  Returning to FIG. 1, the program 40 first executes the color format conversion unit 100. FIG. 3 is a diagram showing an RGB color model that defines a color image with color image information relating to the three primary colors R, G, and B. The color image data 21 is assumed to be data having color image information regarding the three primary colors R, G, and B defined by the RGB color model shown in FIG. Here, it is assumed that R, G, and B have values ranging from 0 to 1. The color format conversion unit 100 converts the RGB color image information of the color image data 21 into color image information in the HIS space consisting of hue (H), lightness (I), and saturation (S), and the lightness data to be improved. 22. Saturation data 22A and hue data 22B are generated. The lightness I of the HIS is sometimes called lightness L.

  FIG. 4 is a diagram showing a bihexagonal color model in which color image information is defined. In this embodiment, color image information defined by the bihexagonal pyramid color model shown in FIG. 4 is used as color image information in the HIS space. Here, H is a hue and takes a value of 0 to 360 degrees. I is lightness, and takes a real value from 0 to 1. S is saturation and takes a value of 0 to 1. In the following, FIG. 2 is assumed to be a geographic image shown by an example of the improvement target brightness data 22 generated by the color format conversion unit 100. The lightness I represents the brightness component of the color image and matches the lightness of the monochrome image generated from the color image information. It should be noted that other color models can be used in place of the color image information in the HIS space shown in FIG. Further, the color image data 21 itself may be data having color image information in the HIS space shown in FIG. In that case, needless to say, the color format conversion unit 100 need not be executed. In the present invention, the lightness may be information indicating the brightness of the image, and the type of the color model is not limited. Regarding the bihexagonal pyramid color model and the algorithm of RGB → HIS conversion, see, for example, Masahide Hirabayashi, “Latest Program Dictionary Volume 2 in C Language”, Technical Reviewer, Inc., November 27, 1992, p. 104-110 (hereinafter referred to as references).

  Returning to FIG. 1, the program 40 next executes the shadow area detection unit 200. FIG. 5 is a schematic flowchart of the shadow area detection unit 200. The shadow area detection unit 200 includes modules called a binarization unit 210 and a noise removal unit 220, and these modules are sequentially executed in this order. The binarization unit 210 binarizes the improvement target lightness data 22 using an appropriate binarization threshold value, and converts the binarized image data 23 </ b> A having pixels corresponding to the pixels of the improvement target lightness data 22. That is, a pixel having a lightness smaller than the binarization threshold is detected as a pixel belonging to the shaded area where the shaded part exists, and the value of the pixel of the binary image data 23A corresponding to the pixel is set to 1. A pixel having a lightness greater than the binarization threshold is detected as a pixel belonging to a non-shadow portion area where a non-shadow portion exists, and the value of the pixel in the binary image data 23A corresponding to the pixel is set to zero.

  The binarization unit 210 can determine the threshold value from the improvement target lightness data 22 using any of various known techniques such as a mode method, a discriminant analysis method, a P tile method, and a differential histogram method. The appropriate method for determining the threshold value depends on the improvement target lightness data 22, and an appropriate method may be selected in accordance with the improvement target lightness data 22. In general, when a discriminant analysis method is used, it is easy to distinguish a shadow area from a non-shadow area. Also in this embodiment, discriminant analysis is used in specific processing. In the discriminant analysis method, when the brightness histogram of the entire image is divided into two classes (classes) at a certain value ft, the calculation result of inter-class variance / (class 1 intra-class variance + class 2 intra-class variance) This is a method of determining ft as a threshold value when s is maximum.

  FIG. 6 is a diagram illustrating an example of an image represented by the binary image data 23 </ b> A generated from the image illustrated in FIG. 2 by the binarization unit 210. If the binarization unit 210 simply binarizes with a threshold value and tries to detect a shadow area, even a very small area is detected, and the detected shadow area becomes noise and is difficult to see. . For example, as shown in this figure, a small image portion such as a part of the boundary line of the road or a part of the boundary line of the roof of the building is also detected. Such an image portion is a partially dark portion existing in a bright region other than the shaded portion in the original geographic image. These are not image portions showing shadow portions, but if they are in the shadow portion region, these image portions are noises when viewed from the shadow portion region. Furthermore, a bright small image may be detected when a bright small object is present in the shaded area. Such an image should be detected as a shaded area, but is detected as a non-shadowed area, so that it is a noise part when viewed from the shaded area. Or the boundary of a shadow part area | region may become jagged.

  The noise removing unit 220 (FIG. 5) uses a black small image, a thin line, or a bright small image to be detected as a shaded portion or a jagged outline, as noise. Remove. Similarly, the noise removing unit 220 removes white small images, thin lines, and the like, which are considered not to be non-shaded parts, as noise. Known examples of noise removal include small area removal processing such as expansion / contraction processing and labeling processing (see, for example, Non-Patent Document 2).

  FIG. 7 is a schematic flowchart illustrating an example of processing of the noise removing unit 220. First, the binary image data 23A is read (step S221), and the number of expansions and contractions α and the small area size β are input from the input device 31 by the user (step S222). Next, the non-shaded area is subjected to α times for each pixel in the order of expansion → shrinkage → shrinkage → expansion. That is, the α-time expansion process (step S223), the α-time contraction process (step S224), the α-time contraction process (step S225), and the α-time expansion process (step S226) are executed so as to expand or contract the non-shadow region. To do. By performing the above processing, the elongated black or white region that becomes noise disappears from the binary image data 23A, and the jagged contour becomes a smooth contour. For example, the value 2 can be used as α. As shown in FIG. 7, the total number of expansion processes (4 in this case) and the total number of contraction processes (4 in this case) need to be the same. Next, the noise removing unit 220 performs processing for removing a region having an area of β or less to remove a minute region. That is, a black area having a size of β or less is converted into a white area (step S227), and a white area having a size of β or less is converted into a black area (step S228). As β, for example, 800 pixels can be used. In the present embodiment, the process of converting the white area into the black area (step S228) is performed after the process of converting the black area into the white area (step S227), but the order may be reversed. The above processing result is stored in the storage device 20 as the shadow area data 23B (step S229).

  In the above, the expansion / contraction process is performed on the non-shadow region, but if it is done as such, pixels in the region that appears to be a shadow region when visually observed may not be detected as pixels belonging to the shadow region, It is rare that pixels in a region that appears to be a non-shadow portion when visually observed are not detected as pixels belonging to the non-shadow portion region. For this reason, the bright part in an image looks clear, and an image looks clear. On the contrary, if the expansion / contraction process is performed on the shadow area, that is, if the expansion / contraction process is performed so as to expand and contract the shadow area, the reverse phenomenon occurs and it belongs to the non-shadow area when visually observed. There are cases where a visible pixel is detected as a pixel belonging to the shaded area, and the image looks unclear compared to the case of the above embodiment.

  As for the order of expansion and contraction, as shown in FIG. 7, it is better to perform the expansion first. As a result, some of the pixels that are visually recognized to belong to the shaded area may be detected as pixels belonging to the non-shadowed area, but conversely, if they belong to the non-shadowed area, The visible pixel is almost certainly detected as a pixel belonging to the non-shaded area. Since the non-shaded portion of the image after such processing is clearly visible, it is easy to see as an image and is clear when viewed from the user. When the order of expansion and contraction is performed in the reverse order of the case of FIG. 7, pixels that can be visually observed to belong to the shaded area are almost detected as pixels that belong to the shaded area. A part of pixels that appear to belong to a region may be mistakenly detected as pixels belonging to a shaded region. In this way, when the expansion is performed first as in the former case, the obtained image becomes an easy-to-view image for the user as a whole.

  It may be desirable that the value of β be different between the non-shadow area and the shadow area. For example, β = 500 pixels in the non-shadow area, and β = 1500 pixels in the shadow area. In this way, if β in the shadow area is made larger than β in the non-shadow area (for example, 2 to 3 times), the shadow area is considered to be relatively large dust compared to the non-shadow area. This is advantageous in that the processed image can be seen with less noise when viewed from the user. On the other hand, in the non-shaded area, a relatively large area that is removed in the shaded area is not deleted, so that the resolution of the image looks high in the non-shaded area, and the image looks clearer to the user. There is an advantage.

  Each pixel of the shadow area data 23B corresponds to one pixel of the improvement target brightness data 22, and indicates whether or not the corresponding pixel in the improvement brightness data 22 is located in the shadow area. Value image data. The value of each pixel in the shadow area data 23B is 1 in the area where the shadow area exists, and 0 in the other areas, but the pixel value in the shadow area data 23B is opposite to these. It may be determined to have a value of.

  FIG. 8 is a diagram illustrating an example of the shadow area represented by the shadow area data 23 </ b> B obtained after the processing of the noise removing unit 220. The area shown in black is that area. As can be seen from the figure, only a relatively large shadow portion such as a building is shown, and the noise portion included in FIG. 6 is removed. In the present invention, in order to improve the contrast of the shadow portion, the brightness conversion is performed on the shadow portion area. The lightness conversion is performed on the pixels included in the shaded area shown in black in FIG. 8 in the image of FIG.

  Returning to FIG. 1, the program 40 then executes the shaded area brightness conversion unit 300. The shadow area brightness conversion unit 300 executes the shadow area histogram detection unit 600. The shadow area histogram detection unit 600 detects a lightness histogram for a plurality of pixels in the shadow area, and stores actual histogram data 24A representing the detected histogram in the storage device 20. In addition, the pixel which does not belong to a shadow part area | region among the pixels in an image is not processed in this Embodiment. This is because the non-shaded part is an image that is already easy to see because it is a portion irradiated with sunlight. In this embodiment, the processing time can be shortened by performing the brightness conversion only on the shaded area.

  The shadow area lightness conversion unit 300 then executes the target histogram generation unit 700. The target histogram generation unit 700 determines a target histogram after improving the contrast of the pixels belonging to the shaded area, generates target histogram data 24B representing the determined target histogram, and stores it in the storage device 20. FIG. 9 is a diagram for explaining the outline of the processing of the target histogram generation unit 700. In FIG. 9A, f (i) is a function that schematically represents the histogram represented by the actual histogram data 24A, and is a function that indicates the number of pixels having the lightness i. In the figure, A indicates the lower region of the histogram, B indicates the central region, and C indicates the upper region.

  In the present embodiment, the central area B of the real histogram is expanded so that the image has a brightness in the range from the minimum brightness 0 to the maximum brightness MAX that the image can take, as shown in FIG. i). At this time, according to the prior art, a plurality of pixels having lightness belonging to the lower area A and the upper area C of the histogram shown in (a) have a lightness of 0 or MAX as shown in FIG. Each is converted. If this is the case, the brightness i satisfying the condition i ≦ a + 1 becomes 0 after processing, and a phenomenon of collapse occurs. Further, the lightness i satisfying the condition b-1 ≦ i has a lightness of MAX after processing and a phenomenon of overexposure occurs. There is a problem that when the overexposure or collapse occurs, the information in that part is lost. In order to eliminate such a problem, in the present embodiment, these areas are combined, and the range of the minimum brightness 0 to the maximum brightness MAX, as shown in FIG. Is converted to a histogram in which pixels are distributed (in the present specification, this histogram is referred to as a combined histogram), and the enlarged histogram g (i) and the combined histogram are combined to obtain a target histogram h (i ) Is generated. In the present embodiment, as an example of the combination histogram, a histogram having a substantially uniform number of pixels in the range from the minimum brightness 0 to the maximum brightness MAX is used as shown in FIG.

  As a result, the central area B of the histogram shown in FIG. 10A is expanded to the maximum range from the minimum brightness 0 to the maximum brightness MAX, and the lower area B and the upper side that are not converted at this time are converted. The target histogram h is set so that the pixels belonging to the region C belong to the combined histogram in which the pixels are combined and distributed in the range of the minimum brightness 0 to the maximum brightness MAX as shown in FIG. (i) is determined. In the present invention, the pixels belonging to the lower area A and the upper area C of the original histogram are cut out as described above, and the combined histogram in which the pixels are distributed in the range of the minimum brightness 0 to the maximum brightness MAX as shown in FIG. It may be called clipping to belong to. The lightness limit value of the lower region A and the lightness limit value of the upper region C may be referred to as a lower limit value or an upper limit value, respectively, or the respective limit values are referred to as a lower clipping value and an upper clipping value. Sometimes. Details of the processing of the target histogram generation unit 700 will be described later.

  Returning to FIG. 1, the shadow area lightness conversion unit 300 next executes the lightness conversion unit 800. The lightness conversion unit 800 first generates lightness conversion data 25. The brightness conversion data 25 is data for converting the brightness histogram of the pixels in the shaded area of the image of the improvement target brightness data 22 into an image having the histogram indicated by the target histogram data 24B. Are generated based on the actual histogram data 24A and the target histogram data 24B. Next, the lightness conversion unit 800 performs lightness conversion on the shaded area of the improvement target lightness data 22 based on the lightness conversion data 25, and the lightness histogram is substantially equal to the target histogram illustrated in FIG. 9C. As described above, the lightness data 26 after the contrast improvement is generated by converting the lightness of the pixels in the shadow area, and is stored in the storage device 20. Details of the processing of the brightness conversion unit 800 will be described later.

  Instead of updating the lightness of the pixels in the improvement target lightness data 22, the lightness conversion unit 800 stores, in the storage device 20, copy image data (not shown) obtained by copying the improvement target lightness data 22, and this copy image. What is necessary is just to determine the converted value of the brightness of the pixel belonging to the shaded area in the data and replace the brightness of the pixel with the determined converted brightness. Thereby, the image data (lightness data after contrast improvement) in which the lightness of the shadow area is converted can be generated separately from the original improvement target lightness data 22. Alternatively, copy image data of the original improvement target lightness data 22 may be generated and stored in the storage device 20, and the lightness conversion may be performed on the shadow area in the original improvement target lightness data 22.

  In any method, it is possible to obtain image data having the same content as the original improvement target lightness data 22 and image data in which the lightness of the shadow area in the original improvement target lightness data 22 is updated. Alternatively, other methods may be used as long as the same result data can be obtained. Therefore, any method may be used for the execution of the lightness conversion unit 800. Therefore, in this specification, even when the copy image data of the improvement target lightness data 22 is updated, it is easy to understand for the sake of simplification. For this reason, it may be called to update the original improvement target lightness data 22, but even in that case, the update process includes a case where a copy image of the improvement target lightness data 22 is updated.

  FIG. 10 is a diagram illustrating an example of an image represented by contrast-improved brightness data 26 obtained by performing brightness conversion on the improvement target brightness data 22 illustrated in FIG. Since the brightness of the shaded area is converted, the contrast of the shaded area is improved from the original improvement target brightness data 22 (FIG. 2), and the visibility is increased. For example, a white spot image showing a car can be seen even in a portion within a shaded portion of a road extending slightly upward to the right and a road extending downward to the right in the approximate center of the screen.

However, in some places, the change in brightness is unnatural. Some examples of unnatural areas are described below.
(Case 1) In the region extending in a band shape along the boundary between the shaded region and the non-shadowed region, exemplified by the regions 10a, 10b, and 10c, the brightness is rapidly increased, and compared with the brightness in the vicinity thereof And the brightness is discontinuous.
(Case 2) The region 10d is a region representing a part of one wall surface of the building, and in the original improvement target lightness data 22 (FIG. 2), it appears as a shaded part when viewed, but in FIG. The lightness along the horizontal path of the wall surface suddenly increases in the middle, and the lightness is discontinuous at that portion.
(Case 3) As can be seen from FIG. 2, the area 10e is an area in which the shadow of another building is projected on a part of the roof of the building. In FIG. A bright area on the roof, a black shadow area adjacent to the area, and a white area located below the area are included. The brightness is discontinuous between these black areas and the white area below them. Referring to FIG. 2, the black area in the area 10e and a lower area adjacent to the black area are areas belonging to the same shaded portion where brightness is almost continuous, similar to the area 10d. Nevertheless, in FIG. 10, the brightness of these two regions changes discontinuously.

  The region where the lightness is discontinuous as described above is a result of the lightness conversion by the shadow region lightness conversion unit 300. The areas 10a to 10c described in the case 1 are compared with FIG. 2 in detail. These areas appear to be non-shaded areas when viewed in FIG. 2, but are adjacent to the shaded areas when viewed. It can be seen that this is an area. That is, the region 10a or the like is a region having low brightness and lower than the binarization threshold, although it appears to belong to a non-shaded part when visually observed. The binarization threshold value determined from the improvement target lightness data 22 does not always come to the boundary between the shaded part when viewed and the non-shadowed part when viewed, but may be larger than the brightness of the boundary. In this case, as described above, the brightness of a part of the non-shadow portion when visually observed may be smaller than the threshold value for binarization.

  FIG. 11 is a diagram schematically showing the brightness distribution before the brightness conversion, the brightness distribution after the brightness conversion, and the brightness distribution after the gradation process in the area 10a and the vicinity thereof. In the figure, the dotted line indicates the brightness before the brightness conversion, and the broken line indicates the brightness after the brightness conversion. The solid line indicates the brightness after gradation processing described later. These line type differences also apply to FIGS. The vicinity of the region 10a or the like is a region where a non-shadow portion when viewed and a shadow portion when viewed are visible. However, the non-shaded portion when visually observed includes a portion having a lightness smaller than the binarization threshold and a portion having a lightness larger than the binarization threshold. If there is a region whose brightness is lower than the threshold for binarization in the non-shadow region when viewed, the region is detected as a region belonging to the shadow region. Other areas in the non-shadow area when visually observed are detected as non-shadow areas.

  The area 10a and the like are the former areas. Such a region is subjected to lightness conversion by the shadow region lightness conversion unit 300. As a result, the lightness of a pixel having a lightness close to the binarization threshold value has a large value as shown in FIG. Converted to lightness. For this reason, the brightness of the area such as the area 10a becomes very large after the brightness conversion. The brightness of an area that is detected as a non-shaded part area because it appears as a non-shadow part when viewed in the vicinity and the actual brightness is greater than the binarization threshold value does not undergo brightness conversion. Therefore, the brightness of the area such as the area 10a is larger than the brightness of the area that appears to be a non-shaded part when the vicinity is visually observed, and this causes an area where the brightness of the area 10a or the like becomes extremely large. When viewed, the discontinuity of brightness appears to have occurred.

  FIG. 12 shows the lightness distribution before the lightness conversion, the lightness distribution after the lightness conversion, and the lightness distribution after the lightness conversion on the wall surface of the building including the region 10d in FIG. It is a figure which shows typically the brightness distribution after a gradation process. On the wall surface of the building including the region 10d, the entire wall surface is seen as a shaded portion in FIG. However, the shaded portion when visually observed also includes a portion whose brightness varies depending on the location and a portion having a brightness smaller than the binarization threshold and a portion having a brightness greater than the binarization threshold. The threshold of binarization determined from the brightness data 22 to be improved does not always come to the boundary between the shaded portion when visually observed and the non-shadowed portion when visually observed. It happens to be included. Therefore, the two parts as described above are included in the shaded part when visually observed. If there is an area having a lightness lower than the threshold for binarization in the shadow area, the area is detected as an area belonging to the shadow area. Of the shaded area when viewed, an area higher than the threshold is detected as a non-shadowed area.

  In the area 10d, both areas are included. The former area is subjected to lightness conversion by the shadow area lightness conversion unit 300 as in the previous area 10a and the like, and as a result, the lightness of pixels having lightness close to the binarization threshold is shown in FIG. Converted to a large value brightness. For this reason, the lightness of the region having lightness lower than the threshold among the regions such as the region 10d becomes very large after the lightness conversion. The brightness of an area detected as a non-shadow area is not subjected to brightness conversion because it appears as a shadow area when viewed in the vicinity and the actual brightness is greater than the binarization threshold. Therefore, in the region 10d, the lightness of the region lower than the threshold value and close to the threshold value appears as a shaded portion when the vicinity is visually observed, but is much larger than the lightness value of the nearby region having a lightness value higher than the threshold value. Thus, the brightness becomes extremely large in a part of the region 10d, and it appears that discontinuity of the brightness occurs when visually observed.

  FIG. 13 shows the lightness distribution before the lightness conversion, the lightness distribution after the lightness conversion, and the gradation process on the roof of the building including the area 10e shown in FIG. It is a figure which shows the latter brightness distribution typically. On the rooftop of the building including the area 10e, before lightness conversion, there are an area that can be clearly seen as a non-shadow part and an area that can be seen as a shadow part when visually observed. However, as described in the case of the region 10d, the brightness varies depending on the location even in the shaded portion when visually observed, and includes a portion having a brightness smaller than the binarization threshold and a portion having a brightness greater than the binarization threshold. It is. The reason why such two portions are included in the shaded portion when visually observed is the same reason as described for the region 10d.

  If there is a region whose brightness is lower than the threshold for binarization in the shadow region when visually observed, the region is detected as a region belonging to the shadow region. Of the shaded area when viewed, an area higher than the threshold is detected as a non-shadowed area. Therefore, in the area 10e, a bright non-shadow area (first area) and a shadow area when viewed by being adjacent to the bright area are visible, but the brightness is higher than the threshold value and is detected as a non-shadow area. There are three regions: a region (second region) and a region (third region) that appears as a shaded portion when visually observed, and is detected as a shaded portion region whose brightness is lower than the threshold and close to the threshold. The brightness of the third area is increased by the brightness conversion, the brightness is higher than that of the second area, and the brightness is discontinuous between the second area and the third area.

  As described above, when the shadow area brightness conversion unit 300 performs brightness conversion for improving the contrast of the shadow area, the contrast of the shadow area is improved and the visibility is improved. In some areas 10a to 10e, etc., lightness discontinuity occurs. As already described, since such a lightness discontinuity is lower than the binarization threshold value, it is detected as a shadow region by the shadow region detection unit 200, but because the lightness is close to the threshold value, It has been found that the lightness conversion by the shadow area lightness conversion unit 300 results in conversion to a very large lightness.

  A pixel having a brightness lower than the threshold and close to the threshold is a pixel that belongs to the detected shadow area and is in an area adjacent to the detected non-shadow area (referred to as a boundary adjacent area in this specification). In the present embodiment, in order to reduce the discontinuity of the lightness after the lightness conversion, the lightness of the pixels having lightness lower than the threshold value and close to the threshold value is not changed to the lightness value after the lightness conversion. As it goes inward, a process of converting to a further improved brightness having a value that approaches the brightness after the brightness conversion (referred to as gradation process) is executed.

  Returning to FIG. 1, the program 40 next executes the gradation processing unit 400. The gradation processing unit 400 first executes the gradation area determination unit 900. The gradation area determination unit 900 generates gradation area data 27 based on the shadow area data 23B. The gradation area data 27 is data for designating a plurality of gradation areas in the boundary adjacent area for executing gradation processing.

  FIG. 14 is a diagram for explaining a plurality of gradation areas and gradation area data 27. FIG. 6A shows the value of the shaded area data 23B. In the figure, the shaded area data 23B is represented by data S (x, y) representing the value of the shaded area data 23B at the same coordinate position as the coordinates x, y of the pixel on the improvement target lightness data 22. In the present embodiment, the shadow area data S (x, y) (23B) has a value 1 in the shadow area 231 and a value 0 in the non-shadow area 232. FIG. 2B schematically shows the gradation area data 27. In the figure, the gradation area data 27 is similarly represented by data G (x, y) representing the value of the gradation area data with respect to the pixel position of coordinates x and y. As shown in the drawing, a plurality of gradation areas 271 to 274 are provided in the boundary adjacent area 270 adjacent to the non-shadow area 232 in the shadow area 231. In the present embodiment, it is assumed that the gradation stage number Gnum is 4. In the present embodiment, the gradation level number Gnum can be specified by the user. The number of gradation regions set in the boundary adjacent region 270 is equal to the number of gradation steps.

  In the present embodiment, the plurality of gradation areas 271 to 274 are generated by repeatedly executing contraction processing for one pixel on the shadow area 231. The gradation areas 271 to 274 are assigned weight values of 0.2 to 0.8 sequentially increasing as 1 / (Gnum + 1) = 0.2, and these weights correspond to the pixels in the gradation areas 271 to 274, respectively. Stored in the pixels in the gradation area data G (x, y) (27). Therefore, the gradation area data G (x, y) (27) retains data that makes it possible to identify the weight data in addition to the weight data corresponding to each pixel of the improvement target brightness data 22. Also good. For example, numbers such as numbers 2, 3,... May be given in order to the weights 0.2, 0.4, etc., and this number may be stored in each gradation area data G (x, y) (27). Data that makes such weights identifiable is also regarded as data representing weights for the present invention.

  The weight corresponding to each gradation area is determined in advance as described above, but in this embodiment, the weight is automatically determined depending on the number of gradation steps (= number of gradation areas). Yes. Further, the gradation area data G (x, y) (27) corresponds to a pixel belonging to any gradation area, and holds data representing the weight determined for the gradation area, thereby obtaining the pixel. Indicates that the pixel belongs to the gradation area, and also represents the weight for the gradation area.

  FIG. 15 is a diagram showing the existence ranges of a plurality of gradation areas determined for the shadow area (FIG. 8) indicated by the shadow area data 23B. In the figure, the area shown in gray indicates the existence range of the boundary adjacent area including a plurality of gradation areas.

  FIG. 16 is a schematic flowchart of an example of processing of the gradation area determination unit 900. In step S901, the gradation area determination unit 900 first causes the user to input the gradation stage number Gnum to be used from the input device 31. In step S902, the shadow area data S (x, y) is copied to the work data T (x, y) and the gradation area data G (x, y). The work data T (x, y) can accommodate binary data of pixels of the shadow area data S (x, y) corresponding to the position of each pixel (x, y) of the improvement target brightness data 22. It is data. At this time, the gradation area data G (x, y) (27) and the work data T (x, y) are the shadow area data S (x, y) (23) shown in FIG. The same data.

  In step S903, the counter i is initialized to 1. In step S904, contraction processing is performed on the work data T (x, y). This is the same as performing the contraction process on the detected shadow area 231 (FIG. 14A). As a result of the contraction process, an area formed by pixels that are no longer included in the shaded area 231 is used as the first gradation area 271. Accordingly, in step S905, the gradation area 271 in the gradation area data G (x, y) (27) corresponding to the pixel whose value of the work data T (x, y) has changed from 1 to 0 by the contraction process is displayed. The value i / (Gnum + 1) (0.2 at this stage) is stored as a weight in the pixel to which it belongs.

  In step S906, the counter i is incremented. In step S907, it is determined whether i> Gnum. If so, the process of the gradation area determining unit 900 is terminated. If not, the process returns to step S904, and the processes after that step are repeated. That is, processing for further contracting the working data T (x, y) by one pixel is executed, gradation areas 272 to 274 are sequentially determined, and weights 0.4, 0.6, and 0.8 are sequentially stored in the pixels belonging to the gradation areas. In this way, the non-zero weight is stored in the plurality of pixels belonging to each of the gradation areas 271 to 274 in the gradation area data S (x, y) (27), and the processing of the gradation area determining unit 900 is completed.

  As can be seen from the above, the weight for the pixels in the plurality of gradation areas is the smallest in the area where the pixel position is adjacent to the boundary between the non-shadow area and the shadow area, and the pixel position is the non-shadow area and the shadow area. It is determined so as to increase as it goes from the boundary to the inside of the shadow area. In the above description, the weight for the first gradation area 271 that is the outermost area of the shadow area is a non-zero value, but the number of gradation areas is increased by one and the weight of the first gradation area is set to 0. Then, the weights after the second gradation area 272 may be sequentially set to 0.2, 0.4,. In this case, in the first gradation area 271, as will be described later, the image of the improvement target brightness data 22 is used as it is.

  Returning to FIG. 1, the gradation processing unit 400 then executes the gradation processing execution unit 950. The gradation processing execution unit 950 executes gradation processing as a process for reducing the discontinuity in brightness at the boundary between the detected shadow area and the non-shadow area and smoothing the image at the boundary. The gradation processing execution unit 950 stores copy image data (not shown) obtained by copying the contrast-improved brightness data 26 in the storage device 20, and uses the previously generated gradation area data 27 to store the copy image data in the copy image data. The brightness value after conversion of the pixels belonging to the gradation area is determined, and the brightness of the pixel of the copy image data is updated with the determined brightness after conversion, and stored as the brightness data 28 after gradation processing. Store in device 20.

  The gradation processing execution unit 950 sets the brightness at the coordinates (x, y) of the brightness data 22 to be improved to Io (x, y) and the brightness at the coordinates (x, y) of the brightness data 26 after contrast improvement as Ic (x, y). ) If the lightness at the coordinates (x, y) of the lightness data 28 after gradation processing obtained by gradation processing is Ig (x, y), the lightness data Ig (x, y) after gradation processing Ig (x, y) (28 ).

Ig (x, y) = Ic (x, y) × G (x, y) + Io (x, y) × {1−G (x, y)} (1)
That is, for the pixels in the non-shaded area where the weight G (x, y) is 0, the post-gradation lightness data Ig (x, y) (28) is the improvement target lightness data Io (x, y). ) (22) and the weight G (x, y) is 1, the shaded region brightness data Ig (x, y) (28) is the post-contrast brightness value. It becomes equal to the data Ic (x, y) (26). For pixels in the gradation area where the weight G (x, y) is between 0 and 1, the improvement target brightness data Io (x, y) (22) and the contrast improved brightness data Ic (x, y) (26) is added using the weight (1-G (x, y)) and the weight G (x, y), respectively, and the gradation-processed brightness data Ig (x, y) (28) is determined. Yes.

  The weights for the pixels in the boundary adjacent region 270 are determined so that the weight increases as the pixel moves from the boundary between the non-shadow region and the shadow region to the inside of the shadow region. The post-processing lightness data Ig (x, y) (28) is close to the improvement target lightness data Io (x, y) (22) when the pixel is close to the boundary, and as the pixel goes into the shadow area, The lightness data Ic (x, y) (26) after the contrast improvement is approached. Therefore, the lightness data Ig (x, y) (28) after gradation processing does not change abruptly for pixels in the shaded area and close to the boundary, and the lightness discontinuity is reduced.

  FIG. 17 is a diagram illustrating an example of the brightness image represented by the brightness data 28 after the gradation process obtained as a result of executing the gradation process. This image is a gradation process generated based on the post-contrast lightness data 26 representing the lightness image with improved contrast (FIG. 10) generated from the lightness data 22 to be improved representing the lightness image illustrated in FIG. It is the brightness image which the back brightness data 28 represents. In this figure, there is no region where the brightness is discontinuous, as illustrated in FIG. That is, there is no discontinuity in lightness in regions located at the boundary between the shaded portion and the non-shadowed portion, such as the regions 10a, 10b, and 10c detected as the shaded region in FIG. The discontinuity of the brightness between the portion having the brightness higher than the binarization threshold and the portion having the brightness lower than the binarization threshold in the region that can be seen as one shaded portion when being visually observed is almost eliminated. As described above, by executing the gradation process, the change in brightness at the boundary becomes smooth, and the unnaturalness is reduced.

  For example, the regions 10a, 10b, and 10c in FIG. 10 are detected as belonging to a shaded portion because they appear as non-shadowed portions when viewed visually but have a lightness smaller than the binarization threshold as shown in FIG. Since the brightness after gradation processing of the displayed area is lighter than the brightness of the adjacent non-shaded area when viewed visually, it appears as a shaded area when viewed adjacently and is smaller than the binarization threshold value. It changes in the range up to the lightness close to the lightness after the lightness conversion is performed by the conversion unit 300. Therefore, it can be seen that the phenomenon that the lightness caused by the shadow area lightness conversion unit 300 changes suddenly is eliminated after gradation processing. Furthermore, even if gradation processing is performed, the brightness difference between the originally shaded portion and the sunny portion remains. However, there is no problem for humans because the boundary between the shadow that originally exists and the part of the sun appears natural.

  More specifically, as shown in FIG. 12, regarding the region 10d in FIG. 10, this region appears to be a non-shaded portion when visually observed, but has a lightness slightly larger than the binarization threshold. , The region that is detected as belonging to the non-shadow portion and is not subjected to the lightness conversion by the shadow portion region lightness conversion unit 300, appears adjacent to the shadow portion when viewed visually, and has a lightness smaller than the binarization threshold. And a region which is detected as a partial region and is subjected to lightness conversion by the shaded region lightness conversion unit 300. In the latter area, the lightness after gradation processing appears to be a shaded part when viewed adjacently, but since it is larger than the binarization threshold, the lightness from the lightness close to the lightness not subjected to lightness conversion by the shaded area lightness conversion unit 300 When viewed, it appears as a shaded portion, and since the brightness is smaller than the threshold value, the brightness changes to a brightness close to the brightness after the brightness conversion. Therefore, it can be seen that the phenomenon that the lightness caused by the shadow area lightness conversion unit 300 changes suddenly is eliminated after gradation processing.

  Similarly, regarding the region 10e in FIG. 10, as shown in FIG. 13, this region is adjacent to a region that appears as a shaded portion when viewed and a region that appears as a non-shadowed portion when viewed. However, the region that appears to be a shadow portion when visually observed has a lightness that is slightly larger than the binarization threshold, and thus is detected as belonging to a non-shadow portion, and is not subjected to lightness conversion by the shadow region lightness conversion unit 300. An area and an area that appears as a shadow part when viewed adjacently and has a lightness smaller than a binarization threshold and is detected as a shadow part area and subjected to lightness conversion by the shadow part lightness conversion unit 300 are included. Yes. In the latter area, the lightness after gradation processing appears to be a shaded part when adjacent to it as in the case of the area 10d. It changes from a lightness close to the lightness not received to a shaded portion when visually observed, and to a lightness close to the lightness after undergoing lightness conversion because the lightness is smaller than the threshold value. Therefore, it can be seen that the phenomenon that the lightness caused by the shadow area lightness conversion unit 300 changes suddenly is eliminated after gradation processing.

  If the number of gradation steps is small, as a result of the brightness conversion of the shadow area, only a part of the part where the brightness has risen sharply can be suppressed, and there is a part that can not suppress the rapid change in brightness. Will remain. Therefore, the number of gradation steps needs to have a certain size. However, if the number of gradation steps is increased, the following phenomenon may occur and become a problem. (a) The edge of the shadow remains dark, the portion is noticeable, and the image is difficult to see. The effect of this phenomenon becomes stronger with smaller shadows. (b) As shown in FIG. 15, the plurality of gradation areas are positioned so as to border the non-shadow area. If the number of gradation steps is large, the border becomes thick and the image looks unnatural due to the border. From the above, it is important that the number of gradation steps is not too large and not too small, and as a result of trial and error, it is found that about 4 to 6 is good. Therefore, the gradation step number Gnum is set to 4 in the present embodiment.

  Returning to FIG. 1, the program 40 then executes the color format inverse conversion unit 500. In this process, HIS → RGB conversion is performed using the brightness data 28 after gradation processing, the saturation data 22A, and the hue data 22B, and improved color image data 29 of the RGB model is generated and stored in the storage device 20. For the HIS → RGB conversion algorithm, see, for example, the aforementioned reference.

  In the following, the process of the shadow area lightness conversion unit 300 suitable for improving the brightness contrast of the shadow area will be described. As described above, the shadow region brightness conversion unit 300 first uses the shadow region histogram detection unit 600 to perform actual histogram data related to pixels in the shadow region indicated by the shadow region data 23B in the improvement target lightness data 22. After generating 24A, the target histogram generation unit 700 determines target histogram data 24B.

  FIG. 18 is a schematic flowchart of the target histogram generator 700. The target histogram generation unit 700 includes a plurality of modules including a central region determination unit 710, a central region enlargement unit 720, and an upper and lower side region enlargement unit 730, and sequentially executes these modules in this order. The target histogram generation unit 700 first executes the center area determination unit 710.

  FIG. 19 is a schematic flowchart of an example of processing of the central area determination unit 710. In the present embodiment, the lower region A is a set of pixels having a constant ratio R with respect to the total number Ns of pixels belonging to the shaded region existing in the improvement target lightness data 22 and having a lower lightness. A set of pixels on the higher brightness side having a constant ratio R with respect to the total number Ns of pixels in the shaded area is defined as the upper area C. The constant ratio R is specified in advance by the user.

  In FIG. 19, first, the brightness value n is set to an initial value 0 (step S711), and the number of pixels from the minimum brightness value 0 to the brightness value n is determined based on the function f (i) representing the obtained histogram data. It is determined whether or not the accumulated value exceeds the product Ns × R of the total number of pixels Ns in the shaded area and a certain ratio R (step S712). If not, the brightness value n is increased by 1 (Step S713) is repeated, and when it is determined in Step S712 that the integrated value of the number of pixels has exceeded the product for the first time, the value of the brightness value n at that time is set as the upper limit brightness a of the histogram lower limit area A. Setting is made (step S714). The lower limit lightness of the central area B is a + 1.

  Thereafter, in step S715, the value of the lightness value n is increased by 1, and the cumulative value of the number of pixels with respect to the value from the minimum lightness value 0 to the lightness value n is the total number of pixels Ns in the shadow area and (1-R). When Ns × (1−R) exceeds Ns × (1−R) for the first time, the brightness value n is repeatedly incremented by 1 until it is determined in Step S716 (Step S717), and when the cumulative value exceeds the product for the first time. The value 1 smaller than the lightness value n at that time is set as the lower limit lightness b of the upper area C of the histogram (step S718). The upper limit of the central area B is b-1.

  As a result, a histogram portion for the lightness from lightness a + 1 to b-1 is determined as the central region B. The maximum brightness of the lower area of the histogram is a, and the minimum brightness of the upper area C is b. Needless to say, the values a and b obtained as described above may be used as the minimum brightness and the maximum brightness of the central area C of the histogram, respectively. At this time, the upper limit brightness of the lower area A is a-1, and the lower limit brightness of the upper area C is b + 1.

  When the processing of the central area determination unit 710 is completed in this way, the central area enlargement unit 720 is executed. The central area enlargement unit 720 uses the lower limit lightness a + 1 and the upper limit lightness b-1 of the determined central area B, and the central area B shown in FIG. As shown in FIG. 9B, the center region B is expanded into an enlarged histogram g (i) distributed in the enlarged brightness range from the minimum brightness 0 to the maximum brightness MAX. Convert. Here, linear transformation is used to enlarge the central region B, but other transformations such as nonlinear transformation may be used in some cases.

  As shown in FIG. 9B, in the linear conversion in the prior art, the brightness of the pixels belonging to the lower area A of the original histogram is converted to the brightness 0 and belongs to the original upper area C. The brightness of the pixel is converted to the maximum brightness MAX of the brightness, but in the present embodiment, the linear conversion is not performed on the pixels belonging to these regions. That is, returning to FIG. 18, the target histogram generation unit 700 executes the upper and lower side region enlargement unit 730 after the execution of the central region enlargement unit 720. As shown in FIG. 9C, the upper and lower side region enlargement unit 730 distributes pixels from the minimum brightness to the maximum brightness MAX by combining these areas A and C, as shown in FIG. 9C. Conversion into a combination histogram is performed, and the combination histogram generated by the conversion is combined with the previously generated enlarged histogram g (i) to generate a target histogram h (i). As a result, the overexposure that occurs in the prior art does not occur. As described above, it can be said that the processing performed by the upper and lower side area enlargement unit 730 is a process of placing an overflowed pixel generated by linear conversion below the brightness histogram after linear conversion. In this specification, such processing may be referred to as clipping.

  FIG. 20 is a schematic flowchart of an example of processing of the upper and lower region enlargement unit 730. In step S731, the average number of pixels ave is obtained by dividing the sum of the accumulation of the number of pixels in the lower region A of the original actual histogram and the accumulation of the number of pixels in the upper region C by the total number of different brightness (MAX + 1). calculate. The average number of pixels ave is the number of pixels at each brightness when a uniform histogram is used in the brightness range from the minimum brightness to the maximum brightness as a combination histogram for the combination of the lower area A and the upper area C. To give. In step S732 and subsequent steps, the combined histogram and the enlarged histogram g (i) previously obtained by the linear transformation are combined to generate the target histogram h (i).

  First, in step S732, the brightness i is set to the minimum value 0. Thereafter, in step S733, the enlarged histogram g (i) and the average pixel number aver are added to the brightness i, and a function h (i) representing the combined histogram is calculated. Thereafter, in step S733, the lightness i is increased by 1, and the calculation in step S734 is repeated within a range where the lightness does not exceed the maximum lightness MAX. Thus, a histogram h (i) obtained by combining the previously obtained enlarged histogram and the combination histogram is obtained, and this histogram h (i) is used as the target histogram. Target histogram data 24B representing the target histogram is stored in the storage device 20.

  Note that the average pixel number aver calculated in step S731 is generally not an integer. Accordingly, the combined target histogram h (i) is not an integer. However, when the target histogram h (i) is determined in step S733, the number of pixels h (i) is determined to be an integer.

  One method for this is to calculate the average to be added according to the brightness i when adding the pixel number g (i) of the enlarged histogram and the average pixel number aver in step S733 when the average pixel number aver is not an integer. As a value of the pixel number aver, a set of integers of the maximum integer not exceeding the average pixel number aver calculated in step S731 and an integer larger than that are alternately selected and used for all the brightness i. The average pixel number aver may be adjusted according to the lightness i so that the total sum of the used average pixel number aver is equal to the total pixel number calculated by the numerator of the average calculation formula in step S731. Of course, other methods can be used. In the following, it is assumed that the number of pixels h (i) in the target histogram is determined to be an integer in this way.

  It should be noted that when the pixel number h (i) of the target histogram is left as a non-integer and brightness conversion is performed later, a non-integer average value h (i) is used by using a method similar to that described above. The number of assigned pixels may be adjusted so that the number of assigned pixels becomes an integer. Thus, the processing of the target histogram generation unit 700 is finished.

  Returning to FIG. 1, the program 40 next executes the lightness conversion unit 800. FIG. 21 is a schematic flowchart of the lightness conversion unit 800. The lightness conversion unit 800 includes a plurality of modules, a lightness conversion data generation unit 810 and a lightness conversion execution unit 830, and sequentially executes these modules. The lightness conversion unit 800 first executes the lightness conversion data generation unit 810 to generate the lightness conversion data 25. This data is data for converting the brightness of the pixels in the shaded area of the improvement target brightness data 22 so that the brightness of those pixels changes so that the actual histogram data 24A changes to the target histogram data 24B. Before describing the details of the processing of the lightness conversion data generation unit 810, how lightness conversion is performed will be described.

  FIG. 22 is a diagram for explaining brightness conversion performed based on the actual histogram data 24A and the target histogram data 24B. FIG. 6A is a diagram showing an example of actual histogram data 24A relating to pixels in the shaded area. FIG. 6B is a diagram showing an example of the target histogram data 24B. FIG. 10C is a graph showing a function f (i) representing the actual histogram data 24A shown in FIG. FIG. 4D is a graph showing a function h (i) representing the target histogram data 24B shown in FIG. In FIG. 5C, the pixels having the same brightness are shown using the same hatched pattern or the like. In FIG. 6D, the pattern used for the pixel before the lightness conversion in FIG. 5C is also applied to the pixel after the lightness conversion.

  For example, the lightness 0 of the target histogram data 24B is assigned a total of three pixels, one pixel each of lightness 0 and 1 of the actual histogram data 24A and one of the two pixels of lightness 2. Similarly, the lightness 1 of the target histogram data 24B includes the sum of the remaining one of the two pixels of lightness 2 of the actual histogram data 24A and the two pixels of the three pixels of current lightness 3. Three pixels are allocated. Similarly, the lightness 2 of the target histogram data 24B includes the sum of the remaining one of the three pixels of lightness 3 of the actual histogram data 24A and the three pixels of the four pixels of current lightness 4. Four pixels are allocated. Similarly, the lightness 3 of the target histogram data 24B includes the sum of the remaining one of the four pixels of lightness 4 of the actual histogram data 24A and the four pixels of the five pixels of current lightness 5. Five pixels are allocated. The same applies hereinafter.

  FIG. 23 is a diagram showing an example of a brightness conversion table as an example of the brightness conversion data 25 for the actual histogram data 24A and the target histogram data 24B shown in FIGS. 22 (a) and 22 (b). Hereinafter, the lightness conversion table may also be referred to as the lightness conversion table 25 by quoting the reference numeral 25. The lightness conversion data 25 includes the lightness i (25a) of the conversion source pixel, the total number f (i) (25b) of conversion target pixels having the lightness i, and the conversion destination lightness T (i) (25c) for the lightness i. ) And a plurality of conversion destination lightness T (i) for the same lightness i, of the total number f (i) of conversion target pixels having the same lightness i, each of the conversion target lightnesses T (i) is converted. And the number of pixels subject to partial conversion (25d).

  The destination lightness T (i) (25c) is the target histogram h (i), and the histogram data 24A of the original image is added in order from the lowest lightness, and the lightness assignment of each pixel at the time of lightness conversion is performed. Reconfigure. Of course, the pixels may be assigned in order from the pixel with the highest brightness. At this time, when there are a plurality of conversion destination brightness T (i) (25c) for the same conversion source brightness i, the number of partial conversion target pixels to be converted to the brightness for each conversion destination brightness is determined. I have decided. Note that when converting the lightness of the original pixels, the pixels to be converted into the respective lightness values of the conversion destination are randomly determined from the total number of lightness conversion target pixels. This determination will be described in more detail later using an example.

  FIG. 24 is a schematic flowchart of an example of processing of the lightness conversion data generation unit 810. Here, the lightness conversion table 25 illustrated in FIG. 23 is generated as the lightness conversion data 25. Hereinafter, lightness conversion data generation is performed by appropriately referring to the conversion source actual histogram data 24A shown in FIG. 22 (a) or (c) and the target histogram data 24B shown in FIG. 22 (b) or (d). The processing of the unit 810 will be described.

  In step S811, the lightness conversion data generation unit 810 first sets the lightness i to 0 and sets the variables High and Low to 0. The variable High is the maximum brightness of the original image assigned to the target brightness i, and the variable Low is the minimum brightness of the original image assigned to the target brightness i. In step S812, it is determined whether or not the brightness i is equal to or less than the maximum brightness MAX. If so, the variable Sum is set to an initial value 0 in step S813. The variable Sum represents the number of pixels of the original image that are candidates for assignment to the number of pixels h (i) at lightness i of the target histogram.

  In step S814, it is determined whether the variable Sum is smaller than the number of pixels h (i) at the brightness i of the target histogram. That is, when the pixel count h (i) is smaller than the pixel number h (i) at the lightness i of the target histogram, there is a possibility that a pixel having a lightness High in the original image may be assigned. If the result of determination in step S814 is Yes, in step S815, the number of pixels f (High) at the lightness High of the original actual histogram data 24A, in this example, f (0) is added to the variable Sum. Increase the value of the variable High by 1. In this example, the variable Sum is equal to f (0), and the variable High has the value 1. Thereafter, step S814 and subsequent steps are executed again. In step S814, it is determined whether or not the updated value of the variable Sum updated in step S815 is smaller than the number of pixels h (i) at the brightness i of the target histogram data 24B. Step S815 is repeated as long as it is smaller.

  In the case of FIG. 22 (a) or (c), the number of pixels h (0) at lightness 0 of the target histogram data 24B is 3, and the number of pixels at lightness 0, 1, 2 of the original image is 1, respectively. Since 1, 2, Sum becomes f (0) (= 1), the variable High becomes 1, and then Sum is increased to f (0) + f (1) = 2, and the variable High also has the value 2 In step S814, it is determined that Sum is still smaller than the target pixel number h (0) (= 3), and the next original pixel number f (2) is added to Sum again in step S815. Thus, the value of Sum becomes f (0) + f (1) + f (2) (= 4), and the variable High is increased to 3. The sum value 4 after the addition is larger than the target pixel number h (0) = 3. This means that pixels of lightness 0 and 1 in the original image can be assigned to the target lightness i = 0, but all pixels of lightness 2 cannot be assigned. Therefore, the process proceeds to step S816. There, the variable High is decremented by 1, resulting in a value of 2 in the present example, and the difference Delt is determined by the following equation 2.

Delt = f (High) − (Sum−h (i)) (2)
Here, (Sum−h (i)) is the total number of pixels exceeding the target pixel number h (i) in the total number Sum of allocation candidate pixels, and the difference Delt is the allocation candidate pixel Out of the maximum number of pixels f (High), the number of pixels smaller by the total number of pixels exceeding the target pixel number h (i), that is, the target brightness i of the number of pixels f (High). Represents the number of pixels that can be assigned to. For example, in the case of FIG. 22 (a) or (c), Sum is 4, h (0) is 3, and f (2) is 2, so the difference Delt is 1. That is, one of the two pixels of lightness 2 of the original image is assigned to the target lightness 0, but the remaining one is not assigned to the target lightness 0.

  In step S817, the variable Low (0 in this case) is smaller than the variable High (2 in this case), so in step S818, the variable Tmp is set to the value of the variable Low (0 in this case) In step S820, the lightness i (0 in the present example) is set as the conversion destination lightness T (Tmap) (T (0) in the present example) for the lightness Tmap (0 in the present example) of the lightness conversion table 25. That is, the lightness 0 in the shaded area is determined as the conversion destination T (0). Each time step S820 is repeated, it is determined in step S819 whether or not the variable Tmp is smaller than the variable High. When the variable Tmp is smaller, step S820 is repeated. In this way, step S820 is repeated, and the variable Tmp is increased by 1. Similarly, the lightness 0 is determined as the conversion lightness T (1) for the original lightness 1.

  As described above, when the lightness i of the original pixel is 0 or 1, the lightness of the conversion destination is determined to be lightness 0. When the lightness of the conversion destination is determined for the lightness of the conversion source in this way, the determination is made to the conversion destination lightness T (i) (25c) corresponding to the conversion source lightness i in the lightness conversion table 25. The brightness of the converted destination is stored. In step S820, the variable Tmp is further increased by 1. As a result, the variable Tmp becomes 2. Thereafter, when it is determined in step S819 that the variable Tmp is not smaller than the variable High, the process proceeds to step S821.

  In step S821, out of the number of pixels f (High) (for example, f (2) (= 2)) with respect to the brightness High of the conversion source image, the number of pixels corresponding to the number of difference Delts calculated by Expression 2 (1 in this example). Are assigned to the target brightness i. That is, T (High) = i for this pixel. That is, in the present example, the conversion destination lightness T (2) = 0 is determined for one pixel having the original lightness 2. Since the number of pixels having lightness High (2 in this example) of the original image is 2, the total number of pixels (number of pixels to be partially converted) assigned to the target lightness i (0 in this example) is 1. It becomes. This partial conversion target pixel number is stored in the lightness conversion table 25 as a partial conversion target pixel number 25d corresponding to the combination of the conversion source lightness High and the conversion target lightness i. Hereinafter, every time the number of pixels subject to partial conversion is determined, the number of pixels subject to partial conversion is stored in the brightness conversion table 25 as the number of pixels subject to partial conversion 25d.

Thereafter, in step S822, the number f (High) of pixels with high brightness of the original image is reset to the total number of unallocated pixels among the pixels with high brightness of the original image by the following equation 3.
f (High) = Sum−h (i) (3)

  In the current example, this value is equal to 1. In step S822, the variable Low is further made equal to the value of the variable High (2 in this example), and the target brightness i is increased by 1 to a value of 3. In the present example, the lightness i has a value of 1. Thereafter, the process returns to step S812. Since the number of pixels of brightness 1 in the target histogram is 3 in the case of FIG. 22B or 22D, the value of f (High) (one unassigned pixel) is used for the brightness 1 of the target. Minute) is added to the variable Sum in step S815, Sum becomes the value 1, the variable High is increased by 1, and becomes the value 3. When Step S814 is executed again, Sum (= 1) is smaller than the number of pixels h (1) (= 3) as in the present assumption. At that time, Step S815 is executed again, and the variable Sum is set to f (3). (3 in this example) is added, and Sum becomes 4. Thereafter, the variable High is increased by 1 to 4. In step S814, the variable Sum (= 4) is larger than the target pixel number h (1) (3 in the present example), so the process returns to step S816.

  In step S816, the value of variable High is decreased by 1 and returned to 3. Further, the difference Delt is calculated by the equation 2 described above. In this case, the difference Delt is 2. Thereafter, in step S817, since the variable Low is 2 and the variable High is 3, in this case, it is determined that the variable Low is smaller than the variable High. In step S818, the variable Tmp is set to the value 2 of the variable Low. In step S819, since it is determined that the variable Tmp is smaller than the variable High, in step S820, the lightness 1 as the lightness T (2) of the assignment destination for one unassigned pixel among the original lightness 2 pixels. Is determined. That is, lightness 1 is assigned to one unassigned pixel among the two pixels having original lightness 2. Thus, for the two pixels having the original lightness 2, the number of partial conversion target pixels assigned to the target lightness 0 is 1, and the number of partial conversion target pixels assigned to the target lightness 1 is 1.

  Thereafter, when the variable Tmp is increased by 1 to 3, it is determined in step S819 that the variable Tmp is not smaller than the variable High (3), and the process proceeds to step S821. In step S821, as described above, the difference Delt (value in this case) calculated in step S816 out of the original number of pixels f (High) (3 in this example) having brightness High (= 3). The lightness i (1 in the present example) is determined as the lightness T (3) of the assignee for the number of pixels of 2). The probability of selecting an assignment destination is 2/3 determined by the number of assigned pixels (in this case, 2) / the number of assignment candidate pixels (in this case, 3). In this way, lightness 1 is assigned to two of the three pixels having the original lightness 3.

  Thereafter, as described above, the number of unallocated pixels of the original pixel having the brightness of 3 is calculated by the above-described equation 3, and the number is set to the number of pixels f (High) with respect to the brightness of the original pixel of 3. The In the same manner, the lightness T (i) of the new assignment destination is determined for the subsequent lightness pixels of the original image. In the above description, the case where the variable Low is not smaller than the variable High does not occur in the determination in step S817. Such a case occurs because step S822 is executed to make the variable Low equal to the value of the variable High, and then step S815 is executed to set the value f of the number of pixels of the original pixel to which the variable Sum is not assigned. If the pixel number f (High) is smaller than the pixel number h (i) of the next target brightness, it is determined in step S814 that the variable Sum is smaller than the pixel number h (i). . In that case, the process proceeds to step S816. In that case, in step S816, the difference Delt is calculated according to Equation 2, and its value is equal to the number of pixels h (i). In such a case, in step S817, it is determined that the variable Low is not smaller than the variable High (both are equal). In this case, the process proceeds to step S821. Since the process of step S821 is as already described, it is omitted. As described above, the lightness conversion data generation unit 810 generates the lightness conversion table 25 as shown in FIG.

  Returning to FIG. 21, the lightness conversion unit 800 executes the lightness conversion execution unit 830 after the lightness conversion data generation unit 810 executes. In this process, copy image data (not shown) obtained by copying the improvement target lightness data 22 is stored in the storage device 20, and a shadow area in the copy image data is generated using the lightness conversion table 25 generated previously. What is necessary is just to determine the brightness after conversion with respect to the brightness of the pixel belonging to, and replace the brightness of the pixel with the determined brightness after conversion.

  For example, in the case of the lightness conversion table 25 in FIG. 23, when the conversion source lightness i is 0 or 1, the conversion destination lightnesses T (0) and T (1) are 0. When the lightness i of the original pixel is converted, if there are a plurality of lightness T (i) of the conversion destination for the same lightness i of the same conversion source, as described above, for each conversion lightness, The number of partial conversion target pixels to be converted to the lightness is determined and stored in the lightness conversion table 25. When converting the lightness of such pixels, the pixels to be converted into the respective conversion lightnesses are determined from the total number of pixels of the lightness conversion target pixels with the probability of the partial conversion target pixel number / the total number of lightness conversion target pixels. do it.

  For example, in the case of the lightness conversion table of FIG. 23, when the original lightness i is 2, the conversion lightness T (2) has two, 0 and 1, and the number of partial conversion target pixels for each is 1 is there. Therefore, one of the total number 2 of the conversion target pixels having the original lightness i of 2 is converted into the conversion lightness 0 and 1. For each of the two pixels with the original lightness 2, conversion destination lightness 0 and 1 is set to the number of partial conversion target pixels / probability of the total number of lightness conversion target pixels, that is, 1 / It may be determined randomly with a probability of 2. Similarly, when the original lightness i is 3, the total number of lightness conversion target pixels is 3, the conversion destination lightness has two, 1 and 2, and the number of partial conversion target pixels to be converted to each is 2 And 1. Therefore, the total of three pixels of lightness conversion target pixels may be converted to conversion destination lightness 1 and 2 at random with a probability of 2/3 and 1/3.

  In order to randomly determine the lightness conversion destinations of the two pixels having the lightness value 2, different conversion lightness values are generated by the number of partial conversion target pixels for the respective conversion lightness values, and the lightness conversion target pixels in total are generated. A group of destination lightness values equal to the total number is generated, and when a pixel to be converted with a lightness of 2 is first found, one destination lightness value is randomly extracted from this group, and that value is converted into that The brightness value of the pixel to be converted is set as the brightness value of the pixel to be converted, and the extracted brightness value of the conversion destination is also deleted from the group. What is necessary is just to determine at random.

  As can be understood from the processing of the shadow area lightness conversion section 300 described above, the value of the ratio R between the number of pixels in the lower area of the histogram of the original image or the number of pixels in the upper area and the total number of pixels is adjusted. Thus, the contrast of the image after brightness conversion can be freely adjusted. That is, as the ratio R is increased, the number of pixels included in both the lower region and the upper region increases, and the brightness of those pixels falls within the brightness range from the minimum brightness to the maximum brightness. Therefore, the contrast becomes strong and can be clearly recognized. Further, if the ratio R is reduced, the contrast is not strong, but a natural image corresponding to that. The example of FIG. 10 is an image obtained when R = 0.05.

  FIG. 25 is a diagram showing a histogram of brightness of the shadow area data 23B detected from the image of FIG. 2, and shows an actual example of the histogram schematically shown in FIG. In this histogram, it can be seen that there are many pixels at low brightness, and the brightness is distributed from the minimum brightness to the maximum brightness as a whole.

  FIG. 26 is a diagram illustrating a brightness histogram of the shaded area in the contrast-improved image shown in FIG. 10 obtained by executing the process of the shaded area brightness conversion unit 300 in the present embodiment. Compared to FIG. 25, the lightness distribution extends over the entire range from the minimum value 0 to the maximum value MAX, and the frequency distribution is not excessively concentrated on the specific lightness. As shown in the image shown in FIG. 10, this histogram reflects that the brightness contrast of the shaded area is improved and the visibility is improved.

  FIG. 27 is a diagram illustrating an example of an image obtained as a result of performing the linear transformation according to the conventional technique described with reference to FIG. 31 on the shaded area of the image of FIG. The contrast of the shaded area has increased, visibility has improved, and the outlines of cars and buildings have become easier to understand. However, the brightness change of the bright part of the building disappears, and it jumps uniformly in white, or the dark part of the road is uniformly dark, making it difficult to grasp the road structure. That is, overexposure occurs, and the visibility of the entire image is inferior to that of the image obtained in the present embodiment shown in FIG.

  FIG. 28 is a diagram showing a brightness histogram of the shaded area in the image of FIG. Compared to the histogram of FIG. 25, the brightness distribution range is the same as that of FIG. 26 in that it extends over the entire range from the minimum value 0 to the maximum value MAX, but in FIG. 28, the number of pixels at the minimum brightness and the maximum brightness. Has become extremely large. This indicates that overexposure seen in the image shown in FIG. 28 occurs.

  FIG. 29 is a diagram showing an image obtained as a result of performing histogram equalization according to the conventional technique for the shaded area in FIG. 2 for comparison. In the figure, the visibility in the shaded area is high, but the contrast is too strong and noise is conspicuous. Further, since the brightness changes extremely at the boundary between the shadow portion and the non-shadow portion, the image is unnatural. On the other hand, the image obtained by the present embodiment shown in FIG. 10 has less noise, is a more natural image, and has excellent visibility.

  Therefore, as shown in the present embodiment, by converting the lightness of the shadow region by the shadow region lightness conversion unit 300, not only can the contrast of the shadow region be improved, but an image having a natural lightness change can be obtained. be able to. In other words, as a result of performing linear transformation on the central area of the brightness histogram and uniforming the histogram on the lower area and upper area pixels of the brightness histogram, the contrast obtained by only the conventional linear transformation is obtained. It is strong and has a weaker contrast than when only conventional histogram equalization is performed, so that the contrast of the shadow part can be improved so as to have a contrast of about the middle of both conversions, and the image after conversion of the shadow part is A more natural image can be obtained. That is, the image of the shadow portion can be improved so as to achieve both the visibility in the shadow portion and the naturalness of the image.

  When the lightness conversion execution unit 830 is executed as described above, the processing of the lightness conversion unit 800 ends, and the processing of the shadow area lightness conversion unit 300 ends. Returning to FIG. 1, the program 40 executes the gradation processing unit 400 as described above after the execution of the shadow area lightness conversion unit 300.

  In the present embodiment, the lightness conversion executed by the shadow area lightness conversion unit 300 is very effective. However, there remains a problem that lightness discontinuities occur and the image looks unnatural in those areas. As described above, this problem can be improved by further executing the gradation processing unit 400 in the present embodiment.

  It goes without saying that the present invention is not limited to the above-described embodiment, and may be implemented in other forms with changes or modifications within the scope not changing the gist of the present invention. The gradation processing unit 400 used in the above embodiment is effective to be executed on the execution result of the shadow area lightness conversion unit 300 in the present embodiment, but the gradation processing unit 400 itself has been conventionally used. The present invention can be applied to an image obtained as a result of the linear transformation (for example, FIG. 27) or an image obtained by conventional histogram equalization (for example, FIG. 29). Have.

  Moreover, in the said embodiment, the shadow part was considered as one block, and it processed. However, the present invention can also be applied to the case where the brightness image to be improved is divided into a plurality of blocks. For example, when shadows with shadows with various brightness levels are included due to overlapping shadows, the contrast of dark shadows may not be improved sufficiently if the shadows are considered to belong to one block. . In such a case, it is possible to determine the brightness after conversion for each block so that the image is divided into a plurality of blocks and the contrast of the shadow area is improved for each block. Further, as described above, it is possible to adjust the value of the ratio R between the number of pixels in the lower region of the histogram of the original image or the number of pixels in the upper region and the total number of pixels for each block. It is also possible to distinguish between a block in which a partial partial area is desired to be clearly seen and a block in which a shadow partial area is desired to be viewed in a natural contrast, and a ratio R suitable for each block can be used.

  In the above embodiment, the central region of the original histogram is expanded using linear transformation, but other transformations such as nonlinear transformation and piecewise linear transformation may be used instead of linear transformation. Also. When combining the lower area and the upper area of the original histogram and using a combined histogram in which pixels are distributed over the brightness range from the minimum brightness to the maximum brightness, the histogram will be almost uniform over the entire brightness range. However, instead of this, it is not necessary to be completely uniform. Furthermore, it may not be substantially uniform, for example, it may be converted so as to be unevenly distributed in the entire brightness range.

  In the above embodiment, the contrast improvement target image is a color image. However, the present invention can also be applied to a monochrome grayscale image. In the case of a monochrome grayscale image, the hue is arbitrarily designated and the saturation is 0. And processing the lightness data of the image. Alternatively, in the case of a monochrome grayscale image, the shaded area lightness conversion unit 300 and the gradation processing unit 400 are executed using the density data as the improvement target lightness data 22 in the above embodiment, and the color format conversion unit 100. And the color format inverse conversion unit 500 need not be executed.

It is a schematic block diagram of one Embodiment of the apparatus which can improve the contrast of the shadow part of the image based on this invention. It is a figure which shows an example of the geographic image which improvement object image data shows. It is a figure which shows the RGB color model which defines a color image with the color image information regarding three primary colors of R, G, and B. It is a figure which shows the bihexagonal pyramid color model in which color image information is defined. The schematic flowchart of a shadow part area | region detection part is shown. It is a figure which shows the example of the binary image produced | generated from the image of FIG. 2 by the binarization part. It is a schematic flowchart which shows an example of a process of a noise removal part. It is a figure which shows the example of the shadow part area | region which the shadow part area | region data obtained after the process of a noise removal part represents. It is a figure for demonstrating the outline | summary of the process of a target histogram production | generation part. It is a figure which shows the brightness data after contrast improvement obtained by lightness conversion of the improvement object brightness data shown in FIG. 2 by a shadow part area | region lightness conversion part. It is a figure which shows typically the brightness distribution before the brightness conversion in the area | region 10a etc. in FIG. 10, the brightness distribution after the brightness conversion, and the brightness distribution after the gradation process. FIG. 10 schematically shows a lightness distribution before lightness conversion, a lightness distribution after lightness conversion, and a lightness distribution after gradation processing along a substantially horizontal path in the figure on the wall surface of the building including the region 10d in FIG. FIG. 10 schematically shows a lightness distribution before 0 lightness conversion, a lightness distribution after lightness conversion, and a lightness distribution after gradation processing on a rooftop of a building including the region 10e in FIG. FIG. It is a figure for demonstrating several gradation area | regions and gradation area | region data. It is a figure which shows the existing range of the some gradation area | region determined with respect to the shadow part area | region of FIG. 8 which a shadow part area | region data shows. It is a schematic flowchart of an example of a process of a gradation area | region determination part. It is a figure which shows the example of the brightness image which the brightness data after gradation process obtained as a result of performing gradation process represents. It is a schematic flowchart of a target histogram production | generation part. It is a schematic flowchart of an example of a process of a center area | region determination part. 12 is a schematic flowchart of an example of processing of an upper and lower side region enlargement unit 730. It is a schematic flowchart of a brightness conversion part. It is a figure for demonstrating the brightness conversion performed based on real histogram data and target histogram data. It is a figure which shows the example of the lightness conversion table as an example of lightness conversion data. It is a schematic flowchart of an example of a process of the brightness conversion data generation part. It is a figure which shows the histogram of the brightness of the shadow part area | region data detected from the image of FIG. It is a figure which shows the example of the histogram of the brightness of the shadow part area | region of the contrast improvement image obtained by performing the process of a shadow part area | region brightness conversion part. It is a figure which shows the example of the image obtained as a result of performing the linear transformation by a prior art with respect to the shadow part area | region of the image of FIG. It is a figure which shows the histogram of the brightness of the shadow part area | region of the image of FIG. It is a figure which shows the image obtained as a result of performing the histogram equalization by a prior art with respect to the shadow part area | region of FIG. It is a figure for demonstrating the linear transformation of the brightness histogram by a prior art, and its problem. It is a figure which shows typically the brightness histogram of the image obtained as a result of performing the linear transformation by the prior art to the image which contains the shadow part and the bright non-shadow part, and the image.

Explanation of symbols

  270 ... border adjacent area, 271 to 274 ... gradation area, f (i) ... function representing the real histogram, g (i) ... function representing the enlarged histogram after enlarging the central area of the real histogram, h (i ) ... Function that represents the target histogram.

Claims (8)

A shadow area detection means for detecting a shadow area where a shadow area exists in the brightness image;
With respect to the brightness of a plurality of pixels in the detected shadow area, the brightness for improving the contrast of the shadow area is determined according to a predetermined brightness conversion rule, and the brightness of the plurality of pixels is determined A shaded area lightness conversion means for converting the lightness to
A gradation for performing gradation processing on a group of pixels near the boundary in the boundary adjacent region located adjacent to the boundary with the non-shadow portion region where the non-shadow portion exists in the shadow portion region in the brightness image Processing means,
Function as a computer
The gradation processing determines a group of further improved brightness values for the group of boundary neighboring pixels, and converts the brightness values of the group of boundary neighboring pixels to the group of brightness values determined by the shadow area lightness conversion means. Instead, it is a process of changing to a further improved brightness of the group,
The further improved brightness of the group is obtained by the shade area lightness conversion means of each boundary vicinity pixel as the respective positions of the group of boundary vicinity pixels move away from the boundary toward the inside of the shadow area. It changes from a lightness close to the lightness before being determined to a lightness close to the lightness after being determined by the shadow area lightness conversion means of each boundary neighboring pixel.
A program that can improve the contrast of shadows. Boundary adjacent region dividing means for dividing the boundary adjacent region into a plurality of gradation regions depending on a distance from the boundary;
As a computer,
The gradation processing is determined for each of a plurality of pixels in each gradation area by the brightness of each pixel before the brightness is determined by the shadow area brightness conversion means and the shadow area brightness conversion means. Using the brightness of each pixel that has been determined and the weight for the gradation region that is predetermined depending on the distance from the boundary to the gradation region, the further for each pixel in the gradation region Determine improved brightness,
The program capable of improving the contrast of a shadow portion according to claim 1. Histogram detection means for detecting a brightness histogram relating to a plurality of pixels in the detected shadow area;
A target histogram determination means for determining a target brightness histogram representing an improved contrast relating to the shaded area from the detected brightness histogram;
As a computer,
The predetermined brightness conversion rule executed by the shadow area brightness conversion means includes a plurality of values in the detected shadow area so that an improved brightness histogram related to the shadow area is substantially the target brightness histogram. Is a conversion rule for converting the brightness of the pixel of
The target histogram determination means includes
A central area determining means for determining a central area of the brightness histogram based on the brightness histogram to be expanded to a range from a predetermined maximum brightness to a minimum brightness for the brightness image;
A central area enlarging means for enlarging the central area of the brightness histogram to an enlarged histogram distributed in a range from the maximum brightness to the minimum brightness;
A pixel in a brightness range from the minimum brightness to the maximum brightness by combining a histogram lower area for brightness smaller than the determined center area and a histogram upper area for brightness greater than the center area in the brightness histogram. Is converted into a combined histogram, and the combined histogram generated by the conversion is combined with the expanded histogram generated by the central region converting means to generate the target brightness histogram, and upper and lower area expanding means,
Comprising
The program capable of improving the contrast of the shadow portion according to claim 1 or 2. Area dividing means for dividing the brightness image into a plurality of blocks;
As a program to further function the computer,
The shadow area lightness conversion means determines the lightness for improving the contrast of the shadow area included in each of the plurality of blocks;
The program capable of improving the contrast of a shadow portion according to any one of claims 1 to 3. A shadow area detection step for detecting a shadow area where a shadow area exists in the brightness image;
With respect to the brightness of a plurality of pixels in the detected shadow area, the brightness for improving the contrast of the shadow area is determined according to a predetermined brightness conversion rule, and the brightness of the plurality of pixels is determined A shading portion lightness conversion step for converting to a lightness value,
A gradation for performing gradation processing on a group of pixels near the boundary in the boundary adjacent region located adjacent to the boundary with the non-shadow portion region where the non-shadow portion exists in the shadow portion region in the brightness image Processing steps;
Including
The gradation processing determines a group of further improved brightness values for the group of boundary neighboring pixels, and replaces the brightness values of the group of boundary neighboring pixels with the group of brightness values determined in the shadow portion brightness conversion step. The process of changing to a further improved brightness of the group,
The further improved brightness of the group is determined in the shadow part brightness conversion step of each boundary neighboring pixel as the position of each of the group of boundary neighboring pixels moves away from the boundary toward the inside of the shadow part region. Changes from a lightness close to the lightness before being changed to a lightness close to the lightness after being determined in the shaded portion lightness conversion step of each pixel near the boundary,
A method for improving the contrast of a shadow portion characterized by the above. A boundary adjacent region dividing step for dividing the boundary adjacent region into a plurality of gradation regions depending on a distance from the boundary;
Further including
The gradation processing is determined in the lightness of each pixel before the lightness is determined in the shadow part lightness conversion step and in the shadow part lightness conversion step for each of a plurality of pixels in each gradation region. The further improvement for each pixel in the gradation area using the brightness of each pixel and the weight for the gradation area determined in advance depending on the distance from the boundary to the gradation area. Determine the brightness
The method for improving the contrast of a shadow portion according to claim 5. A histogram detection step of detecting a brightness histogram for a plurality of pixels in the detected shaded area;
A target histogram determination step for determining a target brightness histogram representing an improved contrast with respect to the shaded area from the detected brightness histogram;
Further including
The predetermined lightness conversion rule executed in the shadow part lightness conversion step includes a plurality of lightness values in the detected shadow part region so that an improved lightness histogram related to the shadow part region becomes substantially the target lightness histogram. It is a conversion rule for converting the brightness of a pixel,
The target histogram determination step includes
A central region determining step for determining a central region of the lightness histogram to be expanded in a range from a predetermined maximum lightness to a minimum lightness for the lightness image, based on the lightness histogram;
A central region enlargement step of enlarging the central region of the brightness histogram into an enlarged histogram distributed in a range from the maximum brightness to the minimum brightness;
A pixel in a brightness range from the minimum brightness to the maximum brightness by combining a histogram lower area for brightness smaller than the determined center area and a histogram upper area for brightness greater than the center area in the brightness histogram. Is converted into a combined histogram, and the combined histogram generated by the conversion is combined with the expanded histogram generated in the central region converting step to generate the target brightness histogram, and an upper and lower region expanding step,
Including
7. The method for improving the contrast of a shadow portion according to claim 5 or 6, wherein: An area dividing step of dividing the brightness image into a plurality of blocks;
Further including
The shade area brightness conversion step determines brightness for improving the contrast of the shadow area included in each of the plurality of blocks.
The method for improving the contrast of a shadow portion according to any one of claims 5 to 7.