JP2005269451A - Image processing apparatus, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of properly correcting an image (especially, shading correction), and to provide an image processing method and a computer. <P>SOLUTION: A digital camera 1 performs projective transformation for an original document image obtained by photographing an original 2, and segments the original document image. The digital camera 1 divides segmented original document image into a plurality of blocks, and acquires the maximum luminance for each block. The paper of the document 2 usually has a bright color, and its maximum luminance corresponds to the luminance of the paper of the document 2. If the difference between the maximum luminance and the minimum luminance of a block is not less than a predetermined value, the digital camera 1 judges that the luminance value of the block is a proper value, obtains a shading correction value of each block on the basis of the acquired maximum luminance value of each block, and performs shading correction of each pixel of each block on the basis of the obtained shading correction value. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  With the development of digital cameras and the low price of storage-type memory, digital cameras are not only used for taking pictures of landscapes and people, but are also used for paper documents, business cards, and blackboards written at meetings. An application is being considered in which what is displayed is photographed, and these images are digitally stored in a personal computer or the like for management.

  When photographing a document or a blackboard in this way, it is desirable to photograph the photographing object in front and vertically. However, it is difficult to shoot a document placed on the desk vertically and vertically. Also, when shooting a blackboard, it may be difficult to photograph the blackboard from the front depending on the position of the photographer. Even if the image can be taken from the front, it may be preferable to avoid taking the image from the front for reasons such as reflection of light. Thus, when an object such as a document is photographed from an oblique direction, characters and the like are distorted obliquely or trapezoidally. Of course, these characters can be interpreted. However, even if it can be read, it is very tiring to read. Moreover, since these images have distortion, it is difficult to reuse these images.

  In order to solve such a problem, for example, there is an image processing apparatus that corrects image distortion by using two parallax images (see, for example, Patent Document 1).

  This image processing apparatus creates a corrected image that is synthesized by performing image conversion and combining two or more images so as to eliminate parallax.

  Some of such image processing apparatuses correct shading (brightness unevenness). For shading correction of an input image, a p × q image is divided into i × i blocks, and pixel values of each block are averaged. Is calculated, the correction amount of each pixel is calculated based on the calculated average value, and the input pixel value is multiplied by the correction amount (for example, see Patent Document 2).

As an image processing apparatus that performs shading correction, there is an apparatus that switches a filter coefficient for each block in accordance with the spatial frequency or dispersion value of each block obtained by dividing an input image (see, for example, Patent Document 3).
JP 2001-134751 (5th page, FIG. 1) JP 2003-153132 A (page 2-7, FIG. 1) JP 2003-1116048 (page 2-6, FIG. 2)

  However, in Patent Document 2, since the correction amount of each pixel is obtained based on the pixel value average of each block, a figure, a picture, or the like is inserted in a large area greater than or equal to ixi in the photographed document In that region, the shading correction cannot be performed correctly.

  Further, in Patent Document 3, since the filter coefficient is switched for each block, shading correction cannot be performed satisfactorily when the entire document has insufficient contrast.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing method, and a computer that can correct an image correctly.

In order to achieve this object, an image processing apparatus according to the first aspect of the present invention provides:
In an image processing apparatus that performs image processing on a document image obtained by photographing a document,
A block dividing unit for dividing the document image into a plurality of blocks;
A block luminance obtaining unit that examines the luminance value of each pixel in each block divided by the block dividing unit and obtains a luminance value corresponding to the original paper for each block;
A shading correction unit that calculates a shading correction value for each block based on the block luminance value acquired by the block luminance acquisition unit, and performs shading correction for each pixel of each block based on the calculated shading correction value; It is characterized by.

  The block luminance acquisition unit may acquire a maximum luminance value as a luminance value corresponding to the original sheet.

The luminance value of each pixel of each block divided by the block dividing unit is examined, the minimum luminance value of the luminance values of each pixel is obtained for each block, and the maximum luminance of each block acquired by the block luminance acquiring unit A luminance range acquisition unit that acquires a difference between the value and the acquired minimum luminance value as a luminance range of each block;
The shading correction unit compares the luminance range of each block acquired by the luminance range acquisition unit with a setting value set in advance to determine an appropriate luminance range, and the luminance range exceeds the setting value. The shading correction value of each block may be obtained so that the maximum luminance value of each block acquired by the block luminance acquisition unit becomes a target luminance target value for all blocks. .

  When the luminance range acquired by the luminance range acquisition unit is equal to or less than the set value, the shading correction unit determines the luminance of the corresponding block based on the maximum luminance value of the peripheral block acquired by the block luminance acquisition unit. An interpolation value may be obtained, and a shading correction value for a block in which the luminance range is equal to or less than the set value may be obtained so that the obtained luminance interpolation value becomes the luminance target value.

  The block luminance acquisition unit and the luminance range acquisition unit may generate a luminance histogram for each block, and acquire the maximum luminance value and the minimum luminance value from the generated luminance histogram.

  The shading correction unit compares, for each block, the luminance range of the block acquired by the luminance range acquisition unit with a lower limit value of shading correction preset for the luminance range, and the luminance range is the lower limit value. The shading correction may be stopped for blocks that are less than the number.

The luminance value of each pixel of each block divided by the block dividing unit is examined, the average luminance value in each block is acquired for each block, and the maximum luminance value of each block acquired by the block luminance acquiring unit and the acquired An average luminance difference acquisition unit that acquires a difference from the average luminance value for each block,
The shading correction unit compares, for each block, the difference acquired by the average luminance difference acquisition unit and an upper limit value of shading correction set in advance for the difference, and the difference exceeds the upper limit value. For the block, the obtained shading correction value may be reduced.

  You may provide the brightness correction part which acquires the parameter used for the said shading correction, and performs brightness correction with respect to the image which the said shading correction part performed the shading correction with respect to each pixel of each block.

  The luminance correction unit may extend the specified luminance value range to a preset range based on the acquired parameter.

An image processing method according to a second aspect of the present invention includes:
Dividing the document image into a plurality of blocks;
Examining the luminance value of each pixel in each of the divided blocks, and obtaining the luminance value corresponding to the original paper for each block;
Obtaining a shading correction value for each block based on the acquired block luminance value, and performing a shading correction for each pixel of each block based on the obtained shading correction value. To do.

The program according to the third aspect of the present invention is:
On the computer,
The procedure to divide the original image into multiple blocks,
A procedure for examining the luminance value of each pixel in each of the divided blocks and obtaining the luminance value corresponding to the original paper for each block;
A procedure for obtaining a shading correction value for each block based on the acquired block luminance value, and performing a shading correction for each pixel of each block based on the obtained shading correction value;
Is to execute.

  According to the present invention, it is possible to correct an image correctly.

Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.
In the present embodiment, a case where the image processing apparatus is applied to a digital camera will be described.

A configuration of a digital camera 1 according to the present embodiment is shown in FIG.
The digital camera 1 according to the present embodiment shoots a document 2 such as a character, a figure, and a photograph as a target shooting target, and generates an image as if it was shot from the front from a document image obtained by shooting. Is. The digital camera 1 includes a photographic lens unit 11, a liquid crystal monitor 12, and a shutter button 13.

  The photographing lens unit 11 includes a lens that collects light and the like, and collects light from the document 2.

The liquid crystal monitor 12 is for projecting an image taken in through the photographing lens unit 11.
The shutter button 13 is pressed when shooting a shooting target.

  As shown in FIG. 2, the digital camera 1 includes an optical lens device 21, an image sensor 22, a memory 23, a display device 24, an image processing device 25, an operation unit 26, a computer interface unit 27, An external storage IO device 28 and a program code storage device 29 are provided.

  The optical lens device 21 includes a photographic lens unit 11 and a driving unit thereof, and forms an image on the image sensor 22 by condensing light from characters, drawings, photographs, and the like of the document 2. .

  The image sensor 22 is for capturing a formed image as digitized image data, and is constituted by a CCD or the like. If the image sensor 22 is controlled by the CPU 30 and the shutter button 13 is not pressed, digital image data having a low preview resolution is generated, and the image data is periodically stored in the memory 23 at intervals of about 30 sheets per second. To send. Further, when the shutter button 13 is pressed, the image sensor 22 generates image data with high resolution and sends the generated image data to the memory 23.

  The memory 23 temporarily stores low-resolution preview images from the image sensor 22, high-resolution image data, original image data processed by the image processing device 25, and processed image data. The memory 23 sends the temporarily stored image data to the display device 24 or the image processing device 25. The memory 23 stores data necessary for the image processing device 25 and the CPU 30 to execute processing and various threshold values.

  The display device 24 includes the liquid crystal monitor 12 and displays the image on the liquid crystal monitor 12. The display device 24 displays a low-resolution preview image or a high-resolution image temporarily stored in the memory 23 on the liquid crystal monitor 12.

  The image processing device 25 is for performing image processing such as image data compression, image distortion correction, and image effect processing on the image data temporarily stored in the memory 23.

  As shown in FIG. 3A, when the digital camera 1 shoots from the left direction and the right direction toward the document 2, the liquid crystal monitor 12 displays the images shown in FIGS. 3B and 3C, respectively. As described above, the original image is displayed in a distorted manner. The image processing device 25 is controlled by the CPU 30 to cut out a quadrangle from the distorted image in order to correct the image distortion of the trapezoidal image (original image) as shown in FIGS. Then, the image processing device 25 performs projective transformation of the photographed image into the cut rectangle, thereby generating an image as if photographed from the front as shown in FIG.

  Further, the image processing device 25 cuts out the document image after the projective conversion, performs shading correction and image effect processing, and generates a sharpened document image as shown in FIG. Shading correction and image effect processing will be described later.

  The operation unit 26 includes switches and keys for controlling the document projection function. When the user presses these switches and keys, the operation unit 26 responds and transmits the operation information at this time to the CPU 30.

  The operation unit 26 includes a projective conversion key, a photographing key, a reproduction key, a cursor key, a control key, and the like (not shown). The projective conversion key is a key for instructing the vertical and horizontal reduction and rotation of the image as the projective conversion.

  The shooting key is a key for selecting a shooting mode when shooting a shooting target. The reproduction key is a key for selecting a reproduction mode for reproducing an image to be photographed obtained by photographing. The control key is a key having functions such as a YES key for confirming the operation, a NO key for canceling the operation, and an editing key for performing editing.

  The computer interface unit 27 operates as a USB store registration class driver when the digital camera 1 is connected to a computer (not shown). Thus, when the computer is connected to the digital camera 1, the computer handles the memory card 31 as an external storage device of the computer.

  The external storage IO device 28 inputs and outputs image data and the like with the memory card 31. The memory card 31 stores image data and the like supplied from the external storage IO device 28.

  The program code storage device 29 is for storing a program executed by the CPU 30, and is configured by a ROM or the like.

  The CPU 30 controls the entire system according to a program stored in the program code storage device 29. The memory 23 is also used as a work memory for the CPU 30.

  When operation information is transmitted from the operation unit 26 by pressing a switch or key of the operation unit 26, the CPU 30 performs image sensor 22, memory 23, display device 24, image processing device based on the operation information. 25 etc. are controlled.

  Specifically, when the operation information indicating that the photographing key is pressed is transmitted from the operation unit 26, the CPU 30 sets each unit to the photographing mode. The CPU 30 sets the image sensor 22 to the preview mode if the shutter button 13 is not pressed in the state set to the shooting mode, and reads the high-resolution shooting target image if the shutter button 13 is pressed. Set to. Further, when the operation information indicating that the reproduction key is pressed is transmitted, the CPU 30 sets each unit to the reproduction mode.

  Further, when the operation information indicating that the projection conversion key and the rotation correction key are pressed is transmitted from the operation unit 26, the CPU 30 transmits the operation information to the image processing device 25, and causes the image processing device 25 to operate. Control.

  When the image data is temporarily stored in the memory 23, the CPU 30 records the preview image and the high-resolution image data in different storage areas.

  Further, the CPU 30 records preview image and high-resolution image data on the memory card 31 via the external storage IO device 28, and reads the recorded image data from the memory card 31. For this reason, the CPU 30 creates an image file for storing image data in the memory card 31.

  Then, for example, the CPU 30 records the corrected image data compressed in the JPEG format into image files. Further, the CPU 30 creates a header information storage area of the image file for each image file and records the image data, such as the projection parameters and image effect processing correction parameters obtained by the image processing device 25 when the image data is recorded. Is recorded in this header information storage area.

Next, the operation of the digital camera 1 according to the embodiment will be described.
When the user turns on (turns on) the digital camera 1, the CPU 30 acquires program data stored in the program code storage device 29. When the user switches the mode to the shooting mode with the mode switch and presses the shutter button 13, the operation unit 26 transmits this operation information to the CPU 30. The CPU 30 receives this operation information, and the CPU 30, the image processing device 25, etc. execute a photographing process according to the flowchart shown in FIG.

The CPU 30 sets the image sensor 22 to the preview mode (step S11).
The CPU 30 determines whether or not the shutter button 13 has been pressed based on the operation information transmitted from the operation unit 26 (step S12).

  When it is determined that the shutter button 13 has been pressed (Yes in Step S12), the CPU 30 controls the image sensor 22 by switching from the preview mode to the high resolution mode (Step S13).

  The CPU 30 records the high-resolution photographic target image data generated by the image sensor 22 in a storage area different from the preview image on the memory 23 (step S14).

The CPU 30 determines whether the reading of the image data has been completed (step S15).
If it is determined that the reading has not been completed (No in step S15), the CPU 30 continues to control the image sensor 22 to read the image data.

  When it is determined that all the image data has been read and the image transmission has been completed (Yes in step S15), the CPU 30 generates a low-resolution preview image from the captured image (high-resolution image), and uses the preview image in the memory 23 for the preview image. The preview image data is written in the storage area (step S16).

  Under the control of the CPU 30, the image processing device 25 acquires the outline of the document image that is the shooting target from the shot image, and creates a projective conversion image based on the acquired outline of the shooting target (step S17).

The image processing device 25 cuts out the document image after projective conversion (step S18).
The image processing device 25 performs image effect processing on the clipped document image (step S19). Details of this will be described later.

  The image processing device 25 performs compression processing on the image data on which the image effect processing has been performed, and creates compressed data of the converted image (step S20).

  The image processing device 25 records the created compressed data of the converted image on the memory card 31 (step S21).

Next, the image effect process (the process of step S19 in FIG. 4) performed by the image processing apparatus 25 will be described.
The image processing device 25 performs the following image effect processing.
(1) Document image cutout (2) Image effect correction parameter extraction (3) Shading correction (4) Brightness correction (5) Color correction The contents of this processing will be described.
(1) Cutout of Document Image The image processing device 25 cuts out a document image from an image as shown in FIG. The image to be photographed may include not only a substantial image such as a photograph, a figure, a character, but also an image irrelevant to the content of the object to be photographed, such as a shadow on the surface of the document 2. .

  If a histogram is generated including an image of the periphery of the object to be photographed, stretching of the histogram may not function effectively. For this reason, the image processing device 25 cuts out the image of the actual target portion only when extracting the image effect correction parameters.

(2) Extraction of image effect correction parameters The image effect correction parameters are variables necessary for image effect processing, such as a maximum value, a minimum value, a peak value of a luminance histogram, and a peak value and an average value of a color difference histogram.

  The luminance histogram indicates the distribution of luminance values (Y) existing in the image, and is generated by counting the number of pixels for each luminance value. FIG. 5 shows an example of the luminance histogram. In FIG. 5, the horizontal axis indicates the luminance value (Y), and the vertical axis indicates the number of pixels. In order to correct the image effect, it is effective to use the maximum value (Ymax), minimum value (Ymin), and peak value (Ypeak) which are luminance histogram parameters. The image processing device 25 generates this luminance histogram, and extracts these image effect correction parameters from this luminance histogram.

(3) Shading correction Shading correction is a process of correcting luminance unevenness that has occurred in a photographed material according to illumination conditions. How to make the white background of the material white has a great effect on the sharpness of the image. When the brightness of an image including shading (brightness unevenness) is uniformly stretched, even the white background color of the document 2 may become gray due to the brightness stretching process, resulting in an unpleasant image. Also, in the case of an image with insufficient contrast, the background of the document 2 may become black, and the image may be difficult to see. For this reason, it is necessary to remove the lighting effect.

  In the present embodiment, in order to acquire a sharpened image more effectively, the luminance correction is performed after the luminance shading correction is performed in advance.

  In this embodiment, it is assumed that a p × q image is divided into I × J blocks, and shading correction is performed for each block. As an example, FIG. 6 shows an image divided into 5 × 5 sub-blocks. In FIG. 6, I and J indicate the indexes of the divided blocks. Ymax (I, J), Ymin (I, J), and Ypeak (I, J) are the luminance maximum value, minimum value, and peak value of the I-th block on each horizontal axis and the J-th block on the vertical axis. .

  In the case of performing shading correction, the average luminance value is not preferable because it is affected by the luminance of the background and the luminance on characters and graphs written on the document 2. It is expected that the illumination on the image of the document 2 to be photographed is reflected in the maximum luminance value of each area of the material. Therefore, in the present embodiment, only the maximum luminance value is acquired as the luminance information of the background of the material.

  Next, when shading is predicted according to the maximum value of each block, in the divided block area, if the block contains only figures and photographs and there is no background, the maximum value of the luminance of the block is It does not indicate the brightness of the groundwork. If the luminance of the background is not indicated, the maximum luminance value adversely affects the shading correction. In order to avoid this, the difference between the maximum value and the minimum value of luminance is used. If this difference is small compared to other blocks, it is preferable to determine that this block is composed only of figures and photographs and not use the luminance information of the corresponding block for shading correction. .

In order to perform shading correction, it is first necessary to calculate the shading correction amount of each block. The calculation of the shading correction amount will be described.
To calculate the shading correction amount, first, the luminance range Yrange of each block is calculated. The luminance range Yrange is expressed by the following formula 1.

  If there are many blocks with a small luminance range Yrange, it is determined that the document 2 is not present, or the sheet 2 has almost no characters or the like written on it. In this case, it is better not to perform shading correction.

  If the luminance range Yrange is small, the image processing device 25 rewrites Yrange to 0, counts the number of blocks where Yrange is 0, and sets the count to SmallDispertionCount. The image processing device 25 compares the luminance range Yrange of the block with a preset threshold Y_RANGE_MIN in order to determine whether there are many blocks with a small Yrange. The image processing device 25 determines that the luminance range Yrange is small if the luminance range Yrange is less than or equal to the threshold Y_RANGE_MIN. The memory 23 stores a count number SmallDispertionCount and a threshold value Y_RANGE_MIN.

  Further, the image processing device 25 compares the count number SmallDispertionCount with a preset threshold value MIN_SMALL_DISPERTION_COUNT in order to determine whether there are many blocks with a small luminance range Yrange. The threshold value MIN_SMALL_DISPERTION_COUNT is a value set in advance for performing this determination, and the memory 23 stores the threshold value MIN_SMALL_DISPERTION_COUNT in advance.

  If the image processing apparatus 25 determines that the count number SmallDispertionCount is greater than MIN_SMALL_DISPERTION_COUNT, the image processing apparatus 25 determines that the document 2 is not present or has almost no characters on the document 2 and performs shading correction. Make it not exist.

Next, in order to make the entire sheet uniform brightness by shading correction, it is desirable that the maximum luminance value Ymax (I, J) of each block be substantially the same target value. If the entire sheet has a uniform brightness, the object of shading correction is achieved.
For this reason, the maximum luminance value is set to Ymax (I, J), and in this embodiment, the maximum luminance value Ymax (I, J) of the block is set to Ymaxmax.

  In this embodiment, the block shading correction amount is Ycv (I, J), and the block shading correction amount Ycv (I, J) is obtained by the following procedure.

  First, it is evaluated whether the obtained maximum luminance value Ymax (I, J) appropriately represents the brightness of the paper block in each block.

Next, an average value Ymeanmax of the maximum luminance value Ymax (I, J) is obtained, and a difference Yd between the maximum luminance value Ymax (I, J) and the average value Ymeanmax is obtained. The difference Yd is expressed by the following equation 2.

  It is checked whether the difference Yd is equal to or smaller than a preset threshold YDIF_UNDER and the luminance range Yrange (I, J) of the corresponding block is not zero. Note that the memory 23 stores the threshold value YDIF_UNDER in advance.

If the difference Yd is less than or equal to the predetermined value YDIF_UNDER and the luminance range Yrange (I, J) of the block is 0, the maximum luminance Ymax (I, J) of the block does not sample the background color. Probability is high. In this case, the peripheral maximum luminance values Ymax (I-1, J), Ymax (I + 1, J), Ymax (I, J-1), Ymax (I, J + 1) are used to determine the corresponding block. An average value of luminance correction amounts is obtained. At this time, if there is a point where the maximum luminance Yrange is 0 in the surrounding blocks, the average value is not calculated. The new maximum brightness Ymax (I, J) is expressed by the following equation (3).

  When the difference Yd is large, the block is expected to be significantly brighter due to spotlight or the like. In this case, if linear interpolation is performed, the periphery of the spotlight becomes dark and unnaturalness occurs in the document 2, so that it is necessary to suppress the shading correction amount.

最後に、各ブロックの輝度シェーディング補正量は次の数５によって求められる。

For this reason, the image processing device 25 compares the difference Yd with a preset threshold value YDIF_UPPER. If the difference Yd exceeds the threshold value YDIF_UPPER, the image processing apparatus 25 determines that the block is significantly brighter, and performs shading correction. Suppress the amount. The memory 23 stores this threshold value YDIF_UPPER in advance. The amount of suppression of the shading correction amount is expressed by the following equation (4).

Finally, the luminance shading correction amount of each block is obtained by the following equation (5).

  Next, shading correction of each pixel is performed based on the shading correction amount of the block obtained in this way. The method will be described. It is assumed that the shading correction amount of each block obtained by the above method represents the luminance correction amount (shading correction amount) at the center point of the block. The luminance correction amount of the pixels other than the center point can be obtained by performing linear interpolation from the four neighboring points from the luminance correction amount described above.

Specifically, the input image of M × N pixels is divided into K × L blocks, and the shading correction amount of each block is calculated. If the coordinates of the pixel for which the shading correction amount is to be obtained are (x, y) and the correction amount is Ypcv (x, y),

When i, j obtained according to Equation 6 is smaller than 0, or when I = K−1, J = L−1, etc., there is no 4-point data in the vicinity. For this reason, linear interpolation is performed using the nearest four points. Linear interpolation is performed according to the following equation (7).

このようにして各画素の輝度Y(x,y)は、次の数９に示すシェーディング補正によって新しい輝度Ｙ'（x,y）に書き換えられる。

In this way, two-dimensional coordinates (i, j) as shown in FIG. 7 are obtained from each pixel (x, y). Note that (i, j) indicates each pixel position of each block. In FIG. 7, the area surrounded by the bold square is the block luminance correction amounts Ycv (I, J), Ycv (I + 1, J), Ycv at the four points near the pixel (x, y) in the area. (I, J + 1), Ycv (I + 1, J + 1). From these four points, the shading correction amount Ypcv (x, y) of this pixel is expressed by the following equation (8).

In this way, the luminance Y (x, y) of each pixel is rewritten to a new luminance Y ′ (x, y) by the shading correction shown in the following equation (9).

(4) Luminance correction Next, it is necessary to perform luminance enlargement correction. In order to perform luminance enlargement correction, first, a corrected image is input, a luminance histogram as shown in FIG. 5 is created from the image, and a maximum value and a minimum value are acquired. Here, the maximum value is the maximum Y value having a certain number of pixels in the histogram, and the minimum value is the minimum Y value having a certain number of pixels. From these values, as shown in FIG. 5, the luminance value is 0 at the minimum luminance value and is 255 at the maximum luminance value. Perform enlargement.

In the present embodiment, since a block histogram is already generated in the shading block, it is preferable to use this histogram. As a result of the above-described shading correction, the maximum luminance value of the block is made uniform to Ymaxmax. Therefore, the maximum value of the histogram is Ymaxmax. Further, since the minimum luminance value Ymin (I, J) of each block is considered to have been corrected by the luminance correction value Ycv (I, J) of that block, the minimum luminance value Ymin (I, J) of each block. ) With the following formula.

Yminmin is searched for the smallest minimum value from these block luminance minimum values.
As a result, in the present embodiment, a conversion table as shown in FIG. 8 is generated that outputs (Y ′) according to the luminance (Y) of the background with the maximum value Ymaxmax and the minimum value Yminmin in FIG. After creating the table, each pixel is converted into output data according to this conversion table as input data, and image effect processing is performed by rewriting the luminance value of each pixel to a new luminance value. In an image on which image effect processing has been performed in this manner, bright pixels are brighter and dark pixels are darker, so that variations in luminance are widened and an image with excellent visibility is generated.

(4) Color Correction Next, color correction will be described. Depending on the photographed image, the white balance of the digital camera may not be adjusted properly, and the whole color may change to yellow or the like. This color change cannot be corrected only by performing image effect processing of the luminance histogram. Therefore, color adjustment is performed by creating a color difference histogram.

  The color difference histogram shows the distribution of color differences (U, V) existing in the actual document portion excluding the contour of the document image, and is generated by counting the number of pixels in the actual document portion for each color difference. FIG. 9 shows an example of the color difference histogram. In FIG. 9, the horizontal axis indicates the color difference, and the vertical axis indicates the number of pixels. Also in the case of the color difference histogram, a peak value (Upeak, Vpeak) of the color difference where the count value of the pixel becomes maximum appears. This peak value is also considered to indicate the color difference of the background color of the document 4.

When the values of the color differences U and V are both 0, the color is an achromatic color, and color adjustment is performed so that the peak values (Upeak, Vpeak) are 0. FIG. 10 shows an example of a color adjustment conversion graph for performing color adjustment. That is, the color conversion line is represented by two line segments. The output values U ′ and V ′ for the input values U and V are given by the following equation (11).

However, U = 127 when U> 127, and U = −128 when U> −128. The same applies to the color difference V. In the present embodiment, the image is more natural when the gain is adjusted for the color difference because the enlargement is performed by the luminance correction. Therefore, the actual conversion formula is as shown in Equation 12.

  In the present embodiment, a conversion table for converting the input color difference (U) into the output (U ′) is created according to the relationship represented by Equation 12 as in the luminance conversion. After creating the table, each pixel color difference U is used as input data, and sequentially converted into output data according to this conversion table, and the image effect processing is performed by rewriting the color difference of each pixel to a new color difference U ′. The same applies to the color difference V.

Based on such a concept, the image processing device 25 executes the image effect processing according to the flowchart shown in FIG.
First, the image processing device 25 divides the cut image into a plurality of (K × L) sub-blocks (step S31).

The image processing device 25 creates a luminance histogram in each sub-block and obtains an image effect correction parameter (step S32).
The image processing device 25 obtains a shading correction amount for each sub-block (step S33).

The image processing device 25 executes shading correction amount calculation processing according to the flowcharts shown in FIGS. 12 and 13.
That is, the image processing device 25 sets the count value of SmallDispertionCount to 0 (step S41).
The image processing device 25 obtains the luminance range Yrange of the block according to Equation 1 (step S42).

The image processing device 25 compares the obtained luminance range Yrange and the threshold value Y_RANGE_MIN, and determines whether or not the luminance range Yrange is less than or equal to the threshold value Y_RANGE_MIN (step S43).
When it is determined that the luminance range Yrange is equal to or smaller than the threshold Y_RANGE_MIN (Yes in step S43), the image processing device 25 rewrites the luminance range Yrange to 0 (step S44).
The image processing device 25 increments the count value of the count SmallDispertionCount (step S45).

  On the other hand, when it is determined that the luminance range Yrange exceeds the threshold Y_RANGE_MIN (No in step S43), the image processing apparatus 25 keeps the count values of the luminance range Yrange and the count SmallDispertionCount.

The image processing device 25 determines whether or not the calculation of the luminance range Yrange has been completed for all the blocks (step S46).
If it is determined that the processing has not been completed for all the blocks (No in step S46), the image processing apparatus 25 executes the processes in steps S42 to S45 again.

  If it is determined that the processing has been completed for all blocks (YES in step S46), the image processing apparatus 25 compares the count value of the count SmallDispertionCount with the threshold value MIN_SMALL_DIPERTION_COUNT, and determines whether the value of the count SmallDispertionCount exceeds the threshold value MIN_SMALL_DIPERTION_COUNT. Is determined (step S47).

  When it is determined that the value of the count SmallDispertionCount exceeds the threshold value MIN_SMALL_DIPERTION_COUNT (Yes in step S47), the image processing device 25 sets the flag value of the flag StopFlattenBright to 1 (step S48), and ends this process.

  On the other hand, when it is determined that the value of the count SmallDispertionCount is equal to or smaller than the threshold MIN_SMALL_DIPERTION_COUNT (No in step S48), the image processing device 25 obtains the maximum value Ymaxmax of the block luminance Ymax (I, J) and the average value Ymeanmax (step). S49).

  The image processing device 25 obtains a difference value Yd between the maximum luminance value Ymax (I, J) and the average value Ymeanmax according to Equation 2 (step S50).

The image processing device 25 compares the obtained difference value Yd with the predetermined value YDIF_UNDER and determines whether or not the difference value Yd is equal to or smaller than the predetermined value YDIF_UNDER (step S51).
When it is determined that the difference value Yd is equal to or smaller than the predetermined value YDIF_UNDER (Yes in Step S51), the image processing device 25 determines whether or not the luminance range Yrange (I, J) of the corresponding block is 0 (Step S52). .

  If it is determined that the luminance range Yrange (I, J) of the block is 0 (Yes in step S52), the image processing device 25 samples the background color for the maximum luminance Ymax (I, J) of the block. It is determined that there is a high possibility that there is not. In this case, the image processing apparatus 25 determines whether there is a peripheral block having a luminance range Yrange (I, J) of 0 (step S53).

  When it is determined that the peripheral block has no luminance range Yrange (I, J) of 0 (No in step S53), the image processing device 25 determines that the peripheral block maximum luminance values Ymax (I-1, J), Ymax Using (I + 1, J), Ymax (I, J-1), Ymax (I, J + 1), the maximum luminance value of the corresponding block is rewritten to the average value of the surrounding blocks according to Equation 3 (step S54). .

  When it is determined that there is a peripheral block having a luminance range Yrange (I, J) of 0 (Yes in step S53), the image processing apparatus 25 excludes blocks whose luminance range Yrange (I, J) is 0. The maximum luminance value of the corresponding block is obtained from the peripheral maximum luminance value (step S55).

  The image processing device 25 calculates the shading correction amount of each block according to Equation 5 (step S56).

  On the other hand, when it is determined that the difference value Yd exceeds the predetermined value YDIF_UNDER (No in step S51), the image processing device 25 compares the difference value Yd with the predetermined value YDIF_UPPER, and the difference value Yd is equal to the predetermined value YDIF_UPPER. It is determined whether or not it exceeds (step S57).

  When it is determined that the difference value Yd exceeds the predetermined value YDIF_UPPER (Yes in step S57), the image processing device 25 obtains the maximum luminance value Ymax in which the correction amount is suppressed according to Equation 4 (step S58). Then, the image processing device 25 obtains the shading correction amount of each block according to Equation 5 (step S56).

  The image processing device 25 determines whether the document is to be seeded based on the flag value of the flag StopFlattenBright (step S34 in FIG. 11). If the flag value of the flag StopFlattenBright is reset to 0, the image processing apparatus 25 determines that the document 2 should be seeded (Yes in step S34). In this case, the image processing device 25 obtains the shading correction amount Ypcv (x, y) of each pixel of each block according to the equations 6 to 9, and performs the shading correction (step S35).

  On the other hand, if the flag value of the flag StopFlattenBright is set to 1, the image processing device 25 determines that the document 2 should not be seeded (No in step S34). In this case, the image processing device 25 does not perform shading correction.

  The image processing device 25 performs luminance correction by rewriting the minimum luminance value Ymin (I, J) of each block according to Equation 9 or using the conversion table shown in FIG. 8 (step S36).

  The image processing device 25 performs color correction by performing color difference conversion according to Equations 10 and 11 or using a conversion table as shown in FIG. 10 (Step S37), and ends the image effect processing.

As described above, according to the present embodiment, an image is divided into a plurality of sub-blocks, and shading correction is performed so that the maximum luminance value of each block becomes the same.
Accordingly, when correcting shading caused by the shadow or light of the paper of the document 2 photographed by the digital camera 1, even if the paper contains blocks with significantly different brightness, appropriate shading correction should be performed. Can do.

  In addition, since shading correction is not performed when the luminance range of each block is small, unnecessary shading correction can be suppressed when the original is not shown or when the original 4 is almost blank. it can.

  Further, when there is a remarkably bright block due to spotlight or the like, the shading correction amount is suppressed, so that inappropriate shading correction due to such a bright block can be prevented.

  In addition, after performing shading correction, luminance correction is performed using the parameters used for shading correction, so the number of operations can be reduced, and the processing time required for shading correction can be shortened. Further, effective image sharpening processing can be performed.

In carrying out the present invention, various forms are conceivable and the present invention is not limited to the above embodiment.
For example, in the above embodiment, the image processing apparatus has been described on the assumption that the paper of the document 2 has a bright color. However, for example, in the case of a manuscript in which white characters are described on black paper, the paper of manuscript 2 does not necessarily have the maximum luminance value, but in this case, the luminance value of the image can be inverted. In the same manner, the above embodiment can be applied.

  In the above embodiment, image correction is performed by the digital camera 1. However, image correction can also be performed by a computer. In this case, a computer is connected to the computer interface unit 27, and the computer executes photographing processing according to the flowchart shown in FIG. If the computer performs such image editing processing, it becomes easier to input operation information using a computer mouse or the like, compared to the digital camera 1, so that operability is improved. Further, since the display device of the computer is generally larger than the liquid crystal monitor 12 of the digital camera 1, the image can be visually confirmed in detail and image correction can be performed with high accuracy.

  In the above-described embodiment, the program is described as being stored in advance in a memory or the like. However, a program for causing a computer to operate as the whole or a part of the apparatus or to execute the above-described processing is a flexible disk, a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), The information may be stored in a computer-readable recording medium such as an MO (Magneto Optical disk) and distributed, installed in another computer, operated as the above-described means, or the above-described steps may be executed.

  Furthermore, the program may be stored in a disk device or the like included in a server device on the Internet, and may be downloaded onto a computer by being superimposed on a carrier wave, for example.

It is explanatory drawing which shows the structure of the digital camera which concerns on this embodiment of this invention. It is a block diagram which shows the structure of the digital camera shown in FIG. It is explanatory drawing of the function of the image processing apparatus shown in FIG. It is a flowchart which shows the content of the imaging | photography process which a digital camera performs. It is explanatory drawing which shows the brightness | luminance histogram which the image processing apparatus shown in FIG. 2 produced | generated. FIG. 5 is an explanatory diagram of a 5 × 5 block as an example of a block obtained by dividing a document image. It is explanatory drawing which shows the two-dimensional shading correction value which the image processing apparatus shown in FIG. 2 calculated | required from each pixel. It is explanatory drawing which shows an example of the conversion table which the image processing apparatus shown in FIG. 2 uses for brightness correction. It is explanatory drawing which shows the color difference histogram which the image processing apparatus shown in FIG. 2 produced | generated. FIG. 3 is an explanatory diagram illustrating an example of a conversion table used for color difference correction by the image processing apparatus illustrated in FIG. 2. It is a flowchart which shows the content of the image effect process which the image processing apparatus shown in FIG. 2 performs. It is a flowchart which shows the content (1) of the calculation process of the shading correction amount which the image processing apparatus shown in FIG. 2 performs. It is a flowchart which shows the content (2) of the calculation process of the shading correction amount which the image processing apparatus shown in FIG. 2 performs.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Original, 11 ... Shooting lens part, 12 ... Liquid crystal monitor, 13 ... Shutter button, 21 ... Optical lens apparatus, 22 ... Image sensor, 23 ... Memory, 24 ... Display device, 25 ... Image processing device, 30 ... CPU, 31 ... Memory card

Claims (11)

In an image processing apparatus that performs image processing on a document image obtained by photographing a document,
A block dividing unit for dividing the document image into a plurality of blocks;
A block luminance obtaining unit that examines the luminance value of each pixel in each block divided by the block dividing unit and obtains a luminance value corresponding to the original paper for each block;
A shading correction unit that obtains a shading correction value for each block based on the block luminance value acquired by the block luminance acquisition unit, and performs shading correction for each pixel of each block based on the obtained shading correction value;
An image processing apparatus. The block luminance acquisition unit acquires a maximum luminance value as a luminance value corresponding to the original paper;
The image processing apparatus according to claim 1. The luminance value of each pixel of each block divided by the block dividing unit is examined, the minimum luminance value of the luminance values of each pixel is obtained for each block, and the maximum luminance of each block acquired by the block luminance acquiring unit A luminance range acquisition unit that acquires a difference between the value and the acquired minimum luminance value as a luminance range of each block;
The shading correction unit compares the luminance range of each block acquired by the luminance range acquisition unit with a setting value set in advance to determine an appropriate luminance range, and the luminance range exceeds the setting value. The block luminance acquisition unit obtains the shading correction value of each block so that the maximum luminance value of each block acquired by the block luminance acquisition unit becomes a luminance target value targeted by all the blocks.
The image processing apparatus according to claim 2. When the luminance range acquired by the luminance range acquisition unit is equal to or less than the set value, the shading correction unit determines the luminance of the corresponding block based on the maximum luminance value of the peripheral block acquired by the block luminance acquisition unit. An interpolation value is obtained, and a shading correction value of a block whose luminance range is equal to or less than the set value is obtained so that the obtained luminance interpolation value becomes the luminance target value.
The image processing apparatus according to claim 3. The block luminance acquisition unit and the luminance range acquisition unit generate a luminance histogram for each block, and acquire the maximum luminance value and the minimum luminance value from the generated luminance histogram.
The image processing apparatus according to claim 3, wherein the image processing apparatus is an image processing apparatus. The shading correction unit compares, for each block, the luminance range of the block acquired by the luminance range acquisition unit with a lower limit value of shading correction preset for the luminance range, and the luminance range is the lower limit value. For blocks that are less than, stop the shading correction,
The image processing apparatus according to claim 3, wherein the image processing apparatus is an image processing apparatus. The luminance value of each pixel of each block divided by the block dividing unit is examined, the average luminance value in each block is acquired for each block, and the maximum luminance value of each block acquired by the block luminance acquiring unit and the acquired An average luminance difference acquisition unit that acquires a difference from the average luminance value for each block,
The shading correction unit compares, for each block, the difference acquired by the average luminance difference acquisition unit and an upper limit value of shading correction set in advance for the difference, and the difference exceeds the upper limit value. For blocks, reduce the calculated shading correction value,
The image processing apparatus according to claim 2, wherein the image processing apparatus is an image processing apparatus. The image obtained by performing shading correction on each pixel of each block by the shading correction unit is provided with a luminance correction unit that acquires the parameters used for the shading correction and performs luminance correction.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The brightness correction unit extends a specified brightness value range to a preset range based on the acquired parameter;
The image processing apparatus according to claim 8. Dividing the document image into a plurality of blocks;
Examining the luminance value of each pixel in each of the divided blocks, and obtaining the luminance value corresponding to the original paper for each block;
Obtaining a shading correction value for each block based on the acquired block luminance value, and performing shading correction for each pixel of each block based on the obtained shading correction value,
An image processing method comprising: On the computer,
The procedure to divide the original image into multiple blocks,
A procedure for examining the luminance value of each pixel in each of the divided blocks and obtaining the luminance value corresponding to the original paper for each block;
A procedure for obtaining a shading correction value for each block based on the acquired block luminance value, and performing a shading correction for each pixel of each block based on the obtained shading correction value;
A program for running

Images