JP4247621B2 - Image processing apparatus, image processing method, and medium on which image processing control program is recorded - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a medium on which an image processing control program is recorded that automatically executes optimum image processing on real image data such as a digital photographic image.

  Conventionally, various types of image processing have been performed on digital image data. However, in the case of conventional image processing, appropriate brightness correction has not been performed.

  For example, when obtaining the average value of the brightness of the image to be imaged, it is preferable to give a large weight to the lightness of the main part and to give a small weight to the unnecessary background part.

  In the case of conventional image processing, a pixel having a sharp image is determined as a main pixel based on an empirical fact that the main portion is sharper than the other portions. When the pixel data is composed of pixels in a dot matrix shape, the difference in pixel data between adjacent pixels becomes large at the edge portion of the image. Therefore, the difference corresponding to the luminance gradient is referred to as an edge degree, and a pixel having an edge degree larger than a predetermined value is determined as a main part pixel having a sharp image.

  However, it includes pixels with a small edge degree, such as the face of a person, but pixels with a high edge degree, even though it is not the main part, such as a pattern (check pattern) of clothes worn by a person or a wall with a pattern. There is an image. In the case of such an image, appropriate brightness correction has not been made.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, a method, and a recording medium on which an image processing control program is recorded that can perform appropriate brightness correction.

  In view of the above problems, the invention according to claim 1 is an image processing apparatus that performs predetermined image processing on a target image, and divides the target image into a plurality of regions, and averages luminance values for each region. A luminance average value calculating means for obtaining a reference area selecting means for selecting, as a reference area, an area having the lowest luminance average value from areas occupied by a main part of the target image; and a luminance average value of the selected reference area The brightness reference value includes brightness correction means that performs sampling and brightness correction based on the difference between the brightness reference value and the average brightness value of each area and the position of each area.

  According to the image processing apparatus that performs predetermined image processing on the target image configured as described above, first, the target image is divided into a plurality of regions by the luminance average value calculation unit, and the luminance average for each region is obtained. A value is determined. Then, the reference area selection means selects the area having the lowest luminance average value from the areas occupied by the main part of the target image as the reference area, and the brightness correction means determines the luminance average value of the selected reference area as the luminance reference value. As values, sampling is performed based on the difference between the luminance reference value and the average luminance value of each region, and the position of each region, and brightness correction is performed.

  The invention according to claim 2 is the image processing apparatus according to claim 1, and is configured to set an area occupied by a main part of the target image based on the shape of the target image.

  Furthermore, the invention according to claim 3 is the image processing apparatus according to claim 1, wherein if the target image is vertically long, an area occupied by a main part of the target image is set to be vertically long, and the target image is If it is horizontally long, the area occupied by the main part of the target image is configured to be horizontally long.

  The invention according to claim 4 is the image processing apparatus according to any one of claims 1 to 3, wherein the plurality of divided areas are divided into an inner peripheral area and an outer peripheral area, The reference area is selected from the inner peripheral area.

  Furthermore, the invention according to claim 5 is the image processing device according to claim 4, wherein the number of area divisions of the inner peripheral area is larger than the number of area divisions of the outer peripheral area. .

  The invention according to claim 6 is the image processing apparatus according to any one of claims 1 to 5, wherein the difference between the luminance reference value and the luminance average value of each region, and the position of each region. The brightness average value of the target image is calculated, and when the brightness average value of the target image is equal to or higher than the brightness correction reference value, the brightness correction is performed so that the target image is darkened. If it is less than the lightness correction reference value, the lightness is corrected so that the target image is brightened.

  Furthermore, the invention according to claim 7 is the image processing apparatus according to claim 6, wherein when the difference between the luminance reference value and the luminance average value of each region is equal to or larger than a predetermined value, the corresponding region is determined. It is comprised so that it may exclude from the calculation reference | standard of the brightness | luminance average value of an object image.

  The invention according to claim 8 is the image processing apparatus according to claim 7, wherein the predetermined value is set to a value different between the inner peripheral region and the outer peripheral region, and the predetermined value in the inner peripheral region is set to the outer peripheral region. It is comprised so that it may become a bigger value than the predetermined value in.

  In view of the above problem, the invention according to claim 9 is an image processing method for performing predetermined image processing on a target image, and divides the target image into a plurality of regions, and averages luminance values for each region. A luminance average value calculating step, a reference region selecting step for selecting a region having the lowest luminance average value from the regions occupied by the main part of the target image as a reference region, and the luminance average value of the selected reference region The brightness reference value includes a brightness correction step of performing sampling and brightness correction based on the difference between the brightness reference value and the brightness average value of each area, and the position of each area.

  According to the image processing method for performing predetermined image processing on the target image configured as described above, first, the target image is divided into a plurality of regions by the luminance average value calculating step, and the luminance average for each region is calculated. A value is determined. Then, the region having the lowest luminance average value is selected as the reference region from the regions occupied by the main part of the target image by the reference region selection step, and the luminance average value of the selected reference region is determined by the brightness correction step. As the reference value, sampling is performed based on the difference between the luminance reference value and the average luminance value of each region, and the position of each region, and brightness correction is performed.

  In view of the above problems, the invention according to claim 10 is a computer-readable recording medium that records a program for causing a computer to execute image processing for performing predetermined image processing on a target image. Divide the target image into multiple areas and calculate the average brightness value for each area, and select the area with the lowest average brightness value from the areas occupied by the main part of the target image as the reference area The reference area selection processing to be performed, the luminance average value of the selected reference area as a luminance reference value, sampling is performed based on the difference between the luminance reference value and the luminance average value of each area, and the position of each area, A program for causing a computer to execute brightness correction processing for performing brightness correction is recorded and can be read by the computer.

  The computer-readable recording medium configured as described above stores a program for causing a computer to execute image processing processing for performing predetermined image processing on a target image. Then, the target image is divided into a plurality of areas by the brightness average value calculation process, and the brightness average value for each area is obtained, and the brightness average value is calculated from the areas occupied by the main part of the target image by the reference area selection process. The lowest region is selected as the reference region. Further, by the brightness correction process, sampling is performed based on the difference between the luminance reference value and the average luminance value of each area, and the position of each area, using the average luminance value of the selected reference area as the luminance reference value. Lightness correction is performed.

  FIG. 1 is a block diagram illustrating an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied. FIG. 2 is a schematic block diagram illustrating a specific hardware configuration example.

  In FIG. 1, the image input device 10 outputs real image data (original image data) represented as a dot matrix pixel such as a photograph to the image processing device 20. The image processing apparatus 20 performs image processing after determining the content and degree of image processing through predetermined processing, and then outputs image processed image data (processed image data) to the image output apparatus 30. . The image output device 30 outputs the image-processed image as a dot matrix pixel.

  Here, the image data output from the image processing device 20 is obtained by performing image processing by determining a luminance reference value under a predetermined condition, performing luminance weighting based on the luminance reference value, and performing brightness correction. It is. The image processing apparatus 20 includes a luminance average value calculation unit 20a, a reference area selection unit 20b, an unnecessary area removal unit 20c, a weighted luminance average value calculation unit 20d, and a brightness correction unit 20e. Details of the data processing of each component will be described later.

  A specific example of the image input apparatus 10 corresponds to the digital still camera 12 or the video camera 14 in FIG. A specific example of the image processing apparatus 20 corresponds to a computer system including a computer 21, a hard disk 22, a keyboard 23, a CD-ROM drive 24, a floppy disk drive 25, a modem 26, and the like. A specific example of the image output device 30 corresponds to a printer 31, a display 32, and the like. The modem 26 is connected to a public telephone line and is connected to an external network via the public communication line so that software and data can be downloaded.

  The image processing control program according to the present invention is normally recorded and distributed on a recording medium such as a floppy disk or a CD-ROM in a form readable by the computer 21. The program is read by a media reader (CD-ROM drive 24, floppy disk drive 25, etc.) and installed in the hard disk 22. The CPU is configured to read out a desired program from the hard disk 22 as appropriate and execute a desired process.

  In this embodiment, the scanner 11 or the digital still camera 12 as the image input device 10 outputs RGB (green, blue, red) gradation data as image data, and the printer 31 as the image output device 30 includes: , CMY (cyan, magenta, yellow) as gradation data or CMYK binary data obtained by adding black to this is required as input. The display 32 requires RGB gradation data as an input. On the other hand, the computer 21 includes an operating system 21a, a printer driver 21b and a display driver 21c corresponding to the printer 31 and the display 32. The image processing application 21d is controlled by the operating system 21a to execute processing, and performs predetermined image processing together with the printer driver 21b and the display driver 21c as necessary. Accordingly, the specific role of the computer 21 as the image processing apparatus 20 is to create RGB gradation data that has been subjected to optimal image processing by inputting RGB gradation data, and displays it via the display driver 21c. 32, and is converted into CMY (or CMYK) binary data via the printer driver 21b and printed on the printer 31.

As described above, in this embodiment, a computer system is incorporated between image input / output devices to perform image processing. However, the computer system is not necessarily required, and image data is not necessarily processed. Any system that performs desired image processing may be used. For example, as shown in FIG. 3, an image processing device that performs desired image processing is incorporated in the digital still camera 12a, color conversion is performed inside the digital still camera, and the converted image data is stored in a memory. ,
A system in which the converted image data is displayed on the display 32a or printed on the printer 31a may be used. As shown in FIG. 4, in the printer 31b that inputs and prints image data without using a computer system, the image data input through the scanner 11b, the digital still camera 12b, the modem 26b, or the like is used. It can also be configured to perform desired image processing automatically.

  Hereinafter, desired image processing by the image processing apparatus 20 shown in FIG. 1 will be described with reference to FIG. Specifically, the desired image processing is performed by the image processing program corresponding to the flowchart shown in FIG. 5 in the computer 21, and the main subject is arranged near the center, and the luminance distribution of the entire image. This is based on the premise that information that is not necessary for correction is held in areas that deviate significantly from the above.

  First, the luminance average value calculation unit 20a in the image processing apparatus 20 divides a target image to be subjected to desired image processing into 14 areas (a0 to a13) as shown in FIG. (Histogram information) is created, and an average luminance value Yave for each area is obtained (step 40).

  In FIG. 6, the 14 divided areas are composed of the entire image a0, nine inner peripheral areas a1 to a9, and four outer peripheral areas a10 to a13. In the present embodiment, the nine inner peripheral areas a1 to a9 have the same shape, and are arranged in three rows in the vertical and horizontal directions as shown in FIG. W / 5 outer peripheral areas are formed. H indicates the height of the target image, and W indicates the width of the target image. In this embodiment, the target image is divided as described above, but the number of area divisions and the division method in the inner peripheral area and the outer peripheral area are arbitrary.

  Next, the reference area selection unit 20b in the image processing apparatus 20 determines an attention area (step 42) and selects a reference area (step 44).

  Specifically, when the target image is vertically long (H ≧ W), a2, a5, and a7 are selected as attention areas, and when the target image is horizontally long (H <W), a4, a5, and a6 are selected. Select as the area of interest. This is based on the probability of the location of the main subject relative to the orientation of the photograph. In other words, this is based on an empirical fact that a vertically long building or the like is not imaged with a horizontal composition.

  Next, an area with the lowest average brightness value is selected as the reference area from the attention areas, and the average brightness value of the reference area is set as the exposure reference value Ystd. The reason for selecting the reference area in this manner is that a bright image is considered to have a wider allowable range for image processing than a dark image, and in the case of backlight photography, the exposure reference value is set to a low value. This is because the background can be easily removed. Of the attention area, an area having a median luminance value may be used as the reference area.

  Next, when the absolute value of the difference between the exposure reference value Ystd of the target image and the luminance average value of each area is equal to or greater than a predetermined threshold, the unnecessary area exclusion unit 20c in the image processing apparatus 20 It judges and excludes from the following processes (steps 46-52).

Specifically, regarding the inner peripheral areas a1 to a9 excluding the attention area,
| Y _std −Y _ave [i] | ＞ Th1?
It is determined whether or not (step 46). If | Y _std −Y _ave [i] |> Th1 (step 46, Yes), the area ai where | Y _std− Y _ave [i] |> Th1 is excluded (step 48).

Moreover, regarding the outer peripheral areas a10 to a13,
| Y _std −Y _ave [i] | ＞ Th2?
It is determined whether or not (step 50). If | Y _std −Y _ave [i] |> Th2 (step 50, Yes), the area ai where | Y _std −Y _ave [i] |> Th2 is excluded (step 52).

  Thus, the determination threshold is set to a different value (Th1> Th2) between the inner peripheral area and the outer peripheral area. Since the inner peripheral area is near the center and there is a high possibility that the main subject exists, the threshold value Th1 is set to be relatively large. On the other hand, the outer peripheral area seems to have a high possibility of background, so the threshold value Th2 is set to be relatively small so that it is relatively easily excluded. The areas a8 and a9 are located near the center as in the case of a1 and a3. In many cases, however, the background (for example, the sky) is likely to enter the upper half. It can also be included.

  Next, the weighted luminance average value calculation unit 20d in the image processing device 20 adds and weights the average luminance value of each area according to the position of each area regarding the area that has not been removed by the unnecessary area exclusion unit 20c. (Step 54).

  In the present embodiment, the weighting coefficients of the reference area, the inner peripheral area other than the reference area, and the outer peripheral area are set to 3: 2: 1. Other weighting factors can also be used. That is, if the weighted sum ΣYave of the average values is not excluded in steps 46 and 50 and the reference area is the center a5,

によって求まる。そして、対象画像の輝度平均値

It is obtained by. And the luminance average value of the target image

は、重み付け有効エリア数をａ_numとすると、

If the number of weighted effective areas is a _num ,

によって求まる（ステップ５６）。ここで、ａ_num＝[基準エリア数]×３＋[基準エリアを除く、有効内周エリア数]×２＋[有効外周エリア数]×１である。除外エリアがある場合は、式（１）および式（２）に関して、当該エリアは計算から除外される。

(Step 56). Here, a _num = [reference area number] × 3 + [effective inner peripheral area excluding reference area] × 2 + [effective outer peripheral area number] × 1. If there is an excluded area, the area is excluded from the calculation with respect to Equation (1) and Equation (2).

  Next, the brightness correction unit 20e in the image processing device 20 calculates the average brightness value of the target image.

に基づき、明度（輝度値）を補正する（ステップ５８）。具体的には、

Based on the above, the brightness (luminance value) is corrected (step 58). In particular,

によって画像全体の明るさを自動補正する。

Automatically corrects the brightness of the entire image.

  Here, Yideal is a light and dark reference value (lightness correction reference value), and the average luminance value of the target image

が明暗の基準値Ｙidealより小さい場合には明度が大きくなるように、大きい場合には小さくなるようにガンマγの値が計算される。当該ガンマγの値を用いて
Ｙ’＝２５５×（Ｙ／２５５）^γ
によりプリンタドライバ２１ｂまたはディスプレイドライバ２１ｃのルックアップテーブルを作成する。ここで、Ｙは元画像の輝度値であり、Ｙ’は明度補正後の輝度値である。

The value of gamma γ is calculated so that the brightness increases when the brightness is smaller than the light / dark reference value Yideal and decreases when the brightness is larger. Using the value of gamma γ, Y ′ = 255 × (Y / 255) ^γ
Thus, a lookup table for the printer driver 21b or the display driver 21c is created. Here, Y is the luminance value of the original image, and Y ′ is the luminance value after lightness correction.

  It is also possible to correct the brightness using a tone curve instead of correcting the brightness in this way. That is, the difference between the brightness reference value (brightness correction reference value) and the average brightness value of the target image

に基づきトーンカーブを制御して明度補正ＬＵＴ（色変換テーブル）を生成する。

The tone curve is controlled based on the above to generate a brightness correction LUT (color conversion table).

  FIG. 7 shows an example of a tone curve for performing brightness correction. As shown in FIG. 7, the tone curve of the lightness correction LUT is a spline that passes through three points of gradation “0”, gradation “255”, and control point (in this embodiment, gradation “64”). Obtained by smoothly interpolating with curves.

  FIG. 8 shows an example of the average brightness value of each area of the target image. Regarding this example, the desired image processing according to the flowchart of FIG. 5 will be described as follows.

  In this example, it is assumed that the luminance average value Yave for each area as shown in FIG. 8 is obtained by the luminance average value calculation unit 20a (step 40).

  Assuming that the target image has a vertical composition, the attention areas are a2, a5, and a7 (step 42), and the reference area selection unit 21b has the lowest average luminance value Yave [2] (= 12) in the attention area. ) Is selected as a reference area (step 44). Therefore, the exposure reference value Ystd is the lowest luminance average value Yave [2] (= 12) in the attention area.

  Next, the unnecessary area exclusion unit 20c obtains an absolute value dif of the difference between the luminance average value and the exposure reference value. The average brightness value for each area and the absolute value dif of the difference between the average brightness value and the exposure reference value are as follows.

さらに、不要エリア除外部２０ｃは、内周エリア（ａ１〜ａ９）に対するエリア判定閾値Ｔｈ１を１００、外周エリア（ａ１０〜ａ１３）に対するエリア判定閾値Ｔｈ２を５０として、不要なエリアを除外する（ステップ４６〜５２）。

Furthermore, the unnecessary area exclusion unit 20c excludes unnecessary areas by setting the area determination threshold Th1 for the inner peripheral areas (a1 to a9) to 100 and the area determination threshold Th2 for the outer peripheral areas (a10 to a13) to 50 (step 46). ~ 52).

  In the example shown in FIG. 8, four areas (a8, a9, a12, a13) are determined as unnecessary areas and excluded. Therefore, the main area can be sampled with priority.

Next, the weighted luminance average value calculation unit 20d weights and adds the luminance average value according to whether each area belongs to the inner peripheral area or the outer peripheral area, and calculates ΣYave as follows. Obtain (step 52).
ΣYave = a2 × 3 + (a1 + a3 + a4 + a5 + a6 + a7) × 2 + (a10 + a11) = 36 + 516 + 74 = 626
The average brightness of the target image is

となる。ここで、有効エリア数ａ_numは、
ａ_num＝[基準エリア数]×３＋[基準エリアを除く、有効内周エリア数]×２＋[有効外周エリア数]×１＝１×３＋６×２＋２×１＝１７
によって計算される。

It becomes. Here, the number of effective areas a _num is
a _num = [number of reference areas] × 3 + [number of effective inner peripheral areas excluding reference areas] × 2 + [number of effective outer peripheral areas] × 1 = 1 × 3 + 6 × 2 + 2 × 1 = 17
Calculated by

  Note that the average luminance value when the conventional object sampling is performed for the same target image is 78. Therefore, the target image is recognized as a darker image by multi-division sampling, and more accurate correction is performed. Is confirmed.

  According to this embodiment, two-step area determination thresholds are set for the outer peripheral area and the inner peripheral area, unnecessary areas are removed, and main areas are weighted by applying a large coefficient. The brightness correction is performed by calculating the brightness average value of the target image and correcting the brightness value. For this reason, important information can be sampled more accurately, and appropriate brightness correction can be performed. Unlike the case of the conventional object sampling, it is not necessary to extract an edge, so the influence due to the difference in sampling rate is considerably reduced.

  According to the image processing apparatus according to claim 1, the image processing method according to claim 7, or the processing of the program recorded in the recording medium according to claim 8, the main part of the target image is occupied A region having the lowest luminance average value is selected as a reference region from among the regions, and the luminance average value of the selected reference region is used as a luminance reference value, and the difference between the luminance reference value and the luminance average value of each region, Since the sampling is performed and the brightness correction is performed based on the position of each region, the main part having a low brightness but important can be sampled more accurately, and the appropriate brightness correction can be performed.

1 is a block diagram illustrating an image processing system to which an image processing apparatus according to an embodiment of the present invention is applied.

It is a schematic block diagram which shows the specific hardware structural example.

It is a schematic block diagram which shows the other application example of the image processing apparatus by this invention.

It is a schematic block diagram which shows the other application example of the image processing apparatus by this invention.

It is a flowchart for demonstrating the desired image processing of the image processing apparatus by this invention.

It is a figure which shows the target image which performs desired image processing.

It is a figure for demonstrating the brightness correction which uses a tone curve.

It is a figure which shows an example of the luminance average value of each area of a target image.

Explanation of symbols

11 Scanner, 12 Digital still camera, 14 Video camera 21 Computer, 23 Keyboard, 24 CD-ROM drive 25 Floppy disk drive, 26 Modem 31 Printer, 32 Display

Claims (10)

An image processing apparatus that performs predetermined image processing on a target image,
By dividing the target image into a plurality of regions, the luminance average value calculation means for calculating an average luminance value for each region,
A reference region selecting means for selecting the lowest area of the brightness average value as the reference area from the area occupied by the main portion of the target image,
Using the luminance average value of the selected reference area as the luminance reference value, the area where the difference between the luminance reference value and the luminance average value of each area is greater than or equal to each predetermined value according to the position of each area is determined. Area discriminating means to perform,
Brightness correction means for performing brightness correction on the target image based on the luminance average value of the target image based on the luminance average value of the area other than the area determined by the area determination means ;
An image processing apparatus.   The image processing apparatus according to claim 1, wherein an area occupied by a main part of the target image is set based on the shape of the target image.   The area occupied by the main part of the target image is set to be vertically long if the target image is vertically long, and the area occupied by the main part of the target image is set to be horizontally long if the target image is horizontally long. Image processing device. By dividing the plurality of divided regions into an inner region and the outer region, the image according to claims 1 to 3 Neu not Re or claim the reference region is selected from among the inner area Processing equipment.   The image processing apparatus according to claim 4, wherein the number of area divisions of the inner peripheral area is larger than the number of area divisions of the outer peripheral area. Wherein when the average luminance value of the target image is greater than or equal brightness correction reference value brightness correction to darken the target image, the target image when the luminance average value of the target image is less than the brightness correction reference value the image processing apparatus according to claim 1 to 5 Neu not Re or claim to brightness correction to brighten. The luminance average value of the target image is added by weighting the luminance average value of the area excluding the area determined by the area determination unit according to the position of each area. The image processing apparatus according to one item. By dividing the plurality of divided regions into an inner region and the outer region, the predetermined value in the inner peripheral region, any one of claims 1 to 7, a value larger than the predetermined value in the outer peripheral region The image processing apparatus according to item . An image processing method for performing predetermined image processing on a target image,
By dividing the target image into a plurality of regions, the luminance average value calculation step of calculating an average luminance value for each region,
A reference region selecting step of selecting the lowest area of the brightness average value as the reference area from the area occupied by the main portion of the target image,
Using the luminance average value of the selected reference area as the luminance reference value, the area where the difference between the luminance reference value and the luminance average value of each area is greater than or equal to each predetermined value according to the position of each area is determined. An area discrimination process to be performed;
A brightness correction step of performing brightness correction on the target image based on the average brightness value of the target image based on the average brightness value of the region other than the region determined by the region determination step ;
An image processing method comprising: A recording medium readable by the recording a computer program for executing the image processing to perform a predetermined image processing in the computer for the target image,
By dividing the target image into a plurality of regions, the average luminance value calculation processing for calculating the average luminance value for each region,
A reference region selecting process of selecting the lowest area of the brightness average value as the reference area from the area occupied by the main portion of the target image,
Using the luminance average value of the selected reference area as the luminance reference value, the area where the difference between the luminance reference value and the luminance average value of each area is greater than or equal to each predetermined value according to the position of each area is determined. Area determination processing to be performed,
A brightness correction process for performing brightness correction on the target image based on the average brightness value of the target image based on the average brightness value of the area other than the area determined by the area determination process ;
A computer-readable recording medium on which a program for causing a computer to execute is recorded.

Images