JP2008021219A - Image processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress unnatural edge enhancement likely to occur near the angle of an image. <P>SOLUTION: In this image processing apparatus for using an edge enhancement signal created by a two-dimensional filter or the like to execute edge enhancement processing, whether there is a corner of an image around the target pixel of the edge enhancement processing is determined on the basis of the pixel signal of pixels around the target pixel, and an edge enhancement gain to be multiplied by the edge enhancement signal is determined on the basis of the determination result. Corner determination values are calculated as an index corresponding to the presence of the corner of the image for each of the upper left, lower left, upper right and lower right of the target pixel to calculate an edge enhancement gain on the basis of the corner determination values. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that performs edge enhancement processing on an input image and an image processing method used therefor. The present invention also relates to an electrical apparatus such as an imaging apparatus and a display apparatus that includes the image processing apparatus.

  Edge enhancement processing for enhancing the contour (edge) of an image has been known for a long time (for example, see Non-Patent Document 1 below). In the edge enhancement processing, an edge enhancement signal is generally generated using a two-dimensional filter having a predetermined matrix size, and the edge enhancement signal is added to a target signal (luminance signal or the like) for edge enhancement processing.

Ryoichi Suematsu (1 other), “Image Processing Engineering”, first edition, Corona Inc., October 26, 2000, p. 100-103

  Usually, a signal obtained by multiplying an edge enhancement signal created using a two-dimensional filter by a certain gain is added to a target signal for edge enhancement processing. The outline of the image is enhanced by such processing, but depending on the image, unnatural edge enhancement may be performed by such uniform processing.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing method, and an electrical apparatus that contribute to suppression of unnatural edge enhancement.

  In order to achieve the above object, according to a first image processing apparatus of the present invention, an image processing apparatus that performs edge enhancement processing on an input image has an image corner around a target pixel of the edge enhancement processing. Whether or not the pixel is present is determined based on pixel signals of peripheral pixels of the target pixel, and the edge enhancement degree for the target pixel is determined based on the determination result.

  When corners of an image exist around the target pixel of the edge enhancement processing, if uniform edge enhancement is performed on the pixel signal of the target pixel, unnatural edge enhancement may be added to the part. Considering this, the degree of edge enhancement (for example, edge enhancement gain multiplied by the edge enhancement signal) is determined according to the presence of the corner of the image. This makes it possible to suppress unnatural edge enhancement. Note that the corner of the image is, for example, an image portion where the shading of the image changes steeply both in the horizontal direction and in the vertical direction.

  Specifically, for example, the first image processing apparatus is configured to calculate the target based on a horizontal edge component and a vertical edge component around the target pixel calculated based on a pixel signal of the peripheral pixel of the target pixel. It is determined whether or not an image corner exists around the pixel.

  In order to achieve the above object, the second image processing apparatus according to the present invention treats each of a plurality of pixels forming an input image as a target pixel and outputs an edge enhancement signal for the pixel signal of each target pixel. Based on the edge enhancement signal generating means to be generated and the pixel signals of the peripheral pixels of the target pixel, the horizontal edge component and the vertical edge component around the target pixel are calculated, and the calculated horizontal edge component and the vertical edge component are calculated Based on both, a contribution degree calculating means for calculating an enhancement signal contribution degree representing a degree of reflecting the edge enhancement signal in the pixel signal of the target pixel, and based on the edge enhancement signal and the enhancement signal contribution degree, Edge enhancement means for performing edge enhancement processing on the pixel signal of the target pixel.

  By using both the horizontal edge component and the vertical edge component around the target pixel, it is possible to evaluate whether a corner of the image exists around the target pixel. Then, it is possible to suppress unnatural edge enhancement by performing edge enhancement processing based on the enhancement signal contribution degree (for example, edge enhancement gain) reflecting the evaluation result and the edge enhancement signal. Become.

  Specifically, for example, in the second image processing apparatus, the edge enhancement unit performs the edge enhancement process on the pixel signal of the target pixel according to a signal calculated from the edge enhancement signal and the enhancement signal contribution degree. Apply.

  More specifically, for example, in the second image processing apparatus, the contribution calculation means defines a plurality of detection areas in a predetermined image area including the target pixel, and pixels of the pixels in each detection area An evaluation value calculation unit that calculates an evaluation value based on a signal for each detection area and an evaluation value corresponding to a detection area adjacent to each other in the horizontal direction, an edge component between the adjacent detection areas is used as the horizontal edge component. On the other hand, it comprises edge component calculation means for calculating an edge component between adjacent detection areas as the vertical edge component from evaluation values corresponding to detection areas adjacent to each other in the vertical direction.

  Instead of this, specifically, for example, in the second image processing apparatus, the contribution degree calculation means defines a plurality of detection areas in a predetermined image area including the target pixel, and sees from the target pixel. In the case where the different detection areas located in the first, second, third, and fourth directions are the first detection area, the second detection area, the third detection area, and the fourth detection area, respectively, each detection area Corresponding to the kth detection area with respect to each of the evaluation value calculation means for calculating the evaluation value based on the pixel signal of each of the pixels for each detection area, and k = 1, k = 2, k = 3, and k = 4 While calculating the kth horizontal edge component according to the difference between the evaluation value to be detected and the evaluation value of the detection area adjacent in the horizontal direction of the kth detection area as the horizontal edge component, the evaluation value corresponding to the kth detection area And the kth detection area Edge component calculating means for calculating the k-th vertical edge component corresponding to the difference between the evaluation values of the detection areas adjacent in the straight direction as the vertical edge component, and depending on the first horizontal edge component and the first vertical edge component Candidate contributions, candidate contributions according to the second horizontal edge component and second vertical edge component, candidate contributions according to the third horizontal edge component and third vertical edge component, and fourth horizontal edge component And a candidate contribution degree according to the fourth vertical edge component, and a selection means for selectively selecting one candidate contribution degree from the four candidate contribution degrees, the selection means The enhancement signal contribution degree is calculated on the basis of the candidate contribution degree selected by (1).

  In order to achieve the above object, a first image processing method according to the present invention is an image processing method for performing edge enhancement processing on an input image. Whether a corner exists is determined based on pixel signals of peripheral pixels of the target pixel, and an edge enhancement degree for the target pixel is determined based on the determination result.

  In order to achieve the above object, the second image processing method according to the present invention treats each of a plurality of pixels forming an input image as a target pixel and outputs an edge enhancement signal for the pixel signal of each target pixel. Based on the first step to be generated and the pixel signals of the peripheral pixels of the target pixel, the horizontal edge component and the vertical edge component around the target pixel are calculated, and both the calculated horizontal edge component and the vertical edge component are calculated. A second step of calculating an enhancement signal contribution representing the degree to which the edge enhancement signal is reflected in the pixel signal of the target pixel, and the target pixel based on the edge enhancement signal and the enhancement signal contribution And a third step of performing edge emphasis processing on the pixel signal.

  In order to achieve the above object, an electrical apparatus according to the present invention includes the first or second image processing device and an imaging device that outputs an image obtained by photographing to the image processing device as the input image. And at least one of a display unit that displays an image after the edge enhancement processing by the image processing device.

  The present invention contributes to suppression of unnatural edge enhancement.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle.

<< First Embodiment >>
FIG. 1 is a schematic block diagram of the overall configuration of an imaging apparatus 1 according to the first embodiment of the present invention. The imaging device 1 is, for example, a digital still camera or a digital video camera. The imaging device 1 includes an imaging unit 2, an image processing unit 3, a display unit 4, and a recording medium 5.

  The imaging unit 2 includes an imaging device such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS) image sensor, an optical system, an amplifier, an A / D converter, and the like (all not shown), and is obtained by photographing. Captured image data representing the received image (hereinafter referred to as a captured image) is sent to the image processing unit 3.

  The image processing unit 3 generates a video signal representing a captured image based on the captured image data. The video signal is composed of a luminance signal (Y) that represents the luminance of the captured image and a color difference signal (U and V) that represents the color of the captured image. The image processing unit 3 performs necessary video processing including edge enhancement processing on the generated video signal, and outputs the video signal after the video processing to the display unit 4. The display unit 4 includes a liquid crystal display panel and the like, and displays a given video signal as an image.

  Further, the image processing unit 3 compresses the video signal after the video processing as necessary, and then sends it to the recording medium 5. The recording medium 5 is composed of a memory card or the like and records a given video signal. Although not shown, the imaging device 1 may further include a CPU (Central Processing Unit), a microphone, an audio signal processing unit, a speaker, an operation unit, and the like as necessary.

  As described above, the image processing unit 3 has a function of performing edge enhancement processing. This edge enhancement processing is realized by, for example, an edge enhancement processing unit (edge enhancement processing device) 10 as shown in FIG. FIG. 2 is an internal block diagram of the edge enhancement processing unit 10, and the edge enhancement processing unit 10 is provided in the image processing unit 3 of FIG. The edge enhancement processing unit 10 includes an edge enhancement signal generation unit 11, an edge enhancement gain calculation unit (contribution degree calculation unit) 12, and an edge enhancement unit 13. The edge enhancement unit 13 includes a multiplier 14 and an adder 15.

  Input image data represented by the video signal generated by the image processing unit 3 is given to the edge enhancement processing unit 10. The edge enhancement processing unit 10 performs edge enhancement processing on the given input image data, and outputs the image data after the edge enhancement processing as output image data.

  FIG. 3 shows an arrangement of each pixel that forms an image represented by input image data or output image data. Each pixel is arranged in a matrix. This array is regarded as a matrix of M rows and N columns with the origin X of the image as a reference, and each pixel is represented by P (m, n). Here, m represents the vertical position of the pixel in the image, and takes an integer between 1 and M. n represents the horizontal position of the pixel in the image, and takes an integer between 1 and N. Hereinafter, an image represented by input image data is referred to as an input image, and an image represented by output image data is referred to as an output image.

  In the description of the present embodiment, it is assumed that the target signal for edge enhancement processing is a luminance signal representing luminance. With respect to a certain pixel P (m, n), the luminance of the pixel P (m, n) increases as the value of the luminance signal of the pixel P (m, n) increases. Hereinafter, the value of the luminance signal is simply referred to as a luminance value. In the following description, it is assumed that the luminance value takes a digital value within a range of 0 to 255 for the sake of concrete description.

  In the edge enhancement processing unit 10 in FIG. 2, each of the plurality of pixels P (m, n) forming the input image is handled as a target pixel for edge enhancement processing. However, according to the matrix size of the edge enhancement filter used in the edge enhancement signal generation unit 11, the pixel located at the end of the input image is not a target pixel for edge enhancement processing.

  The edge enhancement signal generation unit 11 in FIG. 2 generates an edge enhancement signal E for each target pixel. The edge enhancement signal generation unit 11 uses, for example, an edge enhancement filter having a 3 × 11 matrix size (a matrix size of 3 pixels in the vertical direction and 11 pixels in the horizontal direction) as shown in FIG. A signal E is generated.

  FIG. 5 shows an area of image data used for calculation by the edge enhancement filter of FIG. In FIG. 5, luminance values a (0) to a (32) are shown as the luminance values of each pixel. In FIG. 5, the luminance value a (16) is the luminance value of the target pixel for edge enhancement processing.

If the target pixel corresponding to the luminance value a (16) is P (m ₀ , n ₀ ),
When i is an integer of 0 to 10, the luminance value a (i) represents the luminance value of the pixel P (m ₀ -1, (n ₀ -5) + i),
When i is an integer of 11 to 21, the luminance value a (i) represents the luminance value of the pixel P (m ₀ , (n ₀ −5) + (i−11)).
When i is an integer between 22 and 32, the luminance value a (i) represents the luminance value of the pixel P (m ₀ +1, (n ₀ −5) + (i−22)).
Here, m ₀ takes each integer between 1 ≦ m ₀ ≦ (M−1), and n ₀ takes each integer between 5 ≦ n ₀ ≦ (N−5).

The edge enhancement signal generation unit 11 calculates the edge enhancement signal E for the target pixel of interest by applying the edge enhancement filter shown in FIG. 4 to the luminance values a (0) to a (32). . The edge enhancement signal E is expressed by the following formula (1). Here, k _i is a filter coefficient. For example, among the 33 pieces of filter coefficients k ₀ to k _32, only k ₁₆ is a 32, others are all -1.

  The edge enhancement gain calculation unit 12 in FIG. 2 calculates an edge enhancement gain G for the edge enhancement signal E for the target pixel of interest. The edge enhancement gain (enhancement signal contribution degree) G represents the degree to which the edge enhancement signal E is reflected in the luminance value of the target pixel in the edge enhancement processing in the edge enhancement unit 13. A method for calculating the edge enhancement gain G will be described later.

  The multiplier 14 calculates the product of the edge enhancement signal E and the edge enhancement gain G. The adder 15 adds the product (E × G) calculated by the multiplier 14 to the luminance value of the target pixel, and the luminance value obtained by adding the product is the luminance value of the target pixel after the edge enhancement processing. Output as. The output value of the adder 15 is a luminance value that forms the output image data of the edge enhancement processing unit 10. In the output image obtained by performing the edge enhancement processing as described above on each target pixel, the contour of the image is enhanced as is well known.

  The edge enhancement gain calculation unit 12 sequentially calculates an appropriate edge enhancement gain G for each target pixel based on the luminance values of the surrounding pixels of the target pixel. However, the significance of the edge enhancement gain calculation unit 12 can be easily understood. In order to do this, first, it is assumed that the edge enhancement gain G given to the multiplier 14 is always constant, and problems that may occur in that case will be described.

  FIG. 6 schematically shows the shading of a partial image that may exist in the input image. Assume that a rectangular image region 101 having 5 pixels in the vertical direction and 20 pixels in the horizontal direction exists in the input image. Further, in the same input image, there are a rectangular frame-shaped (doughnut-shaped) image region 102 that surrounds the four sides of the image region 101, and a square-framed (doughnut-shaped) image region 103 that surrounds the four sides of the image region 102. Suppose it exists. The image area 102 corresponds to an image area obtained by removing the image area 101 from a rectangular image area having 7 pixels in the vertical direction and 22 pixels in the horizontal direction. The image area 103 corresponds to an image area obtained by removing the image areas 101 and 102 from a rectangular image area having 9 pixels in the vertical direction and 24 pixels in the horizontal direction.

  Now, the luminance values of the pixels in the image area 101 are all Y1, the luminance values of the pixels in the image area 102 are all Y2, and the luminance values of the pixels in the image area 103 are all Y3. It shall be. It is assumed that Y1 = 0 and Y1 <Y2 <Y3. In the same input image, it is assumed that the luminance values of the pixels in the image areas other than the image areas 101, 102, and 103 are all 255.

  That is, when attention is focused on luminance, a black square corresponding to the image area 101 exists in the input image, and the periphery of the black square is white. However, gradation areas corresponding to the image areas 102 and 103 exist between the black image area corresponding to the image area 101 and the white pixel area. The gradation area means an area where the gradation changes stepwise.

  7A, 7B, 7C, and 7D are diagrams in which the 3 × 11 edge enhancement filter illustrated in FIG. 4 is superimposed on the partial image portion illustrated in FIG. Now, consider a case where the application area of the edge enhancement filter is moved in the horizontal direction (left direction in the drawing) in a state where the target pixel corresponding to the luminance value a (16) is the pixel in the image region 102. 7A to 7D, reference numeral 110 denotes an application area of the edge enhancement filter, that is, an image area corresponding to the luminance values a (0) to a (32) for the target pixel of interest. 111 represents the position of the target pixel of interest.

  When the target pixel is at the position shown in FIG. 7A, a (0) to a (10) are all Y3, a (11) to a (21) are all Y2, and a (22) to a ( 32) are all Y1. That is, only the edge component in the vertical direction exists in the edge enhancement filter, and the edge enhancement signal E corresponding to the edge component is added to the luminance value of the target pixel, so that the vertical contour is appropriately enhanced. The same applies when the target pixel is located at the position shown in FIG.

  However, if the target pixel is at the position shown in FIG. 7C, a (0) to a (10) are all Y3 and a (11) to a (21) are all Y2, Part of the pixels corresponding to 22) to a (32) enter the horizontal gradation region. For this reason, the edge enhancement signal E of the target pixel becomes excessively large or excessive as compared with the case in FIGS. 7A and 7B, depending on the gradation level, the filter coefficient of the edge enhancement filter, and the like. It can be smaller. As a result, when the edge emphasis gain G given to the multiplier 14 is constant, the target pixel after the edge emphasis process becomes unnaturally black or uncomparable as compared with the cases in FIGS. 7 (a) and 7 (b). The problem of natural whitening arises. The same applies when the target pixel is at the position shown in FIG.

  The occurrence of such a problem mainly stems from the fact that the corner 112 of the image is included in the application area of the edge enhancement filter (see FIG. 7A). When attention is paid to the luminance, the “image corner” means an image portion where the shading of the image changes steeply both in the horizontal direction and in the vertical direction.

  Specifically, the edge enhancement gain calculation unit 12 in FIG. 2 considers the occurrence of the above-described problem so that the degree of edge enhancement with respect to the peripheral image portion of the corner of the image is suppressed. The edge enhancement gain G is dynamically changed so that the edge enhancement gain G for the peripheral image region is relatively low.

  FIG. 8 shows an example of an internal block diagram of the edge enhancement gain calculation unit 12 of FIG. The edge enhancement gain calculation unit 12 in FIG. 8 includes four candidate gain calculation units 21, 22, 23 and 24, a gain selection unit (selection means) 25, and a multiplier 26.

  Hereinafter, for convenience of explanation, unless otherwise specified, attention is paid to one target pixel as a target pixel corresponding to the luminance value a (16), and the operation for the one target pixel will be described. Of course, as described above, each pixel forming the input image is a target pixel, and output image data is generated by performing edge enhancement processing on the luminance value of each target pixel.

  Candidate gain calculation units 21, 22, 23, and 24 are respectively given luminance values a (0) to a (32) included in the input image data (see FIG. 5). The luminance value of the target pixel of interest is a (16) as described above. Pixels corresponding to the luminance values a (0) to a (15) and a (17) to a (32) can be called peripheral pixels of the target pixel.

  The candidate gain calculation unit 21 calculates a candidate gain (candidate contribution) GAIN_a corresponding to the corner of the image that may exist at the upper left of the target pixel. The upper left of the target pixel represents the position of the pixel corresponding to the luminance values a (0) to a (4) with respect to the target pixel.

  Similarly, each of the candidate gain calculation units 22, 23, and 24 calculates a candidate gain GAIN_b corresponding to the corner of the image that can exist at the lower left of the target pixel, and according to the corner of the image that can exist at the upper right of the target pixel. The candidate gain GAIN_c is calculated, and the candidate gain GAIN_d corresponding to the corner of the image that may exist at the lower right of the target pixel is calculated. With reference to the target pixel, the lower left, upper right, and lower right of the target pixel represent the positions of the pixels corresponding to the luminance values a (22) to a (26), and the luminance values a (6) to a (10 ) And the pixel positions corresponding to the luminance values a (28) to a (32).

  Before describing the detailed operation of the candidate gain calculation units 21, 22, 23, and 24, items that are commonly defined in each candidate gain calculation unit will be described. First, an angle detection image area including a target pixel of interest is defined. In the present embodiment, a corner detection image area having the same matrix size as the edge enhancement filter (that is, 3 × 11) is defined. Considering the cost when the edge enhancement processing unit 10 is integrated, it is advantageous that the edge enhancement filter and the corner detection image area have the same matrix size. Of course, both matrix sizes can be made different, and both matrix sizes can be arbitrarily set independently.

  The corner detection image area is composed of 3 × 11 pixels corresponding to the luminance values a (0) to a (32) as in the application area of the edge enhancement filter, and the luminance value a (16) is focused on. This is the luminance value of the target pixel. Then, the corner detection image area is divided into six detection areas AREA (0) to AREA (5) and a remaining area as shown in FIG. 9A.

  The detection area AREA (0) is composed of five pixels corresponding to the luminance values a (0) to a (4). The detection area AREA (1) is composed of five pixels corresponding to the luminance values a (11) to a (15). The detection area AREA (2) is composed of five pixels corresponding to the luminance values a (22) to a (26). The detection area AREA (3) is composed of five pixels corresponding to the luminance values a (6) to a (10). The detection area AREA (4) is composed of five pixels corresponding to the luminance values a (17) to a (21). The detection area AREA (5) is composed of five pixels corresponding to the luminance values a (28) to a (32). The remaining area is composed of three pixels corresponding to the luminance values a (5), a (16) and a (27).

  Each candidate gain calculation unit (candidate gain calculation unit 21 and the like) performs an average value Bave (j of luminance values in each detection area, as shown in FIG. 9B, according to equation (2) consisting of the following six equations: ) Can be calculated (where j is an integer from 0 to 5).

  In addition, the average deviation values of the luminance values in the detection areas AREA (0), AREA (2), AREA (3), and AREA (5) are represented by Basc (0), Basc ( 2), expressed as Basc (3) and Basc (5). They are calculated according to equation (3) consisting of the following four equations:

  Next, detailed operations of the candidate gain calculation units 21, 22, 23, and 24 will be described. FIG. 10 is an internal block diagram of a candidate gain calculation unit that can be used as each of the candidate gain calculation units 21, 22, 23, and 24.

  The functions of the candidate gain calculators 21, 22, 23, and 24 are similar to each other, and the internal block diagrams of the candidate gain calculators 21, 22, 23, and 24 are common. Hereinafter, for the sake of specific description, the description will be given with particular attention to the operation of the candidate gain calculation unit 24 corresponding to the lower right of the target pixel among the candidate gain calculation units 21, 22, 23, and 24. That is, the following description will be made on the assumption that the internal block diagram of FIG. 10 is an internal block diagram of the candidate gain calculation unit 24. A description of some differences that exist between the candidate gain calculation units will be given later.

  In addition, a case where a black corner of an image as shown in FIG. 6 is detected and a function of suppressing the degree of edge enhancement with respect to the peripheral image portion of the black corner is exemplified.

  The candidate gain calculation unit in FIG. 10 includes an average value calculation unit (evaluation value calculation unit) 31, a difference value calculation unit 32, an edge value conversion unit 33, an angle determination value calculation unit 34, and a candidate gain conversion unit 35. . Note that “*” in FIG. 10 is intended to be an integer.

  The average value calculation unit (evaluation value calculation unit) 31 refers to the luminance values a (0) to a (32), and calculates the average value Bave (j) as the evaluation value according to the equation (2) (here) And j is an integer of 0 to 5).

  The difference value calculation unit 32 calculates the horizontal difference value DIF1 and the vertical difference value DIF2 according to the following equations (4a) and (4b). The horizontal difference value DIF1 represented by the equation (4a) represents a horizontal edge component between the detection area (5) and the detection area (2) adjacent to the detection area (5) in the horizontal direction. Expresses the edge. The vertical difference value DIF2 represented by the equation (4b) represents a vertical edge component between the detection area (5) and the detection area (4) adjacent to the detection area (5) in the vertical direction. Expresses the edge. For this reason, it can be said that the difference value calculation unit 32 functions as an edge component calculation unit.

  The edge value conversion unit 33 converts the horizontal difference value DIF1 and the vertical difference value DIF2 into the horizontal edge value EDGE_VALUE1 and the vertical edge value EDGE_VALUE2, respectively, according to the function or table data as shown in FIGS. 11 (a) and 11 (b). Convert.

  As the horizontal difference value DIF1 increases from 0, the horizontal edge value EDGE_VALUE1 increases from 0 (upper limit is 1). When DIF1 ≧ DIF_TH1, the horizontal edge value EDGE_VALUE1 is set to 1. As the vertical difference value DIF2 increases from 0, the vertical edge value EDGE_VALUE2 increases from 0 (upper limit is 1), and when DIF2 ≧ DIF_TH2, the vertical edge value EDGE_VALUE2 is set to 1. DIF_TH1 and DIF_TH2 are predetermined threshold values, and are set to a value between 0 and 255 or less.

  The angle determination value calculation unit 34 calculates the product of EDGE_VALUE1 and EDGE_VALUE2 as the angle determination value COR_VALUE. The angle determination value COR_VALUE is a value between 0 and 1, and the value increases as both the horizontal edge component and the vertical edge component with reference to the detection area AREA (5) increase. The corner determination value COR_VALUE may be regarded as a probability that an image corner exists in the detection area AREA (5).

  The candidate gain conversion unit 35 converts the angle determination value COR_VALUE into a candidate gain according to a function or table data as shown in FIG. Since the operation of the candidate gain calculation unit 24 in FIG. 8 is now described, the angle determination value COR_VALUE is converted into the candidate gain GAIN_d.

The candidate gain (in this case, candidate gain GAIN_d) is GAIN _REF1 when COR_VALUE = 0, GAIN _REF2 when COR_VALUE = 1, and from GAIN _REF1 to GAIN _REF2 as COR_VALUE increases from 0 to 1 It decreases toward.

GAIN _REF1 is a normal gain that does not suppress edge enhancement, and GAIN _REF2 is a maximum suppression gain that suppresses edge enhancement to the maximum. GAIN _REF1 and GAIN _REF2 are set to arbitrary values that satisfy GAIN _REF1 > GAIN _REF2 . If the edge enhancement gain calculation unit 12 in FIG. 8 finally calculates the edge enhancement gain G corresponding to GAIN _REF1 , and gives it to the edge enhancement unit 13 in FIG. 2, the edge enhancement degree is not suppressed. On the other hand, if the edge emphasis gain G corresponding to GAIN _REF2 is given to the edge emphasis unit 13 in FIG. 2, the edge emphasis degree is suppressed to the maximum.

  As described above, the candidate gain calculation unit 24 calculates the candidate gain GAIN_d. The candidate gain calculation units 21, 22 and 23 similarly calculate the candidate gains GAIN_a, GAIN_b and GAIN_c.

However, in the case of the candidate gain calculation unit 21 corresponding to the upper left of the target image, DIF1 and DIF2 are calculated according to the following formulas (5a) and (5b), and the candidate gain GAIN_a is calculated based on the calculated values.
In the case of the candidate gain calculation unit 22 corresponding to the lower left of the target image, DIF1 and DIF2 are calculated according to the following equations (6a) and (6b), and the candidate gain GAIN_b is calculated based on these calculated values.
In the case of the candidate gain calculation unit 23 corresponding to the upper right of the target image, DIF1 and DIF2 are calculated according to the following equations (7a) and (7b), and the candidate gain GAIN_c is calculated based on the calculated values.

  In addition, among each part which comprises the candidate gain calculation parts 21-24, what can be shared between the candidate gain calculation parts 21-24 is shared suitably. For example, it is sufficient to provide one average value calculation unit 31 in FIG. 10 in the edge enhancement gain calculation unit 12 in FIG.

  The gain selection unit 25 in FIG. 8 selects the minimum candidate gain among the four candidate gains GAIN_a, GAIN_b, GAIN_c, and GAIN_d calculated from the candidate gain calculation units 21 to 24, and selects the selected candidate gain as the selection gain SEL_GAIN. Output as. As understood from the above-described candidate gain calculation method, there is a possibility that corners of an image exist in the detection area corresponding to the minimum candidate gain (for example, AREA (5) in FIG. 9A). high.

  In this way, the candidate gain calculators 21, 22, 23, and 24 and the gain selector 25 detect the corners of the image. More specifically, it is detected whether the target pixel of interest exists around the corner of the image. For this reason, it can be considered that the periphery of the corner of the image is detected. Then, the detection result is reflected in the selection gain SEL_GAIN.

  The multiplier 26 in FIG. 8 calculates the product of the selected gain SEL_GAIN output from the gain selector 25 and a predetermined coefficient K. The product calculated by the multiplier 26 is given to the multiplier 14 of FIG. Then, the edge enhancement unit 13 in FIG. 2 adds the product of the edge enhancement signal E and the edge enhancement gain G to the luminance value of the target pixel of interest, thereby performing edge enhancement.

  As described with reference to FIGS. 7A to 7D, if the edge emphasis gain G is always constant, unnatural edge emphasis is performed near the corners of the image. However, in the present embodiment, the edge enhancement gain G corresponding to the presence of the corner of the image is calculated for each target pixel of the edge enhancement process, and the edge enhancement degree for the peripheral image portion of the corner of the image is determined by the edge enhancement gain G. It is suppressed. For this reason, unnatural edge enhancement near the corners of the image is suppressed, and a good resolution and sharpness can be obtained.

  FIG. 13 shows the result of edge enhancement processing by the method of this embodiment. In FIG. 13, reference numeral 121 denotes an input image. Reference numeral 122 represents the edge enhancement signal E corresponding to each pixel of the input image 121 as a grayscale image. Reference numeral 123 denotes an output image when the edge enhancement gain G is constant, and reference numeral 124 denotes an output image when the method of this embodiment is used.

  If the edge emphasis signal (grayscale image) represented by reference numeral 122 is superimposed on the input image 121 with the edge emphasis gain G kept constant, an unnatural area near the corner of the image as in the output image 123 is obtained. Edge enhancement is added. On the other hand, when the method of the present embodiment is used, the edge enhancement gain G for the pixels near the corners of the image is set to a relatively small gain, so that an unnatural edge enhancement near the corners of the image as in the output image 124 is performed. Is appropriately suppressed.

  Note that the edge enhancement gain G is determined in consideration of the average deviation values Basc (0), Basc (2), Basc (3), and Basc (5) expressed by the above equation (3). Also good.

For example, when Basc (0) ≧ Basc_TH is satisfied, GAIN_a is not selected as SEL_GAIN, or GAIN_a = GAIN _{REF1 is} forcibly set.
Similarly, when Basc (2) ≧ Basc_TH is satisfied, GAIN_b is not selected as SEL_GAIN, or GAIN_b = GAIN _{REF1 is} forcibly set.
Similarly, when Basc (3) ≧ Basc_TH is satisfied, GAIN_c is not selected as SEL_GAIN, or GAIN_c = GAIN _{REF1 is} forcibly set.
Similarly, when Basc (5) ≧ Basc_TH is satisfied, GAIN_d is not selected as SEL_GAIN, or GAIN_d = GAIN _{REF1 is} forcibly set.

  Basc_TH is a predetermined threshold for the average deviation value. When the average deviation value is relatively large, the luminance value variation in the detection area corresponding to the average deviation value is large. Such detection areas are unlikely to have image corners, and visual unnaturalness that may occur even if some extra edge enhancement is added in the vicinity of such detection areas. There are few. For this reason, as described above, the edge enhancement gain G may be determined in consideration of the average deviation value.

  Here, a supplementary explanation of “corner of image” is provided. As described above, when attention is focused on the luminance, the “image corner” means an image portion where the shading of the image changes sharply both in the horizontal direction and in the vertical direction. The intersection of the discontinuous point in the horizontal direction of the image and the discontinuous point in the vertical direction of the image can also be expressed as an “image corner”.

In the present embodiment, the edge emphasis gain G is changed so that the degree of edge emphasis on the peripheral image portion of the corner of the image is suppressed. Here, suppressing the degree of edge enhancement corresponds to making the edge enhancement gain G smaller than the normal gain GAIN _REF1 that does not suppress edge enhancement (see FIG. 12). When the “image corner” is in the pixel P (m, n), for example, the pixel P (m−1, n−1) (and P (m−1, n−2) and P (m−1, n) -3), ...), the edge enhancement gain G is suppressed. Therefore, in this case, the “peripheral image portion” includes the pixels P (m−1, n−1) (and P (m−1, n−2) and P (m−1, n−3), ...) is included. As is clear from the above-described method of calculating the edge enhancement gain G, the size of the “peripheral image region” (the number of pixels included in the “peripheral image region”) is the candidate gain calculation unit 21 in FIG. It changes according to the matrix size of the image area for angle detection defined by the above. In the present embodiment, since the matrix size of the corner detection image area is the same as that of the edge enhancement filter shown in FIG. 4, the size of the “peripheral image region” depends on the matrix size of the edge enhancement filter. It can be said that it changes.

  Further, in the above-described method, the “image corner” around the target pixel of the edge enhancement process and the sharpness of the corner are an index called a corner determination value COR_VALUE based on the luminance value of the peripheral pixel of the target pixel. The edge enhancement degree (edge enhancement gain G) for the target pixel is determined based on the angle determination value COR_VALUE.

  Instead, for example, the edge enhancement gain calculation unit 12 may execute the following processing. First, the minimum value MIN among Bave (0) to Bave (5) is specified.

When the minimum value MIN is Bave (0), it is determined whether or not the first condition that “DIF1 and DIF2 represented by the expressions (5a) and (5b) are both greater than a predetermined difference threshold” is satisfied. To do. If the first condition is satisfied, it is determined that the corner of the image exists in AREA (0), and if it is not satisfied, it is determined that the corner of the image does not exist in AREA (0). Even if the first condition is satisfied, if Basc (0) ≧ Basc_TH is satisfied, it may be determined that the corner of the image does not exist in AREA (0) ( This decision is optional.
When the minimum value MIN is Bave (2), it is determined whether the second condition that “DIF1 and DIF2 represented by the expressions (6a) and (6b) are both greater than a predetermined difference threshold” is satisfied. To do. When this second condition is satisfied, it is determined that the corner of the image exists in AREA (2). Even if the second condition is satisfied, if Basc (2) ≧ Basc_TH is satisfied, it may be determined that the corner of the image does not exist in AREA (2) ( This decision is optional.
When the minimum value MIN is Bave (3), it is determined whether or not the third condition that “DIF1 and DIF2 represented by equations (7a) and (7b) are both greater than a predetermined difference threshold” is satisfied. To do. When this third condition is satisfied, it is determined that the corner of the image exists in AREA (3). Even if the third condition is satisfied, if Basc (3) ≧ Basc_TH is satisfied, it may be determined that no corner of the image exists in AREA (3) ( This decision is optional.
When the minimum value MIN is Bave (5), it is determined whether the fourth condition that “DIF1 and DIF2 represented by the expressions (4a) and (4b) are both greater than a predetermined difference threshold” is satisfied. To do. When the fourth condition is satisfied, it is determined that the corner of the image exists in AREA (5). Even when the fourth condition is satisfied, if Basc (5) ≧ Basc_TH is satisfied, it may be determined that the corner of the image does not exist in AREA (5) ( This decision is optional.
In addition, the difference threshold value compared with DIF1 and the difference threshold value compared with DIF2 may be the same, and may differ.

  When it is determined that the corner of the image exists in AREA (0), AREA (2), AREA (2), or AREA (5), the edge enhancement gain G is smaller than a predetermined reference gain. Is provided to the multiplier 14 of FIG. Otherwise, the reference gain is given to the multiplier 14 in FIG. This processing also suppresses the degree of edge enhancement with respect to the peripheral image portion of the corner of the image.

<< Second Embodiment >>
Next, another example of division of the image area for corner detection as shown in FIG. 9A, which is defined by the edge enhancement gain calculation unit 12 of FIG. Another example of division will be described as a second embodiment. The second embodiment is different from the first embodiment only in the way of dividing the corner detection image area, and is the same as the first embodiment in other points.

  In the second embodiment, a detection area (ALER (0) or the like) defined in the corner detection image area is partially different between the candidate gain calculation units 21, 22, 23, and 24 in FIG. FIGS. 14A, 14 </ b> B, 14 </ b> C, and 14 </ b> D represent detection areas defined by the candidate gain calculation units 21, 22, 23, and 24, respectively.

First, in all of the gain calculation units 21, 22, 23 and 24,
The detection area AREA (0) includes pixels corresponding to the luminance values a (0) to a (4),
The detection area AREA (1) includes pixels corresponding to the luminance values a (11) to a (15),
The detection area AREA (2) includes pixels corresponding to the luminance values a (22) to a (26),
The detection area AREA (3) includes pixels corresponding to the luminance values a (6) to a (10),
The detection area AREA (4) includes pixels corresponding to the luminance values a (17) to a (21),
The detection area AREA (5) includes pixels corresponding to the luminance values a (28) to a (32).

  In the candidate gain calculation units 21 and 22, the detection area AREA (3) further includes a pixel corresponding to the luminance value a (5), and the detection area AREA (5) includes a pixel corresponding to the luminance value a (27). . In candidate gain calculation units 23 and 24, detection area AREA (0) further includes a pixel corresponding to luminance value a (5), and detection area AREA (2) includes a pixel corresponding to luminance value a (27). .

  For this reason, the candidate gain calculation units 21 and 22 according to the second embodiment calculate the average value Bave (j) of the luminance values of each detection area according to the equation (8) including the following six equations, j is an integer of 0 to 5), and candidate gains GAIN_a and GAIN_b are calculated based on them.

  On the other hand, the candidate gain calculation units 23 and 24 according to the second embodiment calculate the average value Bave (j) of the luminance values of the respective detection areas according to the equation (9) including the following six equations (j is , And 0 to 5), candidate gains GAIN_c and GAIN_d are calculated based on them.

  In the first embodiment, the luminance values a (5) and a (27) of two pixels adjacent in the vertical direction of the target pixel are not considered in calculating the candidate gain. In the second embodiment, these luminance values are not considered. Is also taken into account in calculating the candidate gain. For this reason, more precise corner detection is performed, and suppression of unnatural edge enhancement is optimized.

  It should be noted that the method of dividing the corner detection image area shown in the first and second embodiments is merely an example, and other division methods may be employed.

  Each embodiment mentioned above is only an example of an embodiment of the present invention, and the present invention includes various modifications. Below, the modification 1-the modification 4 are illustrated as a modification applicable to this invention. The contents described in each modification can be arbitrarily combined as long as there is no contradiction.

[Modification 1]
In each of the above-described embodiments, the function of detecting a black corner of an image as shown in FIG. 6 and suppressing the degree of edge enhancement for a peripheral image portion of the black corner has been described. However, it can also be applied to the white corners of the image. In the case of applying to the white corner, of course, the above-described formulas and the like can be appropriately changed.

[Modification 2]
In addition, the contour correction using the edge enhancement processing is generally performed not only on the luminance signal but also on the color signal in order to enhance the color boundary. In each of the above-described embodiments, the luminance signal (luminance value) is exemplified as the target signal for edge enhancement. However, the method described in each embodiment can also be applied to a color signal (for example, a color difference signal). It is. That is, the pixel signal to be edge-enhanced in each embodiment can be a luminance signal or a color signal (for example, a color difference signal).

[Modification 3]
In each embodiment, the imaging apparatus 1 in FIG. 1 is handled as an electrical device on which the edge enhancement processing unit 10 in FIG. 2 is mounted. In the imaging apparatus 1, the imaging unit 2 outputs an image obtained by shooting to the image processing unit 3 including the edge enhancement processing unit 10. Then, the image after the edge enhancement processing by the edge enhancement processing unit 10 is displayed on the display unit 4. However, the display unit 4 can be omitted from the imaging apparatus 1.

  Further, the edge enhancement processing unit 10 can be mounted on various electric devices other than the imaging device 1. For example, the edge enhancement processing unit 10 can be mounted on a display device (not shown) such as a liquid crystal display television. In this case, for example, the display device includes a display unit similar to the display unit 4 and the image processing unit 3 including the edge enhancement processing unit 10, and the edge enhancement processing unit 10 includes a video signal or recording by broadcasting. A video signal from the medium is supplied.

[Modification 4]
In addition, the imaging apparatus 1 in FIG. 1 can be realized by hardware or a combination of hardware and software. In particular, the function of the image processing unit 3 in FIG. 1 and / or the function of the edge enhancement processing unit 10 in FIG. 2 can be realized by hardware, software, or a combination of hardware and software.

  When the functions of the edge emphasis processing unit 10 are realized using software, FIGS. 2, 8, and 10 represent those functional block diagrams. Further, all or part of the functions realized by the edge enhancement processing unit 10 are described as a program, and the program is executed on the computer so that all or part of the functions are realized. Good.

<< Note >>
In each embodiment, the edge enhancement gain calculation unit 12 in FIGS. 2 and 8 functions as a contribution calculation unit that calculates the edge enhancement gain G as the intensity signal contribution. The average value calculation unit 31 in FIG. 10 functions as an evaluation value calculation unit that calculates an average value Bave (j) as an evaluation value. The difference value calculation unit 32 in FIG. 10 functions as an edge component calculation unit that calculates the horizontal difference value DIF1 and the vertical difference value DIF2 as the horizontal edge component and the vertical edge component.

1 is a block diagram schematically illustrating the overall configuration of an imaging apparatus according to a first embodiment of the present invention. FIG. 2 is an internal block diagram of an edge enhancement processing unit mounted on the image processing unit of FIG. 1. It is a figure which shows the pixel arrangement | sequence of the input-output image of the edge emphasis processing part of FIG. It is a figure which shows the edge emphasis filter used in the edge emphasis signal generation part of FIG. FIG. 5 is a diagram illustrating data used for the calculation using the edge enhancement filter of FIG. 4 and the calculation of the edge enhancement gain calculation unit of FIG. 2. FIG. 3 is a diagram for explaining the significance of the edge enhancement gain calculation unit of FIG. 2, schematically showing the shade of a partial image portion that may exist in the input image to the edge enhancement processing unit of FIG. 2. . It is a figure for demonstrating the significance of the edge emphasis gain calculation part of FIG. It is an internal block diagram of the edge emphasis gain calculation part of FIG. FIG. 9 is a diagram schematically showing each detection area (a) defined by the edge enhancement gain calculation unit in FIG. 8, an average value (b) and an average deviation value (c) calculated corresponding to each detection area. is there. FIG. 9 is an internal block diagram of a candidate gain calculation unit that can be used as each of the candidate gain calculation units of FIG. 8. It is a figure for demonstrating the function of the edge value conversion part of FIG. It is a figure for demonstrating the function of the candidate gain conversion part of FIG. It is a figure which shows the edge enhancement process result using the edge enhancement process part of FIG. It is a figure which shows the division | segmentation method of the image area for angle detection based on 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging part 3 Image processing part 4 Display part 5 Recording medium 10 Edge emphasis processing part 11 Edge emphasis signal generation part 12 Edge emphasis gain calculation part 13 Edge emphasis part 21, 22, 23, 24 Candidate gain calculation part 25 Gain Selection unit 31 Average value calculation unit (evaluation value calculation means)
32 Difference value calculation unit (edge component calculation means)
33 Edge value converter 34 Angle determination value calculator 35 Candidate gain converter

Claims (9)

In an image processing apparatus that performs edge enhancement processing on an input image,
It is determined whether there is a corner of the image around the target pixel of the edge enhancement processing based on the pixel signal of the peripheral pixel of the target pixel, and based on the determination result, the degree of edge enhancement for the target pixel is determined. An image processing apparatus characterized by determining. Based on the horizontal edge component and the vertical edge component around the target pixel calculated based on the pixel signal of the peripheral pixel of the target pixel, it is determined whether an image corner exists around the target pixel. The image processing apparatus according to claim 1. Edge enhancement signal generation means for treating each of a plurality of pixels forming an input image as a target pixel and generating an edge enhancement signal for the pixel signal of each target pixel;
Based on the pixel signal of the surrounding pixel of the target pixel, the horizontal edge component and the vertical edge component around the target pixel are calculated, and the edge enhancement is performed based on both the calculated horizontal edge component and the vertical edge component. Contribution degree calculating means for calculating an enhancement signal contribution degree representing a degree of reflecting a signal in the pixel signal of the target pixel;
An image processing apparatus comprising: an edge enhancement unit that performs an edge enhancement process on a pixel signal of the target pixel based on the edge enhancement signal and the enhancement signal contribution degree. The image processing according to claim 3, wherein the edge enhancement unit performs the edge enhancement processing on a pixel signal of the target pixel according to a signal calculated from the edge enhancement signal and the enhancement signal contribution. apparatus. The contribution calculation means
Defining a plurality of detection areas within a predetermined image area including the target pixel;
Evaluation value calculation means for calculating an evaluation value based on a pixel signal of a pixel in each detection area for each detection area;
From the evaluation value corresponding to the detection areas adjacent to each other in the horizontal direction, while calculating the edge component between the adjacent detection areas as the horizontal edge component, from the evaluation value corresponding to the detection area adjacent to each other in the vertical direction, The image processing apparatus according to claim 3, further comprising an edge component calculation unit that calculates an edge component between adjacent detection areas as the vertical edge component. The contribution calculation means
Defining a plurality of detection areas within a predetermined image area including the target pixel;
Different detection areas located in the first, second, third, and fourth directions when viewed from the target pixel are defined as a first detection area, a second detection area, a third detection area, and a fourth detection area, respectively. If
Evaluation value calculation means for calculating an evaluation value based on a pixel signal of a pixel in each detection area for each detection area;
For each of k = 1, k = 2, k = 3, and k = 4, depending on the difference between the evaluation value corresponding to the kth detection area and the evaluation value of the detection area adjacent to the kth detection area in the horizontal direction While calculating the kth horizontal edge component as the horizontal edge component, the kth vertical according to the difference between the evaluation value corresponding to the kth detection area and the evaluation value of the detection area adjacent in the vertical direction of the kth detection area Edge component calculating means for calculating an edge component as the vertical edge component;
Candidate contributions according to the first horizontal edge component and the first vertical edge component, candidate contributions according to the second horizontal edge component and the second vertical edge component, the third horizontal edge component and the third vertical edge The candidate contribution degree according to the component and the candidate contribution degree according to the fourth horizontal edge component and the fourth vertical edge component are compared, and one candidate contribution degree is selected from the four candidate contribution degrees. And a selection means for selecting one,
The image processing apparatus according to claim 3, wherein the enhancement signal contribution degree is calculated based on the candidate contribution degree selected by the selection unit. In an image processing method for performing edge enhancement processing on an input image,
It is determined whether there is a corner of the image around the target pixel of the edge enhancement processing based on the pixel signal of the peripheral pixel of the target pixel, and based on the determination result, the degree of edge enhancement for the target pixel is determined. An image processing method characterized by determining. A first step of treating each of a plurality of pixels forming an input image as a target pixel and generating an edge enhancement signal for the pixel signal of each target pixel;
Based on the pixel signal of the surrounding pixel of the target pixel, the horizontal edge component and the vertical edge component around the target pixel are calculated, and the edge enhancement is performed based on both the calculated horizontal edge component and the vertical edge component. A second step of calculating an enhancement signal contribution representing a degree of reflecting the signal in the pixel signal of the target pixel;
And a third step of applying an edge enhancement process to the pixel signal of the target pixel based on the edge enhancement signal and the contribution level of the enhancement signal. An image processing device according to any one of claims 1 to 6,
An image pickup unit that outputs an image obtained by photographing to the image processing device as the input image and at least one of a display unit that displays an image after edge enhancement processing by the image processing device. And electrical equipment.