JP2008066942A - Image processor and processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an edge element included in input image data with high precision and to determine coefficients of weighted sum of an input image and a filtered image in response to the edge element. <P>SOLUTION: The image processor is provided with: a parameter determining means for obtaining a parameter to be used for weighting when the input image and an image filtered with respect to the input image are synthesized by using a non-linear function in accordance with information on the edge element detected from the input image; and an image synthesizing means for performing weighted sum to the input image and the filtered image in response to the obtained parameter. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a processing method in the field of moving picture coding.

  In the field of image signal processing units, image processing devices are used for image conversion to remove redundant portions from image data, improve image quality degradation due to noise contained in images, and to analyze image characteristics to extract image features. Is widely used. As one of such image processing, contour correction processing for blurring or conversely enhancing the contour (edge) of the image is performed.

  Standardized by ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) in 2003. In the H.264 moving image compression encoding method, spatial filtering processing for input image data is often performed as preprocessing for an input image to be encoded. The purpose of this spatial filtering process is to remove white noise contained in the image data, to reduce high frequency components for improving the encoding processing efficiency, etc., and a low-pass filter is used as a filter, and the spatial filtering process is performed.

  However, in such spatial filtering processing, only low frequency components remain in the image data that has passed through the low-pass filter, and frequency components in a high frequency region that is equal to or higher than the cutoff frequency of the filter are removed. For this reason, there are almost no edge components (contours) in the image represented by the high-frequency components, and there is a problem that the contour of the output image is blurred and the image looks blurred.

In Patent Document 1 as a prior art for solving such a problem, a high-accuracy contour correction signal suitable for the input of various image signals having different frequency bands is supplied at an appropriate timing. And a signal processing device that corrects the contour of the input image by synchronizing and adding the contour correction signal and the input image signal.
JP-A-2005-176060 “Signal processing device, image display device, and signal processing method”

  However, in this prior art, edge detection is performed using a one-dimensional data string of luminance differences between pixel pairs adjacent to each other or separated by several pixels. Compared to the case where edge information is detected by two-dimensional processing, for example. Thus, there are problems that the detection accuracy is low and that it is difficult to appropriately set a weighting coefficient for weighted addition when adding the contour correction signal and the input image signal.

  In view of the above-described problems, the problem of the present invention is to improve the detection accuracy of the edge component included in the input image data in order to effectively store the edge component of the image data that has passed through the low-pass filter, for example. It is possible to appropriately set a weighting coefficient for adding an image after filtering processing and an input image using parameters corresponding to the edge component, and to efficiently save the edge component of the input image data. .

  The image processing apparatus of the present invention corresponds to the filter means for performing filtering processing on the input original image, the edge detection means for detecting edge component information of the input original image, and the detected edge component information, Parameter determining means for obtaining a parameter used for weighting when combining the filtered image and the input original image as the output of the filter means using a non-linear function, and the input original image corresponding to the obtained parameter And image synthesizing means for performing weighted addition of the filtered image and the filtered image.

  According to the present invention, by using the information of edge components detected two-dimensionally, the accuracy of edge extraction is improved, and parameters for weighted addition between the filtered image and the input original image are set. Since the constant that determines the shape of the nonlinear function used for determination can be set by the operator, the weighted addition coefficient can be appropriately determined.

  FIG. 1 is a block diagram showing the principle configuration of an image processing apparatus according to the present invention. FIG. 1 shows a principle configuration of an image processing apparatus 1 for storing an edge component of an input image. The apparatus 1 includes at least a filter unit 2, an edge detection unit 3, a parameter determination unit 4, and an image synthesis unit 5.

  The filter means 2 receives an input image such as H.264. A filtering process is performed as a preprocessing for an input image to a moving image encoding apparatus such as the H.264 system, and the edge detection unit 3 detects information on the edge component from the input image. Corresponds to the information of the detected edge component, and a parameter at the time of weighted synthesis between the output of the filter means 3 and the input image is obtained using a nonlinear function, and the image synthesis means 5 corresponds to the determined parameter. Thus, the output of the filter means 2 and the input image are combined.

  In the embodiment of the invention, the edge detection means 3 obtains edge information for each pixel corresponding to the pixel data of the two-dimensional peripheral pixels of the edge information detection target pixel, and is also a nonlinear used by the parameter determination means 4. A constant that determines the function form of the function can be set externally, for example, by an operator of the system.

  As described above, in the present invention, the edge component information of the input image is obtained corresponding to the pixel data of the two-dimensional peripheral pixels of the edge information detection target pixel, and the input image and the input image after filtering processing are obtained. Parameters for weighting synthesis are obtained using a nonlinear function, and a constant that determines the function form of the nonlinear function can be set by an operator, for example.

  FIG. 2 is a block diagram showing the basic configuration of an image processing apparatus for storing edge components according to the present invention. In the figure, input image data X (k) is an original input image sent from an external image input block, and the data is given to the line memory 10 and is used for two-dimensional filtering and edge component information extraction. The vertical line or horizontal line data is stored in the line memory 10. Here, k corresponds to the pixel number in the input image data.

  When necessary data is stored in the line memory 10, the data is given to the edge extraction filter 11 and the spatial filter (low-pass filter) 13. The edge extraction filter 11 is, for example, a Sobel filter, and outputs the absolute value and angle of the edge intensity with respect to the detection target pixel using the pixel values of the edge information detection target pixel and the surrounding two-dimensional pixels, In this embodiment, the edge intensity E (k) for the detection target pixel k is output from the edge extraction filter 11 as the absolute value. On the other hand, the filtered image data F (k) is output from the spatial filter 13. In this filtering process, a low-pass filter is used for noise removal and the like.

  The edge strength E (k) output by the edge extraction filter 11 is given to the nonlinear function processing unit 12. The non-linear function processing unit 12 uses input image data X (k) output from the line memory 10 and a spatial filter using a non-linear function shape determination constant set in a register (not shown) by, for example, a system operator. 13 outputs a parameter Y (k) for addition to the filtered image data F (k) output from 13, that is, weighted addition in the blending process, and an image blending processing unit corresponding to this parameter 14, the input image data and the filtered image data are weighted and added, and output image data P (k) is output.

  With such a configuration, the extraction of edge information from the input image data, the output of the parameter Y (k), and the spatial filtering process can be executed in parallel. As a blend process performed by the image blend processing unit 14 With respect to the image edge component storage processing, Y (k) and F (k) can be synchronized in pixel units and data can be output.

  FIG. 3 is a basic flowchart of edge storage processing in the present embodiment. When the process is started in the figure, first, image data is input in step S1, and the input data is given to the edge information generation process in step S2 and the spatial filtering process in step S4. In step S3 following step S2, the parameter Y (k) is generated as a nonlinear function process, and is given to the blend process in step S5 together with the result of the spatial filtering process in step S4 and the input image. The image and the input original image are weighted and added using a weighting coefficient corresponding to the parameter Y (k), and the process ends.

  FIG. 4 is an example of a nonlinear function used by the nonlinear function processing unit 12 of FIG. In FIG. 2, the edge intensity E (k) output from the edge extraction filter 11 has a value in the range of “0” to “255”, and the value of Y (k) as the output of the nonlinear function processing unit 12. Is in the range of “0” to “256”.

In FIG. 4, the nonlinear function is defined as having a maximum of four sections. The output of the nonlinear function in the first section and the fourth section is fixed to "0" and "256", the second slope in the interval is beta _0/32, the linear slope of the beta _1/32 in the third section It is represented by (line segment).

In FIG. 2, constants that are set in a register (not shown) and determine the function form of the nonlinear function given to the nonlinear function processing unit 12 are first edge strengths E ₀ and 2 that are boundaries between the first section and the second section. Edge strength E ₁ that becomes the boundary between the section and the third section, β ₀ that determines the slope of the straight line in the second section, β ₁ that determines the slope of the straight line in the third section, and the nonlinear function processing unit 12 at the start point of the third section 5 Tsugaaru of _{Y 1} as the output of. Of these, the parameter Y ₁ can be calculated using three constants E ₀ , E ₁ , and β ₀ when the second and third sections are not discontinuous as in FIG. .

FIG. 5 is an explanatory diagram of the slope values of the straight lines corresponding to the values of β ₀ and β ₁ given to the register in order to set the slopes of the straight lines in the second section and the third section. The register to which the value of β ₀ is given and the register to which the value of β ₁ are given are basically independent. For example, if both the values of β ₀ and β ₁ given to these registers are “1”, The value of the slope of the straight line in the second section and the third section is “1/32”. In this way, by allowing constants that determine the function form of the nonlinear function to be freely set from the outside, for example, by an operator via a register, it is possible to appropriately determine the value of the nonlinear function and the corresponding weighting coefficient. it can.

FIG. 6 to FIG. 8 are examples of setting specific nonlinear functions. In FIG. 6, there are four sections in the nonlinear function as in FIG. 4, but the end point of the second section and the start point of the third section are discontinuous, and the nonlinear function processing at the start point of the third section The value of the part output Y ₁ cannot be calculated by the values of E ₀ , E ₁ , and β ₀ . In the figure, E ′ represents an arbitrary value of the edge strength E (k) input to the nonlinear function processing unit 12, and Y ′ represents an output of the nonlinear function processing unit 12 corresponding to the input E ′.

FIG. 7 is an example of a non-linear function when the second interval does not exist. If the second section does not exist is equal to the E ₀ and E ₁ as the value of E (k), the slope of the straight line is given by the slope beta _1/32 in the third section. Figure 8 is an example of a non-linear function is a second section does not exist as in FIG. 7, for the slope of the line beta _1/32 is small in the third section, the edge intensity input E (k) is "255" This is an example of a nonlinear function in which the output Y (k) of the nonlinear function processing unit 12 does not reach “256” even if the value reaches, and as a result there is no fourth section.

The image edge component storage method of the present invention will be described in more detail with reference to FIGS. FIG. 9 is a detailed block diagram of the nonlinear function processing unit 12 of FIG. In the figure, the non-linear function processing unit 12 determines the edge intensity value E _{0 that} is the boundary between the first and second sections of the non-linear function described in FIG. 4 and the edge intensity at the boundary between the second and third sections. the selector 21 register setting value E ₁ is the output Y ₁ of the nonlinear function processing section 12 by the selector 20, the boundary between the second interval and the third interval of 4 input, "0" is input, the The slope register set values β ₀ and β ₁ for determining the slope of the straight line in the second and third sections, the selector 22 to which “0” is input, and the edge strength E ′ output from the edge extraction filter 11 are input. The magnitude comparator 23 that gives selection control signals for the two selectors 20 and 22 is also given the values of E ₁ and E ′ via the flip-flop (FF) 29 and outputs the selection control signal for the selector 21. Big and small The comparator 24, the two's complement conversion unit 25 to which the output of the selector 20 is given, the adder 26 for adding the output of the two's complement conversion unit 25 and the edge strength E ′, and the FF holding the output value of the adder 26 ( Flip-flop) 27 and the output of the FF 28 holding the output of the selector 22 are multiplied by a multiplier 31, and a 5-bit right shift process for performing a process corresponding to an operation of dividing the output of the multiplier 31 by "32" 4, the adder 33 that adds the output of the 5-bit right shift processing unit 32 and the output of the selector 21, and the output Y (k) of the nonlinear function processing unit 12 is saturated with “256” as described in FIG. 4. It is comprised by the saturation process part 34 to be made.

As described with reference to FIG. 4, the output Y (k) of the nonlinear function processing unit 12 is “0” when the value of the edge strength E ′ is “0” to E ₀ . At this time, the selector 21 outputs “0” as a result of the magnitude comparator 24 detecting that the edge strength E ′ is smaller than E ₁ . As a result of detecting that E ′ is smaller than E ₀ by the magnitude comparator 23, the selector 22 outputs “0”, and the value is given to the multiplier 31 via the FF 28. As a result, whatever value the adder 26 outputs, the output of the multiplier 31 becomes “0”, and as a result, the output Y (k) of the nonlinear function processing unit 12 becomes “0”.

Even if the edge strength E ′ becomes larger than E ₀ , the selector 21 continues to output “0”. On the other hand, the selector 22 outputs β ₀ out of the three inputs, and the value is given to the multiplier 31 via the FF 28. On the other hand, since the edge strength E ′ is between E ₀ and E ₁ , the selector 20 gives E ₀ to the 2's complement conversion unit 25 as an output. As a result, the adder 26 subtracts E ₀ from the edge strength E ′, and the result is supplied to the multiplier 31 via the FF 27 and is multiplied by β ₀ as the output of the FF 28. The 5-bit right shift processing unit 32 performs an operation of dividing the output of the multiplier 31 by “32”, and outputs the result as Y (k) through the adder 33 and the saturation processing unit 34. As a result, the output of the nonlinear function processing unit 12 is given by the following equation.

Y (k) = β ₀ {E ′ (k) −E ₀ } / 32
When the edge strength E ′ becomes larger than E ₁ , the selector 20 outputs E _1, and as a result, the output of the adder 26 is a result of subtracting E ₁ from the edge strength E ′. The selector 22 outputs β _1, and β ₁ is multiplied by the output of the adder 26 by the multiplier 31 and is given to the adder 33 via the 5-bit right shift processing unit 32. The selector 21 outputs Y ₁ out of the two inputs under the control of the magnitude comparator 24, and the output of the 5-bit right shift processing unit 32 and Y ₁ are added by the adder 33, via the saturation processing unit 34. Output from the nonlinear function processing unit 12. As a result, when the second section and the third section are continuous as shown in FIG. 4, the output Y (k) is given by the following equation. If this value exceeds “256”, it is clipped at “256”.

Y (k) = β ₁ {E ′ (k) −E ₁ } / 32 + Y ₁ = β ₁ {E ′ (k) −E ₁ } / 32 + β ₀ {E ₁ −E ₀ } / 32
FIG. 10 is a detailed configuration block diagram of the image blend processing unit 14 of FIG. In the figure, an image blend processing unit 14 receives an original image, that is, input image data X (k) and an output of the spatial filter 13, that is, filtered image data F (k) as inputs, and a subsequent stage of the selector 40. 2 FFs 41 and 42, the outputs of two FFs 42 are input, the output of Y (k) of the nonlinear function processing unit 12 is received, the outputs of FF 45 and FF 41, and the output of the twos complement conversion unit 43 Are added to each other, an adder 44 that multiplies the output of the adder 44 and the output of the FF 45, and an 8-bit right shift processing unit 47 that performs a process equivalent to division by dividing the output of the multiplier 46 by “256”. , An adder 49 for adding the output of the 8-bit right shift processing unit 47 and the output of the FF 42 via the FF 48, and saturation processing for the output of the adder 49. And a processing unit 50. Here, the gradation value of the blend image is assumed to be 8 bits from “0” to “255”, and the saturation processing unit 50 sets the value to “255” even if the output of the adder 49 exceeds “255”. "".

  In the present invention, the output Y (k) of the nonlinear function processing unit 12 is used for weighted addition in the addition of the original image (input image data) X (k) and the filtered image data F (k) as the output of the spatial filter 13. In the first case, a process of multiplying the original image X (k) by the weighting coefficient α and multiplying the filtered image data F (k) by (1−α) and adding them will be considered. At this time, it is assumed that the input terminal 1 of the selector 40 is connected to the output terminal 1 and the input terminal 2 is connected to the output terminal 2 according to the selector register setting value (INPUTSSEL). It is assumed that α and Y (k) have the following relationship.

α = Y (k) / 256
The original image X (k) is given to the adder 44 via the selector 40 and FF41. Further, the filtered image data F (k) is supplied to the two's complement conversion unit 43 and the FF 48 via the selector 40 and the FF 42. The adder 44 calculates the difference between X (k) and F (k), and the result is given to the multiplier 46. Y (k) is input to the multiplier 46 via the FF 45 as the output of the nonlinear function processing unit 12, and the output of the multiplier 46 is the difference between X (k) and F (k), This is the product of Y (k).

  The output of the multiplier 46 is subjected to 8-bit right shift processing corresponding to division by “256” by the 8-bit right shift processing unit 47, and the result is output to the adder 49. The adder 49 receives F (k) output from the FF 42 via the FF 48, adds it to the output of the 8-bit right shift processing unit 47, and outputs it as a blend image P (k) via the saturation processing unit 50. . As a result, the blend image P (k) is given by: Note that the symbol “/” in this expression indicates a round-down division.

P (k) = F (k) + [Y (k) {X (k) -F (k)}] / 256
= F (k) + α {X (k) −F (k)}
= ΑX (k) + (1-α) F (k)
Next, as a second case, the filtered image data F (k) is multiplied by the weighting coefficient α corresponding to the output Y (k) of the nonlinear function processing unit 12, and the original image data X (k) has (1- A process of outputting the blend image P (k) as a result of multiplying α) and adding both will be described. At this time, the input terminal 1 of the selector 40 is connected to the output terminal 2, and the input terminal 2 is connected to the output terminal 1. Therefore, the adder 44 outputs the difference between the filtered image data F (k) and the original image data X (k) to the multiplier 46, and Y (k) is output from the adder 44 by the multiplier 46. Multiplication is performed by the 8-bit right shift processing unit 47 to perform a process corresponding to the operation of dividing by “256”, and the original image X (k) output from the FF 48 is added to the output of the 8-bit right shift processing unit 47 to saturate A blend image P (k) is output via the processing unit 50. As a result, the blend image P (k) is expressed by the following equation.

P (k) = X (k) + [Y (k) {F (k) -X (k)}] / 256
= X (k) + α {F (k) −X (k)}
= ΑF (k) + (1-α) X (k)
In the above, two cases have been described for the generation processing of the blend image P (k). However, when the absolute value of the output of the Sobel filter for obtaining the edge vector strength is large and the high-frequency component included in the original image is large. As a blended image, the weight of the original image data X (k) is increased and the weight of the filtered image data F (k) is decreased, so that the image edge component lost by the spatial filtering process can be effectively saved. Expected to be done. The value of α used as the weighting coefficient directly corresponds to the output Y (k) of the nonlinear function processing unit 12, has a large edge strength, and outputs Y (k) of the nonlinear function processing unit 12, that is, When the weighting coefficient α is large, the image edge component is effectively stored by performing the image blending process by applying the first case described above. In this first case, when the edge strength is small and thus the weighting coefficient α is small, the weight of the original image data X (k) is small and the weight of the filtered image F (k) is large, An image blending process is performed on an original image with a small number of high frequency components.

  FIG. 11 is a block diagram showing the configuration of a moving image compression coding processing apparatus using the image edge component storage method of the present invention. In the figure, a moving image compression coding processing device 52 includes a coding unit 55 subsequent to the image processing device 53 described in FIG. 2, and codes the image data P (k) output by the image blend processing unit 14. The data is compressed and encoded by the unit 55 and the compressed image data is output.

In the above, the embodiment has been described using the nonlinear function of FIG. 4 as an example of the nonlinear function for obtaining the output of the nonlinear function processing unit 12 corresponding to the weighting coefficient for weighted addition from the edge strength. The shape is not limited to FIG. 4 and may include a curve and may include any discontinuity. Especially storing very large values as the slope register setting value, for example, by taking a very large value of inclination beta _0/32 of the second section, when the edge intensity E _0, the value of the nonlinear function "0" Of course, it is also possible to use a non-linear function that changes almost discontinuously from “256” to “256”.

1 is a block diagram illustrating a principle configuration of an image processing apparatus according to the present invention. 1 is a basic configuration block diagram of an image processing apparatus of the present invention. It is a basic flowchart of the image edge component preservation | save process of this invention. It is an example of a nonlinear function having four sections. It is explanatory drawing of the inclination register setting value with respect to the nonlinear function of FIG. It is a specific example (part 1) of the nonlinear function. It is a specific example (part 2) of the nonlinear function. It is a specific example (part 3) of the nonlinear function. It is a detailed block diagram of a nonlinear function processing unit. It is a detailed block diagram of an image blend processing unit. It is an example of a structure of a moving image compression encoding processing apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,53 Image processing apparatus 2 Filter means 3 Edge detection means 4 Parameter determination means 5 Image composition means 10 Line memory 11 Edge extraction filter 12 Nonlinear function processing section 13 Spatial filter 14 Image blend processing sections 20 to 22, 40 Selectors 23, 24 Complement conversion units 26, 33, 44, 49 of size comparators 25, 432 Adder 31, 46 Multiplier 32 5-bit right shift processing unit 34, 50 Saturation processing unit 47 8-bit right shift processing unit 52 Video compression code Processing device 55 encoding unit

Claims (5)

An image processing apparatus for storing edge components of an image,
Filter means for performing filtering on the input original image;
Edge detection means for detecting edge component information of the input original image;
Parameter determination means for obtaining, using a nonlinear function, a parameter used for weighting when combining the filtered image as the output of the filter means and the input original image corresponding to the detected edge component information;
An image processing apparatus comprising: an image synthesizing unit that performs weighted addition of the input original image and the filtered image corresponding to the obtained parameter.   The image processing apparatus according to claim 1, wherein the edge detection unit obtains edge component information for each pixel corresponding to pixel data of a detection target pixel and two-dimensional peripheral pixels of the detection target pixel.   2. The image processing apparatus according to claim 1, wherein a constant for determining a function form of the nonlinear function used by the parameter determining means can be set from outside.   The image processing apparatus according to claim 1, wherein the filter unit includes a low-pass filter. An image processing method for storing edge components of an image,
Corresponding to the edge component information detected from the input original image, a parameter used for weighting at the time of synthesizing the filtered image and the input original image as a result of the filtering process on the input original image is obtained using a nonlinear function. Seeking
An image processing method comprising performing weighted addition of the input original image and the filtered image in accordance with the determined parameter.