JPH05266187A - Image restoration system - Google Patents

Abstract

PURPOSE:To eliminate or reduce linking, generated when a character image, etc., including spatial deterioration is restored, as much as possible. CONSTITUTION:A restoration filter group 26 consists of plural restoration filters 26a which performs deterioration restoration processing according to the position of a filter window. An image input part 28 extracts a reference image signal 102 in a reference range and an edge component detection part 34 shifts the same edge detection window as the filter window in order to detect the values of edge components at respective window positions. A decision part 32 decides the window position where the least edge component is obtained and a window setting part 30 sets the filter window according to the decision result. A selection part 36 selects the restoration filter 26a corresponding to the decided window position. The filter window can be set avoiding the edges, so the linking can be evaded.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image restoration filter for improving an original image by performing a spatial convolution operation on the original image containing spatially spread deterioration such as blurring or blurring.

[0002]

2. Description of the Related Art When an image is picked up by an image pickup device such as a camera or an image scanner, the image is deteriorated due to the shooting state at that time. That is, for example, spatially widespread deterioration occurs due to defocusing due to displacement of the optical system, motion blur due to change in relative position between the imaging device and the subject, and the like. Further, generally, at the time of capturing an image, there is a possibility that noise is mixed, and such noise also deteriorates the image, and the captured image becomes unclear.

Therefore, conventionally, as an image filter for eliminating or reducing such deterioration, an image restoration filter utilizing an inverse operator of the deterioration operator has been known. The principle of this restoration filter will be described below.

F is defined as a vector representing an original image, g is a vector representing a degraded image, f'is a vector representing a restored image, H is a matrix representing a degradation operator, and M is a matrix representing a restoration operator. In this case, when the deterioration process is expressed as g = Hf + n (1), the restoration process is expressed as f '= Mg (2). However, n is a vector representing noise.

Here, various forms of the restoration operator M can be considered depending on how the evaluation function for evaluating the restored image f'is taken. Conventionally, a linear filter such as a Wiener filter or an inverse filter is well known as the restoration filter. Is.

[0006]

However, when a character image or the like is restored by the above-mentioned conventional restoration filter, so-called "linking" occurs in the vicinity of a location where a signal changes abruptly, such as an edge, and an unpleasant striped pattern occurs. There was a problem that occurs.

FIG. 4 shows the mechanism of linking generation by edges. FIGS. 4A and 4B show the relationship between the degraded image and the restoration filter at time t = t1 and t = t2. As shown in the figure, the deteriorated image is one-dimensional data, which is indicated by the image signal 10 for the sake of simplicity.

In the figure, reference numeral 12 denotes the impulse response characteristic of the restoration filter, which shows the inverse action of the impulse deterioration action. In the figure, 14 is a filter window, and only the image signal within the filter window is extracted from the input image signal, the filtering process is executed within that range, and the pixel of interest 16 is restored. Incidentally, conventionally, the filter windows are set symmetrically with respect to the pixel of interest 16.

In (A), if the filter window 14 is not applied to the edge 10a of the image signal 10, linking is unlikely to occur. But after some time
As shown in (B), when the filter window 14 includes the edge 10a, the steep part of the image signal and the steep characteristic with folding in the filter work together, and after restoration with the shift of the filter window. The target pixel value of fluctuates greatly up and down, and as a result, linking such as a fine striped pattern occurs.

By the way, in Japanese Patent Application Laid-Open No. 62-188555, with respect to linking occurring at the peripheral edge (corresponding to the edge) of the screen, the image signal near the peripheral edge is gently leveled down to the peripheral edge to suppress the linking. .. However, the suppression of linking can be applied only to the peripheral edge of the screen, and has almost no effect on the linking that occurs in an indefinite part of the screen such as a character image.

The present invention has been made in view of the above conventional problems, and an object thereof is an image restoration filter capable of eliminating or reducing linking that occurs when a character image or the like including spatial deterioration is restored. To provide.

[0012]

To achieve the above object, the present invention sets a filter window for a pixel of interest,
In an image restoration system that performs deterioration restoration processing of the pixel of interest by passing an image signal in the filter window through a restoration filter, image input means for extracting an image signal within a predetermined reference range centered on the pixel of interest, An edge detection unit that sequentially shifts an edge detection window having the same size as the filter window with respect to the image signal within the reference range to sequentially detect the size of the edge component in each edge detection window, and the detected edge detection unit. The window position determining means for determining the window position of the edge detection window with the smallest edge component, and the filter window is set at the window position determined by the window position determining means, and the image signal in the filter window is set. A filter window setting means for outputting and a plurality of restoration filters provided for the number of window positions that the filter window can take within the reference range and performing deterioration restoration processing of the pixel of interest. It, the restoration filter group having a restoring function of the reconstruction filter has corresponding to each window position,
Filter selecting means for selecting the restoration filter corresponding to the window position determined by the window position determining means.

[0013]

According to the above construction, the filter window is set at the position (window position) where the edge component is the smallest in the image signal within the reference range. Therefore, the linking due to the interaction between the edge portion of the image signal and the filter characteristic is prevented. The occurrence can be avoided as much as possible.

That is, in the present invention, the setting position of the filter window is dynamically changed without fixing the positional relationship between the pixel of interest and the filter window, and the restoration process is executed while avoiding the edge. ..

Therefore, the restoration filters for performing the restoration processing of the pixel of interest by the convolution operation are
It is provided for each position of the filter window, and the content of the convolution operation is determined according to the window position.

Similar to FIG. 4, FIG. 3 shows the relationship between the image signal and the filter characteristic. In the related art, as shown by 18 in the figure, the filter window is fixed to the target pixel, so that the restoration process of the target pixel cannot be performed while avoiding the edge 10a. However, in the present invention, the target pixel is restored. It is possible to set the filter window at the window position where the edge component is the smallest, that is, to shift the filter window to avoid the edge while fixing the.

[0017]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a preferred embodiment of the image restoration system according to the present invention. This image restoration system inputs an image signal containing a spatially spread deterioration such as blur, performs a spatial convolution operation by a restoration filter, and outputs the restored image signal. In addition,
For easy understanding of the invention, an example of one-dimensional restoration processing will be described.

In FIG. 1, the deterioration parameter estimation unit 22
Is to input the image signal 100 and estimate the parameters (elements of the deterioration matrix or coefficients of the deterioration function) of the deterioration operator H in the first expression. Here, various methods for obtaining the deterioration operator H have been conventionally proposed. For example, the parameter of the deterioration operator is estimated by using the maximum likelihood estimation method or the like.

The calculation unit 24 calculates the restoration operator M shown in the above second equation from the deterioration operator H constituted by the estimated parameters. In this case, the restoring operator M is
It is represented by the general inverse matrix H ⁺ of the deterioration operator H, and specifically, for example, the general inverse matrix calculation method of Moore-Penrose can be adopted.

Here, the restoration operator H ⁺ (= M) is given by (2
If the matrix has m + 1) rows and (2n + 1) columns, then H
⁺ Is

[0022]

[Equation 1]

It is expressed as follows. Note that H ⁺ (i) is the i-th row vector in H ⁺ , and in the present embodiment, m is 7 and n is 6, for example.

Then, in the present embodiment, each row vector constitutes each filter 26a constituting the restoration filter group 26.
In the image restoration processing, the filter 26 corresponding to the window position of the filter window is stored in the image restoration processing, as will be described later.
a (row vector) is selected. A detailed description of this will be given later.

The image input unit 28 inputs the image signal 100 and extracts an image signal within a predetermined reference range (hereinafter referred to as a reference image signal 102). Here, as shown in FIG. 2, the reference range 200 is defined symmetrically with respect to the target pixel x ₀ that is the restoration processing target, and in the present embodiment, the reference range 200 is (x ₀ − n−m) to (x ₀ + n +
It is set during m). The size of the reference range 200 is determined by the number of rows and the number of columns of the restoring operator H ⁺ , as is clear from the fact that it is defined by m and n.

The window setting unit 30 sets a filter window within the reference range 200 according to the window position determined by the determination unit 32, which will be described later, and sets the reference image signal 102 to an image signal within the filter window (hereinafter, local image). The signal 104) is extracted and output.

In FIG. 2 above, examples of window positions 202a, 202b of the filter windows set for the reference range 200 are shown.
202c and 202d are shown. The filter window shown in the figure is an example, and the filter window is freely variably set in the reference range 200 with one pixel as a step unit forward or backward with respect to the pixel of interest. However, the size of the filter window is fixed, and in this embodiment, 2
The size is n + 1.

Here, the size of the filter window is the same as the number of columns 2n + 1 of the restoring operator H ⁺ , and the number of window positions that the filter window can take within the reference range is 2m + 1. This number is the filter 26a. Is the same as the number of. That is, the number of rows or the number of columns of the restoration operator H ⁺ is equal to the number of filters 26a,
The size of the filter window and the size of the reference range are determined.

In this embodiment, it is possible to set a filter window that does not include the pixel of interest (202d, 20d in FIG. 2).
2e), for example, even if the target pixel constitutes the edge itself, the deterioration restoration processing of the target pixel can be performed, but if it is desired to reproduce the target pixel more faithfully, the vicinity including the target pixel is included. It suffices to set m as small as possible so that the pixels of (4) are within the filter window as much as possible. In this case, the reference range becomes smaller with the value of m.

In the present invention, in order to prevent linking as much as possible, an edge component detection section 34 and a determination section 32 are provided in order to set the filter window while avoiding edges within the reference range.

The edge component detection section 34 uses the reference image signal 1
For 02, an edge detection window having the same shape as the filter window is set, and the edge component of the edge detection window is detected. In the detection, the edge detection window is shifted by one pixel step from end to end of the reference range, and the edge component is detected at each window position. The detection of the edge component is executed by sequentially performing the difference calculation on the image signal, for example.

The judging section 34 receives the output signal from the edge component detecting section 34 and judges the window position having the smallest edge component. That is, in the restoration process of the pixel of interest, the window position where the linking is least likely to occur is determined. Then, the determination result is output as the determination signal 106.

The determination signal 106 is sent to the window setting unit 30, and the window setting unit 30 sets a filter window at the window position designated by the determination signal 106, and the local region extracted by the filter window. The image signal 104 is output to the selection unit 36.

The selection unit 36 selects any one of the filters 26a in the filter group 26, and the filter 26 corresponding to the window position indicated by the determination signal 106.
a is selected. Therefore, the local image signal 104 is input to the selected filter 26a and subjected to the restoration process, that is, the spatial convolution calculation for the pixel of interest. Note that the positions of the filter windows and the filters 26a have a one-to-one correspondence, and regardless of which set is selected, the deterioration restoration process is performed on the pixel of interest.

The image signal 108 of the pixel of interest that has been restored.
Is output to the outside from any of the filters 26a.
Then, the pixel of interest is sequentially shifted for the image signal 100, and the above processing is repeated.

Next, the restoration filter group 26 will be described in detail.

First, each row vector of the restoration operator H ⁺ and 1
Consider the relationship between two-dimensional image signals. The pixel row centered on the pixel x ₀ extracted by the filter window is g _x0 = (g (x0-n), ..., G (x0), ..., G (x0 + n)) ^T (4) ( However, if () ^T is a transposed value), from the first equation, the restoration process is

[0038]

[Equation 2]

And in this case pixel x ₀
Is the target pixel, the row vector H ⁺ (0) is used to obtain the restored target pixel f ′ (x0), as indicated by the dashed line in the fifth equation.

Now, with respect to the pixel row centered on the pixel x ₁ adjacent to the pixel x ₀ , when the restoration process is considered in the same manner as above,

[0041]

[Equation 3]

Then, if the pixel of interest is x ₁ , the restored f ′ (x1) can be obtained from the sixth equation, but at the same time, the sixth pixel
It is understood that f '(x1-1) can also be obtained and a restored value for the pixel of interest x ₀ can be obtained, as shown by being surrounded by a chain line in the equation.

That is, it is possible to obtain almost the same restoration processing result for the target pixel x ₀ not only by the fifth equation but also by the sixth equation. If this is shown by a general formula, f ′ (x0) = H ⁺ (− i) · g _{(x0 + i)} (where −m ≦ i ≦ m) (7), and the pixel of interest and the filter window position It is understood that if the row vector of the restoration operator H ⁺ is selected in accordance with the positional relationship with, the restoration process can be performed on the target pixel even if the target pixel is not located at the center of the filter window.

Here, a specific example of the filter window setting and the filter selection in this embodiment will be described with reference to FIG.

In FIG. 3, when a filter window centering on the pixel of interest x ₀ is set and the filter H ⁺ (0) corresponding to the window position is selected, the edge 10a is added to the filter window.
Enters and causes linking as described above. However, in this embodiment, the optimum window position can be determined by detecting the edge component, and thus the filter window can be shifted while the pixel of interest is fixed. That is, for example, i
It is possible to set the filter window by retreating the pixel, select the filter H ⁺ (i) having a different characteristic corresponding thereto, and execute the restoration process of the target pixel x ₀ . According to this, since the restoration process can be performed without including the edge or while avoiding the edge as much as possible, the occurrence of linking can be sufficiently suppressed.

In this embodiment, the one-dimensional restoration process has been described, but the same restoration process can be extended to two-dimensional.

[0047]

As described above, according to the present invention,
Since the filter window is set at the position where the edge component is the smallest in the image signal within the reference range and the deterioration restoration process of the target pixel can be performed, it is possible to suppress the occurrence of linking in the image when restoring the image, and the image quality of the restored image. It is possible to improve.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of an image restoration system according to the present invention.

FIG. 2 is an explanatory diagram showing a relationship between a reference range and a filter window.

FIG. 3 is an explanatory diagram showing edge avoidance by shifting a filter window.

FIG. 4 is an explanatory diagram showing a relationship between an image signal and a filter characteristic.

[Explanation of symbols]

 26 Restoration Filter Group 28 Image Input Section 30 Window Setting Section 32 Judgment Section 34 Edge Component Detection Section 36 Selection Section

Claims (1)

[Claims] 1. An image restoration system in which a filter window is set for a pixel of interest, and an image signal in the filter window is passed through a restoration filter to perform deterioration restoration processing of the pixel of interest. Image input means for extracting an image signal within the reference range, with respect to the image signal within the reference range, the edge detection window of the same size as the filter window is sequentially shifted, and the edge component within each edge detection window An edge detection unit that sequentially detects the size, a window position determination unit that determines the window position of the edge detection window where the detected edge component is the least, and the window position determined by the window position determination unit Filter window setting means for setting a filter window and outputting the image signal in the filter window, and the number of window positions that the filter window can take within the reference range are provided. A plurality of restoration filters that perform deterioration restoration processing of a pixel of interest, each restoration filter having a restoration action corresponding to each window position, and a window position determined by the window position determination means. An image restoration system comprising: a filter selection unit that selects the corresponding restoration filter.