JPH0324830B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an image processing method, and particularly to an image processing method for correcting an image that is out of focus or an image that is blurred.

Generally, when a signal s(t) is added to a system whose impulse response is h(t), the response output g(t) is g(t)=s(t)*h(t)...(1) Become. Here, * indicates a convolution integral. t
Considering as a one-dimensional variable of the image, the impulse response for an unfocused image is h(t) = δ
(t) and is expressed by a unit shock wave as shown in FIG. 1A. Assuming that this has a certain spread as shown in Figure B due to so-called out-of-focus, h(t) in equation (1) becomes a function indicating this out-of-focus, and is usually a Gaussian function.

If the Fourier transform of this h(t) is H(), it will be as shown in Figure C. In order to restore the original image from the out-of-focus image, the Fourier transform of the impulse response will be 1/H(), that is, as shown in Figure D. It turns out that it is sufficient to pass through a system of characteristics like this. Such processing can be performed in both the time domain and the frequency domain, but is generally performed in the frequency domain because Fourier transform is easier to compute than convolution. Image processing is a two-dimensional expansion of this processing.

This image processing method uses a two-dimensional image as a second
As shown in the figure, this is generally performed by subdividing pixels in the x-axis direction (horizontal direction) and y-axis direction (vertical direction) to create pixels Z _ij . Here, as the simplest example, the image of the point becomes out of focus and the images shown in Figures 3 and 4
A case where the image becomes a two-dimensional Gaussian function image as shown in Figures A and B will be explained. In other words, suppose that the image of the point is an impulse on the Z-axis as shown in Figure 5B, and this can be expressed as ₀ (x, y) = δ (x) · δ (y) ... (2) do. Assuming that this becomes out of focus, (x, y)=e ^-(x2+y2)/ 〓 ² ...(3) (assuming that the x and y components are uncorrelated and linear), Each value in a cross section where both x and y are constant also becomes a Gaussian function from equation (3),
It will look like Figure 4 A and B.

First, y is constant, that is, y=y ₀ , y ₁ , ...,
For the case of y _M-1 , Fourier transform is performed and the correction function 1/
Multiply by H(X)=H ^-1 (X) and perform inverse Fourier transform. Then, the blurred image is converged on _y0 , that is, on the y-Z plane, and the image becomes as shown in FIG. 5A. The same operation is performed this time with x constant, i.e. x = x ₀ , x ₁ , ..., x _N-1
If you follow the path, you will end up with the original image δ
(x)·δ(y)=Z ₀₀ is restored and becomes as shown in FIG. 5B.

In such conventional image processing methods, it takes time for Fourier transform and its inverse transform. For example, if there are 1024 pixels and each transform takes 500 ms, the total time required for correction is at least 500 ms x 1024 x 4. = 2048 seconds, and real-time processing is impossible.

An object of the present invention is to provide an image processing method that enables substantially real-time processing by minimizing the time required for image processing.

In the image processing method according to the present invention, an image signal consisting of first to Mth predetermined direction sweep signals that are continuous in time is subjected to image correction in the predetermined direction via a first image processing filter, and then the first 〜Mth
The signal portions of the corresponding first sections of the sweep signals are sequentially sampled and converted into first orthogonal direction sweep signals orthogonal to the predetermined direction, and The signal portion of each second section of The N-th orthogonal direction sweep signal is converted into a series of temporally continuous signals, which are then passed through a second image processing filter to perform image correction in the orthogonal direction. .

The present invention will be explained below using the drawings.

FIG. 6 is a block diagram showing the principle of the present invention, in which M temporally consecutive (first to Mth) image signals each consisting of a horizontal sweep signal are input to the horizontal synchronization separator 10; The separated horizontal synchronization signal is introduced into the switching controller 11. The image signal (which also contains a horizontal synchronization signal) that has passed through the horizontal synchronization separator 10 is input to the image processing filter 12 having the above-mentioned characteristic H ^-1 (), where correction in the horizontal direction is performed. The corrected image signal is introduced into the signal switching circuit 13, and each horizontal sweep signal is separately outputted to M output terminals of the signal switching circuit 13. For this purpose, the controller 11 controls the signal switching circuit 13 to perform a switching operation in synchronization with the horizontal synchronizing signal.

Each separated output from the switching circuit 13, that is, the first to Mth horizontal sweep signals, is sent to the corresponding delay device 14-
1 to 14-M, respectively, and the corresponding delays are performed. The delay time τ _j of the jth delay device 14-j (j is any number from 1 to M) is τ _j = (M-j)・T+(j-1)・ΔT...(4) It is defined as follows. Here, T indicates one horizontal sweep period, and ΔT indicates T/N. Furthermore, N
is a positive integer.

The output of each delay device is input to a sampling and vertical sweep converter 15, respectively. In this converter 15, sampling of signal portions of N sections of each horizontal sweep signal is performed, and the M horizontal sweep signals are converted into N vertical sweep signals as described later. These N vertical sweep signals are subjected to predetermined delay processing by corresponding delay devices 16-1 to 16-N in order to form a series of continuous sweep signals. The delay time τ i of the i-th (i is any number from 1 to N) delay device 16- _i is selected to be τ _i (i-1)・(M-1)ΔT...(5) has been done.

These respective delayed outputs are combined in an adder 17 to become a series of N continuous vertical sweep signals, and then
Vertical correction processing is performed by the correction filter 18 of H ^-1 (). This corrected output is displayed on the vertical sweep display 19.

FIG. 7 is a waveform diagram showing the operation of the block shown in FIG. 6. In this example, for simplicity, when M=4 and N=5, each of four horizontal sweep signals is Although a case is shown in which M and N are sampled and converted into five vertical sweep signals, it is obvious that M and N can be generalized.

First, first to fourth consecutive horizontal sweep signals as shown in FIG. 7A are applied as image input signals, and correction in the horizontal direction is performed in real time in the filter 12. After that, first, the first horizontal sweep signal 1 is sent to the delay device 14-1 and becomes (M-1)T.
It is output after being delayed by +0・ΔT=3T. The waveform of the first sweep signal 1 delayed by exactly 3T is shown in Figure B, and the second, third and fourth sweep signals 2, 3 and 4 are generated by each delay device using this waveform as a reference. The output waveforms are shown in Figures C, D and E, respectively. Since the second sweep signal is delayed by 2T+ΔT, it is delayed by ΔT with respect to the first sweep signal 1, and similarly, each of the third and fourth sweep signals 3 and 4 is delayed by the first sweep signal 1. 2ΔT and 3ΔT for
will be output with a delay.

Each sweep signal is applied to the sampling circuit 15 at the next stage while maintaining this time delay relationship. This sampling circuit sequentially samples the signal portion of each interval 1 of the first to fourth sweep signals every sampling period of ΔT. This series of sampling outputs is continuously delivered to the output terminal 1 of the sampling circuit 15 for a period of 4·ΔT. Then, when the time ΔT has elapsed after this series of sampling operations was started, the second sampling portion of the first sweep signal is derived from the output of the delay device 14-1,
Using this as a reference, the second sampling portions of the second, third, and fourth sweep signals are derived with delays of ΔT, 2ΔT, and 3ΔT, respectively. Therefore, the sampling operation of the second sampling portion of each sweep signal is performed successively ΔT after the start of the series of sampling operations of the first sampling portion, and this second series of sampling outputs is 4·ΔT. is derived from the output terminal 2 of the sampling circuit 15 during a period of .DELTA.T with respect to the previous first series of sampling outputs.

Similarly, each of the third, fourth, and fifth series of sampling outputs has a period of 4·ΔT and the previous first
The output terminals 3 and 3 of the circuit 15 are delayed by 2ΔT, 3ΔT, and 4ΔT, respectively, with respect to the series of sampling outputs of the circuit 15.
4 and 5. Therefore, each series of sampling outputs overlaps each other in time by ΔT and is outputted from each output terminal of the circuit 15. Therefore, in order to make these into five temporally continuous vertical sweep signals, , delay devices 16-1 to 1
It will pass 6-5.

That is, with respect to the first series of sampling outputs, that is, the first vertical sweep signal, the second vertical sweep signal overlaps with the first vertical sweep signal by 3·ΔT, so the second vertical sweep signal is It is delayed by 3·ΔT in the delay device 16-2. Similarly, each of the third, fourth, and fifth vertical sweep signals is 6ΔT,
It is delayed by 9ΔT and 12ΔT by each delay device 16-3, 16-4, and 16-5. In the end, the signal is converted into five temporally continuous vertical sweep signals as schematically shown in Figure F.

In other words, M horizontal sweep signals with a period T as shown in FIG. 8A are converted into N vertical sweep signals with a period M/NT as shown in FIG. The two-dimensional correction is completed by vertical correction by the image processing filter 18 having the characteristic H ^-1 ( ). The time required to correct one frame of the image signal is M/NT x N = MT, making almost real-time image processing possible.

FIG. 9 shows an embodiment of real-time image processing, in which M consecutive horizontal sweep signals passed through a horizontal image processing filter 12 are input to a delay device 20 whose delay time is externally controlled. . In this case, M
Delay time τ _j of the i-th signal among the main sweep signals
is controlled by the controller 21 so that τ _j (j−1)ΔT (6).

The output of the delay device 20 is input to the signal switching circuit 22, and the output terminal 1,
2..., (j),..., (N+1). Each output terminal of this switching circuit 22 is connected to a recording head of a recordable and reproducible video disk 23. Recording heads 24-1 to 24-N+1 are N+1
They are arranged at equal intervals on one circumference of the recording disk. The i-th head 24-i has the i-th output terminal i of the circuit 22.
is connected. It is assumed that the video disk 23 rotates at a constant angular velocity ω.

In this configuration, the operation will be explained using the charts of FIGS. 10 and 11. As shown in FIG. 10, each of the M sweep signals that have been horizontally corrected in the correction filter 12 is expressed as ΔT=
Consider dividing into N sections each having a length of T/N. First, the first horizontal sweep signal is sent to the delay device 20.
Since the signal 1-1 of the first section is directly applied to the switching circuit 22 without being delayed, the signal 1-1 of the first section is directly led out to the first output terminal 1 of the switching circuit 22, and this is applied to the first recording head 24-1. will be introduced to ΔT
After a period of , the switching circuit 22 switches its input to the second
Therefore, during the next ΔT, the second interval signal of the first sweep signal is supplied to the second head 24-2. Therefore,
At the beginning of the recording track section _D1 recorded by the first recording head 24-1, the first section signal 1-1 of the first sweep signal is also recorded by the second recording head 24-2. recording track section
The second section signals 1-2 of the first sweep signal are respectively recorded at the beginning of _D2 . Similarly, the first
The i-th section signal 1-i of the sweep signal is recorded at the beginning of the recording track section D _i by the i-th recording head 24-i. This state is shown in FIG.

The second sweep signal is transmitted in the delay device 20 as τ ₂ =
The signal is delayed by ΔT and input to the switching circuit 22.
Since the disk 26 has rotated by 1/M at the time when the Nth recording of the first sweep signal is completed, the leading position of the recording track section _D1 is at the N+1st position.
It will face the head 24-N+1 of. Therefore, the first section signal 2-1 of the second sweep signal is applied to the N+1 head 24-N+1 and is recorded following the signal 1-1 of the recording track section _D1 . Similarly, the i-th section signal ji of the j-th sweep signal is recorded in the j-th portion of the recording track section D _i , resulting in a recording track as shown in FIG. 11. That is, each corresponding interval signal of the M horizontal sweep signals is sequentially sampled and continuously recorded in each portion D ₁ to D _N on one circumference of the recording track.

Therefore, as shown in FIG. 12, if the video disk 23 is played back by the playback head 25 while rotating at the same angular velocity ω, then the video disk
A vertical sweep signal is obtained and applied to the vertical sync separator 26 and the image processing filter 18 to perform vertical correction.

In this method, image conversion processing for one field is performed with one revolution of the recording disk, so in order to perform signal processing for one frame, two of this system (or two recording tracks) are prepared, and the recording If playback is performed alternately, real-time image processing becomes possible.

Next, let's consider the image processing filter.

Although an image is a two-dimensional function, we will discuss one-dimensional functions for simplicity, but it is easy to extend the function to two-dimensional functions. Now, the one-dimensional blur function (below
PSF) is h(t) in the time domain and H() in the frequency domain, then the Fourier transform of the impulse response of the correction filter for image restoration is as described above.
H ^-1 (). For example, for Gaussian distribution blur
The PSF is as follows: h(t)=e ^- 〓 ^t2 H()=e ^- 〓 ² H ^-1 ()=e〓 ² ...(7). Further, the PSF of a constant velocity blur is ht=rec(t) H=sinπ/π H ^-1 ()=π/sinπ (8). Note that rect(t) indicates a rectangular wave signal of unit time length. It is necessary to realize a filter with the characteristics of H ^-1 () in equations (7) and (8), but this requires a circuit with infinite gain due to the existence of a singularity (pole). It is becoming difficult to realize this.

Therefore, consider a filter having a feedback circuit configuration as shown in FIG. 27 is a filter having a transfer function G(), and 28 is a feedback circuit having a feedback rate P(). Original image is s(t) (Fourier transform is S(), PSF is h(t) (Fourier transform is H
(), the image degraded due to out-of-focus etc. becomes s(t)*h(t)(S()・H(),
When this is input to the circuit shown in Figure 13, the output is H()・G()/1−P()・G()・S(
)...(9) becomes. To restore the original image, H()・G()/1−P()・G()=1…
…(10) It is sufficient if the following relationship holds true. For that purpose, G()・1/H()+P()...(11). As mentioned above, 1/H() has a singularity, but as can be seen from equation (11), by providing an appropriate P() of the feedback circuit 28, the singularity can be eliminated. G() becomes realizable.

Here, H() in equation (11) is the input signal S
Since it should be determined by ()·H(), G() in equation (11) must be similarly determined according to the input signal. Therefore, in order to set G() to an optimal value, control using an autocorrelation function will be described below. Here, the description will be made on the assumption that an autocorrelation function within a certain divided time period is approximately determined and G() is controlled using this function.

For this purpose, as shown in FIG.
(t)*h(t) is sampled at times t ₁ to _{t 5} to obtain s ₁ to s ₅ , and this sampling value s ₁
~s ₅ is used for approximate calculation of the autocorrelation function. still,
The original image signal s(t) is band-limited, and the sampling interval t _s (t ₁ to _{t 2} , t ₂ to t ₃ , t ₃ to t ₄ , t ₄ to _{t 5}
It is assumed that each time interval) satisfies the Nyquist condition, and that the influence of blur or blurring is sufficiently present in the periods t ₁ to _{t 5} .

The autocorrelation function _ss (τ) of the original image signal s(t) is _ss (τ)=∫ ^∞ _-∞ s(t)・s(t+τ)dt……(
12). This equation (12) can be expressed as the above five sample values.
It can be approximated using s ₁ to s ₅ , and the approximate value is ₀ = s ² ₁ + s ² ₂ + s ² ₃ + s ² ₄ + s ² ₅ (τ = 0) ₁ = s ₁ s ₂ + s ₂ s ₃ +s ₃ s ₄ +s ₄ s ₅ (τ=t _s ) ₂ = s ₁ s ₃ +s ₂ s ₄ +s ₃ s ₅ (τ=2t _s ) ₃ = s ₁ s ₄ +s ₂ s ₅ (τ=3t _s ) ₅ = s ₁ s ₅ (τ = 4t _s ) ...(13).

Now, the input image signal s(t)*h that has undergone degradation
Letting the autocorrelation function of (t) be ′ _ss (τ), ′ _ss (τ)={s(-τ)*h(-τ)}*{s(
t)
*h(t)} = {s(-τ)*s(τ)}*{h(-τ)*h
(τ)} = _ss (τ) * _hh (τ) ...(14), and the autocorrelation function _ss of the original image signal s(t)
(τ) and the autocorrelation function of the impulse response of the PSF
It can be seen that it is a convolution integral with _hh (τ).
In general, when the PSF is not δ(t), that is, it is not an ideal transmission system, but has a low-pass or high-pass element, ' _ss (τ) has a form that is incomparable and spread out compared to _ss (τ). _{The hh} (τ) for such low-pass and high-pass cases is shown below. Now, _hh (τ) = h (-τ) * h (τ) ... (15), and its Fourier transform is H ^* ()・H () = | H () | ² ... (16) ^Since _it ^{is expressed} _as ^{_ _ 2} ...(18), and its Fourier transform is |e ^- 〓 ² | ₂ = e ^-2 〓 ² ...(19). Also, the inverse transformation of equation (19) is becomes. If the Fourier transform of the high-pass PSF is H()=1−Ae ^- 〓 ² ...(21), then the Fourier transform of ′ _hh (τ) is |1−Ae ^- 〓 ² | ² = 1−2Ae ^- 〓 ²・A ²・e ^-2 〓 ²
...(22), and its inverse transformation is becomes.

Therefore, to be precise, it is necessary to calculate equation (14) using equations (20) and (23). however,
For simplification, if s(t) is an impulse δ(t-t ₃ ) at a certain time t=t ₃ , equation (14) becomes ′ _ss (τ)= _ss (τ) * _hh (τ) = δ (τ) * _hh (τ) =' _hh (τ) ...(24). As a result, equations (20) and (23) (A≪
1) at | ₁ / ₀ |, | ₂ / ₀ |, …,
The ratios | _5/0 | are all larger than the corresponding ratios in the ideal transmission system where h(t)=δ ₍ t).

From the above, to find the constant of the filter that restores the original image s(t), change the constant G() of the filter shown in Figure 13 and obtain | ₁ / ₀ |, | ₂ /
The constant should be determined so that ₀ |, ..., | ₅ / ₀ | is minimized. FIG. 15 shows an example of performing image processing in real time while finding the optimum value of the filter constant using the autocorrelation function.

In the figure, in addition to a correction filter consisting of a filter 27 having a characteristic of G() shown in FIG. 13 and a feedback circuit 28 having a characteristic of P(),
Another filter 29 having characteristics of G(a) and P(a) and a feedback circuit 30 are provided, and a sample & memory circuit 32 is provided at the circuit input,
It is designed to sample and store signals as shown in the figure. The sampling value of this signal is read out and controlled by a CPU (Central Processing Unit) 31, and the readout rate at this time of reading is a times the sampling rate, and is input to separately added circuits 29 and 30. In this case, G(a) of the circuit 29 is controlled by the CPU 31, and the sampling values s ₁ to s ₅ are introduced into the additional circuit, the output thereof is input to the CPU 31, and the approximate calculation shown in equation (13) is performed. I'm starting to do it. By this operation, | ₁ / ₀ |, | ₂ / ₀ |, ..., | ₅ / ₀ |
G(a) is controlled so that G(a) is minimized. In this case, since H() generally has a low-pass characteristic, G(a) is made a high-pass characteristic and its constant is changed. The time compression coefficient a is set so that calculations can be performed as many times as necessary within the sampling interval _ts .
If you select , real-time processing becomes possible.

When the optimality of G(a) is determined by the calculation by the CPU 31 and the control of G(a), the switch 33
is closed, and G() of the filter 27 is set to its optimum value.

By doing this, s ₃ at a certain time t=t ₃
The image around δ is the autocorrelation function as much as possible
(τ), and _ss (τ) is processed and restored so that it has a peak as much as possible at τ=0 and is output. During the next t _s , image processing is similarly performed using five sampling values near time t ₄ (s ₄ ), and correction around s ₄ is performed. In addition, since P() and P(a) have a bias role on the frequency axis, P()=1, p
It is preferable to select a characteristic close to (t)=δ(t).

By doing this, the characteristics of the correction filter are controlled depending on the partial blur or blur of the image, so that appropriate image correction can be performed in real time. Therefore, if the filter shown in FIG. 15 is used for each of the filters 12 and 18 shown in FIGS. 6, 9, and 12, good characteristics can be obtained.

[Brief explanation of drawings]

Figures 1 to 5 are diagrams explaining the conventional image processing method, Figure 6 is a block diagram explaining the principle of the present invention, and Figures 7 and 8 are diagrams explaining the operation of the blocks in Figure 6. 9 and 12 are block diagrams of one embodiment of the present invention, FIGS. 10 and 11 are diagrams explaining the operation of the blocks in FIGS. 9 and 12, and FIG. 13 is an image correction Principle diagram of filter, 1st
FIG. 4 is a diagram explaining the operating principle of the filter shown in FIG. 13, and FIG. 15 is a block diagram showing an example of the image correction filter. Explanation of symbols of main parts, 12, 18... Image processing filter, 13, 22... Signal switching circuit, 1
4, 16, 20...delay device, 15...sampling & vertical sweep converter, 23...recording disk, 2
4... Recording head.

Claims (1)

[Scope of Claims] 1. An image processing method in which an image signal consisting of temporally continuous first to Mth predetermined direction sweep signals is subjected to image correction through a predetermined image processing filter,
The image signal is subjected to image correction in the predetermined direction through a first image processing filter, and then signal portions of mutually corresponding first sections of the first to M-th sweep signals are sequentially sampled. Converting to a first orthogonal direction sweep signal orthogonal to the predetermined direction, and then sequentially sampling signal portions of mutually corresponding second sections of the first to M-th sweep signals to perform a second orthogonal direction sweep. This sampling conversion process is performed sequentially to obtain the first to Nth signals.
orthogonal direction sweep signals are obtained, the first to Nth orthogonal direction sweep signals are converted into a temporally continuous series of signals, and the signals are subjected to image correction in the orthogonal direction via a second image processing filter. The method I used to do this. 2 The j-th (j is any number from 1 to M) signal of the predetermined direction sweep signal that has passed through the first image processing filter is converted into (M-j)・T+(j-1)・ΔT
(T is one sweep time in the predetermined direction, ΔT indicates T/N), and the i-th (i is 1 to 1) of the first to N-th orthogonal sweep signals subjected to the sampling conversion processing Claim 1, characterized in that the signals (all numbers N) are delayed by (i-1).(M-1).ΔT to form a series of temporally continuous sweep signals. method.