JP2692853B2 - Image blur detector - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to an image blur detection device that detects the amount of image blur from an image signal in real time.

(Prior Art) In optical imaging devices such as cameras, video cameras, and electronic cameras, regardless of whether they are industrial measurement equipment or consumer equipment, they make it difficult to see camera shake images and cause various malfunctions, especially during walking. Since shooting in a place with a lot of vibration, such as shooting from a moving vehicle or moving vehicle, is likely to cause screen shake, various methods for correcting screen shake have been proposed conventionally.

An example of such a screen blur prevention device is disclosed in
An anti-vibration camera such as 61-248681 can be mentioned. According to this type of camera configuration, an incident image is converted into an electric signal by an image pickup system including a lens system and a photoelectric conversion element, and a television image signal output after being subjected to predetermined signal processing by a signal processing circuit is output to a monitor. In addition to the supply, it supplies the image blur detection circuit, and detects the magnitude and direction of the image blur by correlating the two screens at regular time intervals from the image signal, and based on this detection output the drive control circuit and the motor. It operates and controls the operation of the lens system in the direction of canceling the image blur so that a stable image can be obtained even if the apparatus vibrates.

However, such an image blur detection device cannot distinguish between the local movement of the subject and the shaking of the entire screen, and in order to enable this, it is possible to detect the amount of image blur depending on the area within the screen. It is necessary to give sensitivity a distribution.

As an image blur detection device proposed in consideration of this problem, the Technical Report of the Television Society Vo1.11, No.3 p43 ~
48.PPOE'87-12 (May, 1987) As for "Screen shake correction device", the whole screen is divided into 140 block areas, and the blur detection in each area is turned ON arbitrarily.
It has been proposed that the blurring is detected only in the ON / OFF and ON regions by the representative point matching method.

(Problems to be Solved by the Invention) However, according to the above-described image blur detection device, it is necessary to temporarily store the reference image in the frame memory while maintaining the grayscale multi-value. In addition to having the disadvantage that it requires a large memory, the screens are overlapped with a certain amount of vector shift, and the vector with the highest matching is obtained, so the amount of calculation is large, the circuit scale and device become large, and the calculation time also takes time. It has the drawback of

When the image blur detection device is incorporated in, for example, a small video camera, it must be capable of real-time processing and can be realized with a small circuit configuration. However, since it requires various circuits (A / D converter, frame memory, arithmetic circuit, etc.) and takes a lot of processing time, it was actually extremely difficult to incorporate it into the camera.

(Means for Solving Problems) The present application was made for the purpose of solving the problems described above, and according to the invention described in claim 1,
A first detection method of detecting the generation timing of a predetermined image in the scanning direction of the screen, and a second detection method of detecting a deviation of the generation timing of the predetermined image in the scanning direction between a plurality of temporally different screens. Means, an arithmetic means for detecting the amount of movement of the image between the plurality of screens from the output of the generation second detection means, and a sensitivity for detecting the timing shift detectable by the second detection means. An image shift detection device including a variable control unit is featured.

Further, according to the invention of claim 2 of the present application, a scanning means for scanning a plurality of screens at predetermined time intervals at the same speed,
A clock pulse generating means for generating a clock pulse of a predetermined frequency, and a detecting means for detecting a time difference between timings of generating a predetermined image in the screen by counting the clock pulse in each scan by the scanning means. An image including: a calculation unit that calculates the output of the detection unit to detect the amount of movement of the image; and a sensitivity control unit that changes the detection sensitivity of the detection unit by changing the frequency of the clock pulse. It features a shake detection device.

(Embodiment) An embodiment of the image blur detection device of the present invention will be described in detail below with reference to the screens.

FIG. 1 is a block diagram showing a case where the image blur detection device according to the present invention is applied to a video camera. 100 is a subject, 1 is a variable apex prism, 2 is a photographic lens, and 3 is a photographic lens 2 on the image plane. An image sensor such as a two-dimensional CCD for converting the formed image into an electric signal, and 4 performs gamma correction, blanking processing, addition of a synchronization signal, etc. on the image signal output from the image sensor 3. Is a signal processing circuit that converts and outputs, for example, an NTSC television signal, where OUT is a video output terminal, Y is a luminance signal,
H.SYNC is a horizontal sync signal and V.SYNC is a vertical sync signal.
A feature point extraction circuit 5 detects feature points of the subject image from the luminance signal output from the signal processing circuit 4, and is, for example, a binarization circuit that outputs a binarized signal of an edge portion in the luminance signal.

Reference numeral 6 is a delay circuit for delaying the binarized edge signal output from the feature point extraction circuit for a predetermined time. Reference numerals 7 and 8 are shakes on the image pickup screen based on commands S ₁ and S ₁ from a control microcomputer 28 described later. Area designation circuit that designates the detection area,
Reference numeral 9 is a gate pulse generation circuit that generates a gate pulse while the output of the signal corresponding to the area designated by the area designating circuits 7 and 8 is shifted, that is, only during the period when the characteristic points are shifted. 10 is a crystal oscillator, 11 and 12 are frequency dividers for dividing the output of the crystal oscillator 10, 13 is at least two types of frequency division ratios of the frequency divider 12 according to a control signal S ₂ from a control microcomputer 28 described later. The division ratio setting device that can be switched as above, 14 and 15 are AND circuits, 16, 17 and 18 are pulse counters, 19 is a subtractor that takes the difference between the outputs of the pulse counters 16 and 17, and 20 is the output of the subtractor 19. It is a divider that divides by the output of the counter 18, and the output of the divider 20 is output as the shake amount of the one-way component. Reference numeral 21 is a prism driving mechanism including an actuator such as a piezoelectric element for driving a variable apex angle prism to change its apex angle, and 22 is based on the blur amount output from the divider 20 and cancels the blur amount. The drive circuit drives the actuator 21 in the direction.

On the other hand, 24 to 27 show blocks that detect a position on the screen where there is local movement on the screen, which prevents false detection of local movement on the screen as blurring of the entire screen. To do. As shown in FIG. 5, reference numeral 24 designates a region designating circuit for generating a gate signal for dividing the screen into m × n blocks, and 25 designates m × n blocks divided by the region designating circuit 24, respectively. , A binarization circuit for binarizing the luminance level in the video signal corresponding to the predetermined threshold level, and 26 is a binarization circuit 25
A delay circuit for delaying an image signal corresponding to one screen, which is binarized for each of the n areas by one field,
27 compares the screen of the current field that has been binarized by the binarization circuit with each region of the screen one field before that output from the delay circuit 26, and displays the region in which the binarized information has changed. Motion area information P representing the position on the screen
Is a comparison circuit for outputting to the control microcomputer 28 described later.

Reference numeral 28 is a control microcomputer for collectively controlling each circuit of the device including the area designating circuits 7, 8, 9 and the frequency division ratio control circuit 13, and area designating is performed by the control signals S ₁ , S ₁ , S ₃ , respectively. The generation of the area setting gate signal in the circuits 7, 8 and 24 is controlled, and the control signal S ₂ is output based on the motion area information P (P ₁ , P ₂ ) supplied from the comparison circuit 27, and the The division ratio of the division ratio setting circuit 13 is controlled.

Further, in the figure, A to K indicate signals on the respective signal lines.

Next, the operation of the image blur detection device of the present invention will be described step by step.

The image incident through the variable apex angle prism 1 and the taking lens 2 is converted into an image signal by the image pickup device 3 and output. The edge signal of the image signal to be subjected to the blur detection is binarized by the feature point detection circuit 5, and the binarized signal is delayed by the delay circuit 6 for a predetermined time and supplied to the area designation circuit 7. At the same time, it is directly supplied to the area designating circuit 8. Here, the delay time of the delay circuit 6 is set to an integral multiple of one feed time, and therefore, the edge information, that is, the feature point, of the same screen at a predetermined time before the pixel currently being scanned is output. There is. In the feature detection signal, that is, the binarized signal, only the signal portion corresponding to the area set on the imaging screen by the area designating circuits 7 and 8 is extracted and supplied to the gate pulse generating circuit 9, respectively.

Here, the blur detection area is set, for example, in the vicinity of the outer periphery of the screen, there is a possibility that the feature points may go out of the screen due to the screen blur, and the two fields may often become uncorrelated. For this reason, it is preferable that the outer peripheral portion of a predetermined range (for example, several%) of the screen is set so as not to detect the screen shake.

That is, in the present embodiment, as shown in FIG. 5, two block areas near the outer periphery of the screen are set to be non-detection areas by setting the blur amount detection clock to 0. The detailed description of FIG. 5 will be described later.

In the gate pulse generation circuit 9, the area designation circuit
The outputs A and B of 7, 8 are input, the correlation between them is examined, and a gate pulse having a pulse width corresponding to the time difference of the pulses representing the same image is generated. This gate pulse generating circuit generates a pulse signal H generated when the image of the current field is blurred in the scanning direction side with respect to the image of the reference field, a pulse signal I generated when the image is blurred in the direction opposite to the scanning direction, and also during scanning. , And a pulse signal E generated each time there is a characteristic point. The longer the gate pulse is, the greater the screen blur is. The gate pulse is ANDed with the clock pulse obtained by dividing the oscillation frequency of the crystal oscillator 10, that is, the logical product of J and K is taken to obtain the pulse pulse. Is output, the length of the gate pulse is measured and the screen shake amount at one feature point is obtained. Further, when a plurality of feature points are present on the entire screen, the image blur of the entire screen is obtained by dividing by the total number of pulse counts during the vertical blanking period.

By the way, the movement of the normal image is complicated, and not all the feature points always move in the same direction depending on the state of the subject and the background. Therefore, in this apparatus, the length of the gate pulse is obtained from the signals of both A and B systems in different fields, and the subtraction circuit 19 takes the difference. The subtraction of the subtraction circuit 19 and the division of the division circuit 20 need only be performed once in one field, and since all can be handled as integers, the calculation time is short.
The circuit configuration is simple and requires only a small scale.

With the above configuration and operation, the information on the image blur amount output from the division circuit 20 is supplied to the prism drive circuit 22,
The prism drive circuit 22 operates the actuator of the prism drive mechanism 21 based on the image blur amount, and drives the variable apex angle prism 1 in a direction to cancel the image blur amount. This makes it possible to correct image blur.

Next, a specific gate pulse generation algorithm for realizing the above-mentioned gate pulse generator 9 will be shown,
This will be described with reference to the timing chart shown.

In the figure (a), between the field 1 and the field 2 which are temporally different, the subject pattern 100 is shown on the right side of the drawing.
Has moved, (b) has moved in the opposite direction,
(C) shows a state in which the area of the image pattern to be binarized is narrowed due to changes in illumination, etc., although it has not moved.

2A to 2G show the signal waveforms of the signal lines in FIG. 1, respectively. A is a binarized signal of a field to be referred to, a binarized signal of an image of a predetermined field before (one field before in the present embodiment) output through the delay circuit 6 and the area designating circuit 7, and B is the current value. Binary signal of field,
C is a clock signal output from the crystal oscillator 10 via the frequency divider 12.

D is a signal representing the logical sum (OR) of the signals A and B, and E is a signal representing the logical product (AND) of the signals A and B.

F is a pulse signal that rises at the leading edge of signal A and falls at the trailing edge of signal B, and G similarly rises at the leading edge of signal B and falls at the trailing edge of signal A. A pulse signal, H is a pulse signal having a logical difference that is the difference between the signal F and the signal G, and the length of this pulse, that is, the pulse width, represents the distance that the subject 100 has moved to the right. I is a pulse signal having a logical difference that is the difference between the signals G and F, and similarly, the length or width of this pulse is the subject 10
0 represents the distance moved to the left.

Further, since the sum of the number of pulses in one scanning line of the pulse signal E indicates the number of characteristic points in one scanning line, in the device of this embodiment, the gate pulse generating circuit 9 causes the signals H, I, and
E is output. Note that the signals F and G may be output instead of the signals H and I. However, the former method requires a smaller maximum count value of the counters 16 and 17, so that the circuit scale is smaller. Is advantageous.

In this way, the gate pulse is generated and the AND circuit 1
By calculating the logical product of the clock pulse C and the signals H and I by 4,15, generating the signals J and K, and counting the number of these pulses, the movement amount of the object, that is, the image blur amount can be measured.

Here, in (C), the movement of the subject is small and the pulse width of the binarized signal becomes narrower with time. t ₅ and
The difference in length of t ₆ represents the information on the movement amount. This information can also be extracted sufficiently because the difference in the pulse count numbers is taken at 136 in the subtraction circuit.

Here, the frequency division ratio of the frequency divider 12 that outputs the clock pulse for counting the length of the gate pulse is changed,
If the clock frequency is increased, the movement of the image on the screen can be detected more accurately, in other words, the detection sensitivity can be increased. If you lower the clock frequency,
It is possible to reduce the detection sensitivity to image blur, and if a part of the image is moving, lower the detection sensitivity at the part where the image is moving, so that the local movement becomes a screen blur. It is possible to prevent inconvenience such as making a judgment and malfunctioning.

In this way, by setting the division ratio of the frequency divider 12 by the frequency division ratio setting device 13 and varying the clock frequency during scanning, it is possible to increase or decrease the detection sensitivity depending on the area of the screen. It is configured to be able to.

To explain using an example, for example, applying this device to a video camera with anti-vibration device, like a scene where a dog shakes its tail,
It is assumed that an image with local movement on the screen is to be captured.

In such a case, the image blur is detected in the scenery other than the moving dog, and the movement of the dog's tail is not the image blur.
It is necessary to make settings that prevent detection as much as possible.

Therefore, according to the present invention, by recognizing the image position of the dog, which is the main subject, by some method and increasing the clock frequency in the other regions, the detection sensitivity is increased, and the clock becomes closer to the dog image. This problem is solved by lowering the frequency and lowering the detection sensitivity. Also, the degree of sensitivity change should be set to gradually change in multiple stages in consideration of the error in automatically recognizing the position of the tail image and the image of the shadow of the tail being in the vicinity. This stabilizes the operation of the anti-vibration camera.

Thus, it is a feature of the detection method of the present invention that the sensitivity can be changed stepwise or continuously depending on the clock frequency.

For example, a means for detecting a video position of a dog on the screen, that is, a position or a region having a partial movement in a subject image on the screen is, as described above, a region designation circuit 24, a binarization circuit 25, a delay circuit 26. It is composed of a comparison circuit 27.

FIG. 5 (a) shows a moving image area on the screen divided into m × n blocks. From the comparison circuit 27, for example, as moving area information in the horizontal direction, the moving area M in the horizontal direction is shown. The left and right position information P ₁ and P ₂ may be supplied.

FIG. 5 (b) is a diagram showing the blur detection sensitivity distribution in the moving region M, the peripheral region, and the further distant region,
The horizontal axis represents the position coordinates in block units corresponding to the horizontal direction of the screen, and the vertical axis represents the shake detection clock frequency.

The brightness level in the image signal output from the signal processing circuit 4 is binarized in each of the m × n blocks set on the screen by the area designating circuit 24, as shown in FIG. The comparison circuit 27 compares the current screen with the screen of the previous field delayed by one field by the delay circuit 26.
By comparing with each other and detecting the position on the screen of the block in which the information has changed, a partial motion position on the screen can be detected and output as motion area information P.

In FIG. 5A, m × n blocks are represented by a
When represented by _{i, j} , for convenience of explanation, the area a _{i, j} on the screen is _displayed.
Supposing that 16 areas (i = 0 to 3, j = 0 to 3) are detected as moving areas M, as shown in FIG. 5B, in the moving area M, , The aforementioned divider
The frequency of the blur detection clock pulse output from 12 is set to 0, and that portion is set as the non-detection area. In the vicinity of the motion area M, the motion of the subject is likely to be affected. Therefore, the detection sensitivity is not increased immediately. In this embodiment, the areas M _{1 and} M ₁ having a width of two blocks around the motion area M are provided. Area M ₂ having a width of two blocks outside the area, and further area M ₂
The clock frequency is increased stepwise as the distance from the moving area M increases to the area M ₃ having a width of two blocks outside the area, and the detection sensitivity of image blur is increased. Assuming that the clock frequency in the region M ₃ where the shake detection sensitivity is maximum is f _c , in the region M ₂ f _c / 2 (sensitivity 1 /
2), f _c / 4 (sensitivity 1/4 area M _1), is set to be 0 in motion area M.

As a result, even if there is a local movement within the screen, this position can be detected, and image blur detection and correction can be performed from the image of the portion in which there is no movement.

In addition, as described above, the area M _{4 for} two blocks from the outermost periphery of the screen is a non-detection area in order to prevent malfunction due to detection error, shadow of the subject, and the like.

In the above embodiment, the detection of the moving area in the horizontal direction has been described, but the detection in the vertical direction can be performed in the same manner.

Further, in the above description, the detection sensitivity is changed every two blocks from the moving area, but it goes without saying that the detection sensitivity may not be set every two blocks, and may be appropriately determined from the number of blocks on the screen.

Furthermore, as described above, the detection sensitivity is not controlled stepwise,
You may make it change continuously.

By the way, in order to realize an image blur detection apparatus capable of coping with any change in the image pattern, it is necessary to be capable of coping with the situation described with reference to FIG. 3 below without causing an error. is there.

In FIG. 3 (a), when the subject pattern existing in the field 1 before a predetermined time has disappeared in the current field, in FIG. 3 (b), the two subject patterns existing in the field 1 overlap in the field 2. FIG. 3 (c) shows the case in which the image blur is so large that one subject pattern approaches the vicinity of another subject pattern. In order to deal with these situations without malfunction, not only the circuit configuration described above, but also means for discriminating the image pattern is required.

With respect to this problem, an error can be prevented by providing a discriminating means for discriminating these situations and prohibiting the data acquisition in the above situations.

As an example of the discrimination method, in FIGS. 1A and 1B, if the number of pulse signals E in the pulse signal D of the logical sum of the signals A and B in FIGS. 1 and 2 is counted, Good. As shown in FIG. 2, when the deviation of the subject pattern is small with respect to the size of the pattern and perfect correlation is obtained, the signal pulse E (signal A) in the signal pulse D (logical sum of the signal A and the signal B) is obtained. And the signal B) are always one. However, regarding FIGS. 3 (a) and 3 (b), the pulse numbers are 0 and 2 or more, respectively. Therefore, the discriminating means counts the number of the pulses E, determines that the feature points of both fields can be correlated only when the pulse E is 1, and determines that the data is valid. Judge as invalid. Further, in the case of FIG. 3C, the size of the pattern is checked by the signal A or B, and the pattern smaller than the maximum value of the blur amount for which the image blur is to be measured is not handled so as to avoid it. be able to.

The values of the constants used to actually correct the image blur are shown below.
The blur correction target is up to 25%, and the actually detected blur amount is corrected to about 1/5 to 1/6.

Further, the above-mentioned gate pulse generation circuit can be configured by a logic circuit such as TTL, has a simple circuit configuration, a small scale, and can be integrated in a small shape. The logic of the gate pulse generating circuit is an example for realizing the system of this embodiment, and is not limited to this.

The apparatus described in the present embodiment detects horizontal image blurring of a screen while one field scanning line scans, and the more the subject pattern (feature points) in the screen is, the more averaged the data. The number of samples increases and the accuracy increases. In addition, since it is possible in principle to detect even one horizontal scanning line, it is also possible to detect blurring in one direction using a one-dimensional CCD (line sensor).

Next, an image blur detection method in the vertical direction will be described. Since the video signal has no vertical scanning line, it cannot be applied as it is in the description of FIG. Therefore, a device for producing a luminance signal in the vertical direction is provided as follows.

FIG. 4 shows a specific example thereof, and the detection direction is a vertical line in the screen as shown in FIG. Then, the position for detecting the vertical image blur is determined from the video signal shown in FIG.
The time t ₁₀ from the horizontal sync pulse to this position is obtained. Based on this value, a pulse is generated t ₁₀ hours after the horizontal synchronization pulse for each horizontal scan. This state is shown in FIG. When the luminance of the video signal (a) between the pulses is taken in, it becomes as shown in FIG. The height of the pulse train, that is, the level means a change in luminance in the vertical direction. Therefore, this height is sampled and held to obtain a waveform as shown in FIG. The detection circuit can be used by extracting feature points from this signal waveform by a method such as binarization as in the horizontal direction as shown in FIG. Further, as a feature point extraction method, it is possible to improve the detection accuracy by taking the zero cross of the secondary differential signal instead of the binarized edge. With the above-mentioned structure, the device that realizes the method of the present invention can be composed of normal TTL, C-MOS, etc., has a relatively small number of gates, is easy to integrate, and has a low-cost structure. Therefore, by incorporating it into a small video camera, the conventional anti-vibration camera can be realized in a small size and at a low cost.

(Effects of the Invention) As described above, according to the image blur detection apparatus of the present invention, the deviation of the feature points in each screen is measured during the scanning of the screens that are temporally different, and the image is detected based on this value. Since the amount of blurring can be detected and the sensitivity for detecting the amount of blurring can be changed, it is possible to distinguish between blurring of the entire screen and a screen that has local movement within the screen without malfunction, and always provide optimum image blur correction. Can be done.

Note that, although the case where the present invention is applied to the anti-vibration camera has been described according to the above-described embodiment, the present invention is not limited to this, and the image capturing device of the industrial image measuring device such as the image recognition device of the industrial robot is used. It can be widely applied to the system.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment in which the image blur detection device of the present invention is applied to an image stabilization camera, and FIGS. 2 and 3 are the configurations of gate pulse generation circuits in the image blur detection circuit of the present invention. , A timing chart for explaining the operation, FIG. 4 is a timing chart for explaining the image blur detection method in the vertical direction, and FIG. 5 is for explaining the motion area detection in the screen and the control of the blur detection sensitivity. FIG. 1 ... Variable apex prism, 3 ... Image sensor 4 ... Signal processing circuit, 5 ... Feature point detection circuit 6 ... Delay circuit, 7,8 ... Area designation unit 9 ... Gate pulse generation circuit 10 ... … Crystal oscillator, 11,12 …… Divider 13 …… Division ratio setting circuit 14,15 …… AND circuit 16,17,18 …… Counter 19 …… Subtraction circuit, 20 …… Division circuit 21 …… Prism Drive mechanism 22 …… Prism drive circuit 24 …… Area designation circuit, 25 …… Binarization circuit 26 …… Delay circuit, 27 …… Comparison circuit 28 …… Control microcomputer

 ─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-55-66179 (JP, A) JP-A-56-23085 (JP, A) JP-A-1-264373 (JP, A)

Claims (2)

(57) [Claims] 1. A first detecting means for detecting a generation timing of a predetermined image in a scanning direction of a screen, and a shift of generation timing of the predetermined image in the scanning direction between a plurality of screens temporally different from each other. Second detecting means for operating, a calculating means for calculating a motion amount between the plurality of screens of the image from an output of the generating second detecting means, and a timing shift detectable by the second detecting means. An image blur detection device comprising: a control unit that changes the sensitivity for detecting the. 2. A scanning means for scanning a plurality of screens at a predetermined time interval at the same speed, a clock pulse generating means for generating a clock pulse having a predetermined frequency, and a screen for each scanning by the scanning means. A detection means for detecting the time difference between the timings at which a predetermined image is generated by counting the clock pulses; a calculation means for calculating the output of the detection means to detect the amount of movement of the image; An image blur detection apparatus comprising: a sensitivity control unit that changes the detection sensitivity of the detection unit by changing the frequency.