JPH0437268A - Picture processor for video camera - Google Patents

Abstract

PURPOSE:To display a sharp picture without blur even when a video camera is in vibration by obtaining a vibration component in a prescribed direction by a prescribed processing from a picture signal of a conventional video camera and displaying a picture deviated to cancel the component. CONSTITUTION:A picture signal picked up by a video camera 101 is inputted to a picture memory 102, in which the signal is stored tentatively and a vibration component detection means 103 detects the vibration component of the picture stored in the picture memory 102 in a prescribed direction such as a vertical direction. A picture read means 104 deviates a picture in an opposite direction to the vibration by a quantity corresponding to the detected vibration component to read the picture from the picture memory 102 and the read picture signal is displayed on a display means 105. Since the correction is implemented to cancel the vibration component on the picture caused by the vibration, even when the video camera 101 is in vibration, the deviation on the display pattern is reduced effectively to display sharp picture.

Description

[Detailed Description of the Invention] [Field of Application of the Invention] The present invention relates to a processing device for image signals captured by a video camera, and particularly relates to a processing device for processing an image signal captured by a video camera, and in particular, a device for processing an image signal captured by a vibrating video camera such as a video camera mounted on a vehicle or a helicopter. This invention relates to technology for detecting vibration components from image signals.

[Prior art]

If a video camera is installed on a vibrating object, such as a vehicle or helicopter, vibrations will be superimposed on the captured image signal, so if the detected image is displayed as is, the image will be blurred. The screen becomes very difficult to read. Therefore, it is necessary to perform appropriate image processing on image signals captured by a vibrating video camera to remove vibration components.

As a correction method for the vertical vibration component of conventional video cameras,
For example, there is one shown in FIG.

In FIG. 7, light inputted through a lens 1 is reflected by a mirror 2, then imaged onto a CCDa, and outputted as an image signal via an image signal generation section 4. Further, the rotation angle of the mirror 2 can be changed by a motor 5 or the like. In addition, inside the video camera configured as above,
A G sensor 6 is installed. This G sensor 6 detects pitch fluctuations of the video camera (corresponding to vertical vibrations of the screen).

The mirror 2 is controlled by the mirror control unit 7 according to the detection result.
rotate according to the pitch variation. That is, when the G sensor 6 determines that the video camera has moved downward, the mirror 2 is controlled to move clockwise in FIG. 7, and as a result, the output image is a blur-free image. .

The vertical vibration of a video camera as described above has a considerable effect on a video camera mounted on a vehicle, and for example, in the case of a marathon broadcast, the video becomes difficult to see.

Furthermore, in an image processing system for recognizing lanes on a road, there is a problem in that lane images move up and down, making lane detection processing difficult.

One of the solutions for this is the method using the G sensor and mirror described above.

[Problem to be solved by the invention]

However, the conventional method described above uses a mechanically movable mirror and a G sensor.
Durability is a problem because it includes mechanical moving parts. Also,
It is necessary to adjust the proportionality constant to make the output of the G sensor correspond to the movement of the mirror, and there is a problem that the number of adjustment steps increases.

Additionally, when a video camera is installed in front of the vehicle, for example, when an uphill slope appears in front of the vehicle, it is desirable to capture images slightly upward, but before the vehicle reaches the slope, the video camera images in the horizontal direction. , there is a problem in that it is not possible to correspond to the scenery in front of the vehicle.

The present invention aims to solve various problems in the prior art as described above.

[Means to solve the problem]

In order to achieve the above object, the present invention is configured as described in the claims.

FIG. 1 is a block diagram showing the functions of the present invention, (a
) corresponds to the first claim, and (b) corresponds to the third claim.

First, in FIG. 1(a), 101 is a normal video camera. The image memory 102 also includes the video camera 1
The image signal captured in step 01 is input and temporarily stored.

Further, the vibration component detection means 103 detects vibration components in a predetermined direction (for example, the vertical direction) of the image stored in the image memory 102. This vibration component detecting means 103 includes an image memory 102, for example, as described in the second claim.
The image stored in the image is scanned in a predetermined direction, the difference between the address value when an edge existed and the address value of the edge in the previous screen is calculated, and the amount of vibration is calculated by averaging the difference value within one screen. The amount of change (vibration speed) is determined. Further, the one-image reading unit 104 reads the image from the image memory 102 by shifting the image in the direction opposite to the vibration by an amount corresponding to the detected vibration component.

The read image signal is then displayed on the display means 105.

Note that the vibration component detecting means 103 and the image reading means 104 described above correspond to, for example, the high-speed image processing device 12 shown in FIG. 2, which will be described later, and are composed of a computer or the like. Further, the display means 105 corresponds to, for example, the display device 13 in FIG. 2, and is a CRT display device, a liquid crystal display device, or the like.

As described above, in the apparatus according to the first claim, when the video camera vibrates, correction is performed to cancel out the vibration component on the image caused by the vibration, so even if the video camera vibrates, It is possible to effectively reduce blur on the display screen and display clear images.

Next, in FIG. 1(b), the video camera 101 is
For example, it is fixed to a moving object such as a vehicle or a helicopter. The image memory 102 includes the video camera 101
The image signal captured by the camera is input and temporarily stored. Furthermore, the vibration component detection means 106 scans the image stored in the image memory 102 in a predetermined direction, calculates the difference between the address value when an edge was present, and the address value of the edge in the previous screen, and calculates the difference value. The amount averaged within one screen is output as the amount of change in vibration (vibration speed).

As described above, in the device according to the third claim, the pitch component of the vibration of the moving object can be detected from the image captured by the video camera fixed to the moving object, and the pitch component of the vibration can be used to detect the pitch component of the vibration of the moving object. Various types of control such as posture control of a moving object can be performed.

[Embodiments of the invention]

FIG. 2 is a block diagram of one embodiment of the present invention. In FIG. 2, video camera 10 is a normal video camera. An image signal captured by the video camera 10 is input to a high-speed image processing device 12 and temporarily stored in an image memory 11. Then, the high-speed image processing device 12 detects the vibration component in the vertical direction of the screen, shifts the image in the vertical direction so as to remove the vertical vibration component according to the detected amount, and displays the result on the display device 13 such as a CRT. Output.

Processing calculations in the high-speed image processing device 12 will be described below.

FIG. 3 is a main flowchart showing the entire procedure of image processing. In FIG. 3, first, initialization is performed at Pl, and the device starts operating. Next, image input in P2 and preprocessing in P3 are executed simultaneously.

Edge detection is effective in the preprocessing of P3.

For example, if you want to find edges in the horizontal direction, change the input image to A
(x + y), preprocessed image B (x +
y) is B(x,y)= , :A(x-1,y-1)+2A(x
,y-1)+A(x+1.y-1)-A(x-1,y+
1)-2A(x+y+1)-A(x"Ly"l).

The above A (X ry ) and B (x ry ) are stored in separate image memories, and A (x +
y) is used for display, and B(x, y) is used for detecting vibration components.

Next, in P4, a pitch component (vertical vibration component) detection process is used. In P5, the image A (X T y ) is shifted in the vertical direction (y direction) and displayed by the vibration component detected in P4.

Continuous image processing is performed by repeating the above processing.

Each part P1 and P4 of the main flowchart in FIG. 3 will be explained in detail below.

FIG. 4 shows a flowchart for initializing Pl. In initialization, areas Y(1), D(1), and C(i) (where i=1 to P) are all cleared to O.

Note that p represents the number of divisions of the screen in the vertical direction. For example, if the screen size is 256 x 240, appropriate p is 32 to 256.

Next, FIG. 5 is a flowchart of pitch component detection processing. The process shown in FIG. 5 is performed every time a screen is captured.

In FIG. 5, first, 1=1 and the corresponding coordinate X is determined. In addition, x ” Xp-1, Xp=N
/p, and N is the horizontal size of the screen (256, etc.).

Next, the area y(i) is adjusted, and if y(i)=O, it is assumed that there was no edge point on x'' y
, , (= 240, etc.).

Next, one image B (x + y) is stored as Y(i)=y if it is an edge point, and '! if it is not an edge point. ='! −1, and examine B(x, y) up to a predetermined Y mil+.

On the other hand, if there is no edge point on x=Xp-1, Y(i)
=O, C(i)=O(7) Leave as is and move to the next X coordinate with 1=1+1.

If the above pitch component detection process is repeated for several surfaces, the corresponding y coordinates will mainly be stored in Y(i). If Y(i) is not ○, then in the previous calculation (x,
If an edge point exists at point Y(i)), the width Y
Scanning starts from a point 11 below, ie, y=Y(i)+Yti.

Similarly, examine B (x + y) and repeat until y=Y(i)-Yw if it is not an edge point. If the edge point still exists, set Y(i)=O, C(i)=O and proceed to the next j. When an edge point is found, it is assumed to be a pitch variation, and Y(i) = y D(i) = y-Y(i) (variation) C(i) = C(i) + 1 (number of consecutive periods).

On the other hand, if it is larger than the threshold Ty, it is determined that a point completely different from the previous point has been detected, and Y(i)=OC(i)=
Let it be O.

After repeating the above process from 1=1 to P, C(1)
, C(i)>TC", d is obtained by averaging D(i) using the following equation (1).

Note that C(i) indicates how many consecutive edges exist in i. Further, the appropriate Tc is about 5 screens.

However, the above calculation calculates the average value only for i such that C(1) > Tc. Q is the number of i corresponding to it.

The above d can be regarded as the amount of vertical movement of the image in the current calculation with respect to the image in the previous calculation. Therefore, by integrating the value of d for each screen, it is possible to obtain the reference coordinate value Ys as an absolute coordinate.

Basically, if Ys = O, for example, if you move 2 pixels downward across the screen, and then move 1 pixel downward on the next screen, the first time d = 2, the second time d = 1, and the integrated value is 3 pixels.

However, since the error increases due to integration, Ys,=
Ys is updated at C(Y so 1d ). In addition, in the above equation, Ys□ is the current output of Ys, Y2 on the right side
O is the previous result of Ys. Further, α is a numerical value close to 1 and smaller than 1, and is 1, for example, 0.95.

If the above process is repeated for each screen, the value of Ys will represent the vertical shift amount of that screen.

Therefore, in image display, the original image A (x + y
) is shifted by Y5 in the y direction and displayed, a blur-free image can be displayed.

In the above explanation, the case where the image vibrates in the vertical direction is shown, but if the image vibrates in the horizontal direction, it is necessary to scan in the horizontal direction (for example, from left to right) and detect edges in the horizontal direction. Bye.

Next, FIG. 6 is a time chart of the arithmetic processing described above. In FIG. 6, first, image processing and preprocessing are performed in the first field, and each is stored. Next, a pitch component detection process is performed during the vertical retrace period when the first field changes to the second field. Then, in the second field period, the stored image may be shifted by Ys, that is, the y address may be set to y+Ys, and the original image may be read and displayed.

As described above, if the series of processes is executed according to the time schedule shown in FIG. 6, it is possible to obtain a natural continuous image output without noise.

Furthermore, as can be seen from the above-mentioned effects, if the road in front of the vehicle has a slope, for example if there is an uphill slope, the screen will be shifted downwards, i.e. the upper part of the imaged screen will be displayed on the display screen. It is corrected so that it approaches the center of
The result is the same effect as a video camera corrected slightly upwards. Therefore, the screen that is originally desired to be captured can be displayed even before the vehicle actually starts going uphill.

Further, in this embodiment, an example is given of a device that displays an image without blur by shifting the image by Ys, but the present invention can also be applied to other uses. For example, when performing roadside recognition processing using an in-vehicle video camera, the moving image processing may be performed as if the image had been shifted by Ys.

Further, if Ys is obtained from a signal from a video camera fixed to a moving object such as a vehicle or a helicopter, the pitch component of the vertical vibration (or horizontal vibration) of the moving object can be detected. If the pitch angle is θ, then P θ=jan-1+ Ys. In the above equation, F is the focal length of the lens (unit: mm), and Dp is the size of one pixel (unit: mm). However, in this case, it corresponds to the ground pitch angle on the imaged screen.

As described above, by detecting the pitch component of the vertical vibration or horizontal vibration of a moving object, it can be used for various controls such as posture control of the moving object.

〔Effect of the invention〕

As described above, according to the present invention, a vibration component in a predetermined direction is determined from an image signal of a normal video camera through image processing, and an image shifted so as to cancel out the vibration component is displayed. This makes it possible to display clear images without blur even when the video camera vibrates. In addition, all processing is done electronically,
It has excellent durability because it does not include a moving mechanism like conventional models. In addition, since the vibration component is determined on a pixel-by-pixel basis and has a one-to-one correspondence with the shift amount, gain adjustment is not necessary. Furthermore, when mounted on a vehicle, the shift amount is adjusted in advance before entering the slope according to the slope of the road ahead, so an image of the area originally desired to be photographed can be obtained. Furthermore, since the pitch component of the vibration of a moving object can be determined, excellent effects such as application to various types of control can be obtained.

[Brief explanation of the drawing]

FIG. 1 is a functional block diagram of the present invention, FIG. 2 is a block diagram of an embodiment of the present invention, FIG. 3 is an embodiment of the main flowchart of the arithmetic processing in the present invention, and FIG. 4 is a block diagram of an embodiment of the present invention. FIG. 3 is an embodiment of a flowchart showing an initialization part in arithmetic processing. FIG. 5 is an example of a flowchart of the pitch component detection part in the arithmetic processing of the present invention, FIG. 6 is an example of a time chart of the arithmetic processing of the present invention, and FIG. 7 is an example of a conventional device. be. Explanation of codes> 10. Video camera 11... Image memory 12.
- High-speed image processing device 13...Display device 10...Vibration camera 102--Image memory 103--Vibration component detection means 104--Image read-out means 105--Display means 106--Vibration component detection means

Claims (1)

[Claims] 1. A video camera that outputs a captured optical image as an electrical image signal, an image memory that inputs and temporarily stores the image signal, and a predetermined number of images stored in the image memory. a vibration component detection means for detecting a vibration component in a direction; an image reading means for reading an image from the image memory by shifting the image in a direction opposite to the vibration by an amount corresponding to the detected vibration component; An image processing device for a video camera, comprising: display means for displaying an image signal read out by a readout means. 2. The vibration component detection means scans the image stored in the image memory in a predetermined direction, finds the difference between the address value when the edge was present and the address value of the edge in the previous screen, and calculates the difference value. 2. The image processing device for a video camera according to claim 1, wherein an amount averaged within one screen is used as the amount of change in vibration. 3. A video camera that outputs captured optical images as electrical image signals and is fixed to a moving object; an image memory that inputs and temporarily stores the image signals; and images stored in the image memory. is scanned in a predetermined direction to find the difference between the address value when the edge was present and the address value of the edge in the previous screen, and the amount of the difference value averaged within one screen is the amount of change in vibration. An image processing device for a video camera, comprising: vibration component detection means, and detects the amount of change in vibration as a pitch component of vibration of a moving object.