JP2003101863A - Image jitter prevention device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image jitter prevention device capable of obtaining stable images without jitter by combining the results of image processing of video signals obtained from a camera and a motor-driven universal head system for controlling the photographing direction of the camera. SOLUTION: An image of an object is picked up by a camera 40 loaded on a universal head 10, and the jitter amount of the image is detected by image- processing the obtained video signal with an image processing processor 63. A CPU 64 calculates a viewing angle from the obtained jitter amount and the focal length information of a lens device 44 and generates a control signal for instructing camera movement equivalent to the viewing angle. The control signal is sent to a universal head controller 12, pan/tilt drive parts 52 and 53 are moved and the image jitter is prevented. Also, processing of correcting the video signal on the basis of the obtained jitter amount and enlarging video images by using an electronic zoom function is added so as to compensate omissions of video information at a screen peripheral part.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image shake prevention device, and more particularly to a technique for preventing shake of an image taken by a camera supported by an electric pan head.

[0002]

2. Description of the Related Art A television camera is supported by a pan / tilt electric pan head, and pan / tilt (camera shooting direction) of the pan head, camera focus and zoom are remotely controlled by a control signal from a pan controller. A pan head system that can be operated is known. The platform is often installed at a high place such as the roof of a building or a steel tower, and the mounted lens has a high magnification (high focal length).

Therefore, recently, there has been a problem that a photographed image is shaken and is difficult to see due to a vibration of a building itself due to wind or the like or a shake due to insufficient strength of a platform.

In order to solve this problem, a lens device incorporating a lens for optical shake correction has been conventionally proposed. (JP-A-10-268373, JP-A-12-155347, etc.)

[0005]

However, in the case of such a lens device, it has been necessary to add a special sway preventing means such as an optical shake correcting lens.

The present invention has been made in view of the above circumstances, and by combining an electric pan head system for controlling the shooting direction of a camera and image processing of a video signal obtained from the camera, a special sway prevention means is provided. It is an object of the present invention to provide an image shake preventing device that can obtain a stable image without shaking without adding a shake.

[0007]

In order to achieve the above object, an image shake preventing apparatus according to the present invention converts a camera which converts an optical image into an electric signal and outputs it as a video signal, and a moving direction of the camera. A platform with an electric drive means to
A shake amount detection means for detecting the shake amount of an image by image processing from at least two screen images captured from the camera, and based on the detected shake amount, the camera is moved in a direction to reduce the shake. As described above, a control means for controlling the platform is provided.

According to the present invention, the amount of shaking of an image is detected by capturing an image of a subject with a camera mounted on an electric platform and subjecting the obtained video signal to image processing. The amount of shake of the image on the screen is converted into a movement command amount of the camera, and the operation of the camera platform is controlled so as to move the camera in a direction to reduce the shake. As a result, a stable image with less shaking can be obtained. In the present invention, since the shooting direction itself is moved by the pan head control, it is not necessary to add a lens for optical shake correction or a special image shake preventing means such as a vibration isolation adapter to the lens device side, It can be applied to cameras.

To detect the amount of shaking, a fixed period (for example,
It is preferable to capture two consecutive images in the field period or frame period. Further, it is preferable that the platform is movable in at least one of the pan and tilt movement directions.

According to another aspect of the present invention, signal correcting means for correcting a video signal based on the shake amount is added so as to improve a bad-looking image due to image shake. . For example, there is a mode in which the read start point of the video signal is changed according to the amount of shake so as to remove a region portion of the video signal in which a normal image cannot be obtained due to image shake.

By performing the correction of the video signal together with the control of the pan head, a better image can be obtained.
It is also preferable that the electronic correction means is provided for enlarging the image corrected by the signal correction means by signal processing. When the effective image range is narrowed by the signal correction means, the appearance can be improved by enlarging the image by the electronic zoom so that the image is displayed up to the edge of the screen.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of an image shake preventing apparatus according to the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is an overall configuration diagram of a remote control platform system to which an image shake preventing apparatus according to an embodiment of the present invention is applied. The remote control platform system shown in FIG. 1 includes a remote platform platform 10 (hereinafter, simply referred to as platform platform 10), a platform controller 12, and an image processing device 14. Pan head 1
Reference numeral 0 denotes a housing 15 that houses a television camera (hereinafter, simply referred to as a camera), and a pan head body 16 that rotates the entire housing 15 to pan and tilt the camera in the housing 15. A transparent protective glass 17 is provided on the front surface of the housing 15, and the camera housed in the housing 15 has the protective glass 17
A subject outside the housing 15 is photographed via the.

The housing 15 is supported by a tilt shaft (not shown) extending from the platform main body 16, and the tilt shaft is rotationally driven by a tilt motor built in the platform base 16. When the tilt motor is driven, the housing 15 rotates via the tilt shaft, and the camera in the housing 15 tilts accordingly. Also, the platform unit 1
6 is supported by a pan shaft 18 fixed on a mounting base (not shown), and a pan motor is provided inside the pan body 16. When the pan motor is driven, the pan head body 1
6 rotates around the pan axis 18 as a rotation axis, and the housing 15
The camera inside pans.

The camera platform controller 12 is a device for controlling the camera platform 10 and the camera, and is connected to the camera platform 10 via a cable (connectable via a communication line such as a dedicated line or a public line). Is also possible). The platform controller 12 is provided with a joystick for instructing the pan / tilt direction, a speed adjusting knob, a preset switch, and various other operating members. When the operator operates the operation member, a control signal based on the operation is transmitted from the camera platform controller 12 to the camera platform 10 and the camera, and pan / tilt of the camera platform 10 and zoom / focus of the camera are controlled by the control signals. To be done.

The image processing device 14 is a signal processing device for performing image processing based on a video signal obtained from a camera. The image processing device 14 functions as a control unit of the camera platform 10 together with the camera platform controller 12, and the result of the image processing is used for controlling the camera platform 10.

FIG. 2 is a block diagram showing a configuration example (first embodiment) of the remote control platform system described above. As shown in the figure, the camera 40 mounted on the camera platform 10 is composed of a camera body 42 and a lens device 44 mounted on the camera body 42. The camera body 42 is equipped with an image pickup device and a required signal processing circuit.
Image (moving image) formed on the image sensor via the 4 optical system
Is output to the outside as a video signal (NTSC video signal in this embodiment).

The lens device 44 is provided with optical members such as a focus lens and a zoom lens that can be driven by a motor, and the focus and zoom of the camera 40 are adjusted by moving the focus lens and the zoom lens. .

The camera platform 10 is provided with a control unit 51 (hereinafter referred to as “lens / camera control unit”) 51 for controlling the lens device 44 and the camera body 42.
The operation of the lens device 44 such as the start and end of the shooting in FIG.
It is controlled based on the control signal given from the.

The pan head 10 has a pan drive section 52 corresponding to a pan motor and a tilt drive section 5 corresponding to a tilt motor.
3 is mounted, and these drive units (52, 53) are controlled by control signals from the platform controller 12. By operating the pan drive unit 52 and the tilt drive unit 53, the camera 40 turns in the pan direction and the tilt direction,
The shooting direction of the camera 40 moves.

The image processing device 14 includes a video decoder 6
1, image memory 62, image processor 63 and CP
It is composed of U64 and the like. A video signal output from the camera body 42 is input to the image processing device 14, and the video signal is output by the video decoder 61 to a luminance signal (Y signal).
And the color difference signal. The luminance signal separated here is converted into a digital signal by an A / D converter (not shown) and stored in the image memory 62. Further, the image processor 63 is given a synchronizing signal of a video signal from the video decoder 61, and a command for writing data to the image memory 62 is issued from the image processor 63 at a required timing based on the synchronizing signal. Given. Although details of the image processing will be described later, the image memory 62 stores image data for a plurality of screens at predetermined time intervals. Since the NTSC video signal uses the interlace system, the image for one frame is
It is composed of images for two fields. In the present specification, “one screen of image data” means image data that can form one screen out of a series of image signals that are continuously acquired, and is equivalent to one frame of image data or one frame of image data. Although any of the image data for one field can be applied, the “image data for one screen” means image data for one field in the present embodiment.

The image processor 63 includes an image memory 6
The image data stored in 2 is read out, image processing is executed, and the result is provided to the CPU 64. The CPU 64 performs various kinds of arithmetic processing based on the result of the image processing by the image processing processor 63, detects the amount of image shake, and generates a control signal necessary for controlling the operation of the platform 10. The generated control signal is transmitted from the CPU 64 to the camera platform controller 12, and the camera platform controller 1
2 controls the pan / tilt operation of the platform 10 according to the received control signal.

Next, the operation of the image shake prevention function by the remote control platform system configured as described above will be described.

FIG. 3 is a flow chart showing the processing procedure relating to the image blur prevention function. As shown in the figure, first,
Image data for two consecutive fields are stored in the image memory 62 (step S110). Image processing is performed on these two image data to calculate the shake amount of the image (step S120). The CPU 64 calculates the amount of angle of view required for image shake correction from the obtained shake amount and the focal length information of the lens device 44 notified from the camera platform controller 12 side (step S130), and moves the camera platform for that angle of view. Generates a control signal for instructing (step S14
0).

The control signal thus obtained is sent to the camera platform controller 12, and the pan drive section 52 and / or the tilt drive section 53 are driven in accordance with the control signal to prevent image shake (step S15).
0). After step S150, the process returns to step S110, and the above-described processing is repeated at a constant cycle.

FIG. 4 is a flow chart showing a detailed procedure of processing for obtaining a shake amount by image processing. Although the shake amount in the pan direction will be described below, the same procedure can be applied to the tilt direction.

As shown in FIG. 4, first, as an initial setting, a variable Shift necessary for calculation is set to 0 (step S210). Next, the first image data (image) for one field is loaded into the image memory 62 (step S
212), and subsequently, the second image data (image) is loaded into the image memory 62 (step S214). Thus, when the image data for two screens is stored in the image memory 62, the image processor 63 compares the images with each other, and the value of each pixel of these image data (hereinafter, pixel value).
Is calculated, and the calculation result is stored in association with the variable A (step S216).

Thereafter, the image processor 63 performs a process of shifting the image to the left by one pixel (step S21).
8). FIG. 5 is a conceptual diagram showing an example of an image, an image, and an image 'obtained by shifting the image to the left. According to FIG. 5, the image and the number of horizontal pixels of the image are 74
If the number of pixels is 0, and the image is shifted to the left to obtain the image ', the pixel column S corresponding to the shift amount is removed from the leftmost pixel column in the original image data. Correspondingly, the same number of pixel rows S from the rightmost pixel row are also included in the comparison target image.
Is removed and this is designated as image '.

After step S218 of FIG. 4, step S218 is performed.
Proceeding to step 220, the image obtained by removing the rightmost one-pixel column of the image 'and the image obtained by shifting the image by one pixel to the left'
Are compared with each other to obtain the difference between the pixel values of these image data, and the calculation result is stored in association with the variable A1.

A obtained in step S216 and step S2
A1 obtained in 20 is compared, and the process branches depending on the comparison result. That is, in step S222, A
In the case of> A1, it indicates that the pixel value difference is reduced by shifting the image to the left.
The process proceeds to step S224 so that the image is further shifted in the same direction (left). In step S224, the value of Shift is updated by adding 1 to the value of variable Shift. Next, an image obtained by removing the pixel rows corresponding to the shift amount indicated by the variable Shift from the rightmost pixel row of the image is newly designated as an image ', while the image is obtained by shifting the image to the left by the number of pixels indicated by the variable Shift. The image is newly designated as image ', the difference between these image data is obtained, and the calculation result is stored in association with the variable A2 (step S226).

Then, the magnitude relationship between A1 and A2 is compared (step S228). A1 in step S220
If a determination of> A2 (determination of YES) is obtained, the process proceeds to step S230 to further perform pixel shift in the same direction.
In step S230, 1 is added to the value of the variable Shift and Sh is added.
Update the ift value. Next, the image obtained by removing the pixel rows corresponding to the shift amount indicated by the variable Shift from the rightmost pixel row of the image is newly designated as an image ', while the image is obtained by shifting the image leftward by the number of pixels indicated by the variable Shift. The image is newly designated as image ', the difference between these image data is obtained, and the calculation result is stored in association with the variable A1 (step S23).
2).

After step S232, step S224 is performed.
Then, the description processing is repeated. If the determination of A1 ≤ A2 is obtained in step S228, it means that the shift amount that minimizes the difference between the pixel values is detected. Therefore, the process proceeds to step S234 and 1 is subtracted from the value of the variable Shift to change the variable. Update the value of Shift.

The value of the variable Shift thus obtained means the shake amount of the image with respect to the image.
A calculation for converting the value of to the angle of view is performed (step S26).
0). For example, under the condition that the size of the image sensor is 2/3 inch, the focal length is 300 mm, the shift value is 20, and the resolution in the horizontal direction is 740 pixels, the angle of view θ is calculated by the following equation (1). .

[0034]

## EQU1 ## θ = tan ^-1 [(4.4 × 20/360) / 300] = 0.47 ° (1) On the other hand, when the determination of A ≦ A1 is obtained in step S222 of FIG. Indicates that the pixel value difference is increased by shifting to the left, so that the process proceeds to step S244 so as to shift the image in the opposite direction (to the right). In step S244, the variable Shift
The value of Shift is updated by subtracting 1 from the value of. Then
The image obtained by removing the pixel rows corresponding to the shift amount indicated by the variable Shift from the leftmost pixel row of the image is newly designated as an image ′, while the image obtained by shifting the image to the right by the number of pixels indicated by the variable Shift is obtained. A new image 'is obtained, the difference between these image data is obtained, and the calculation result is stored in association with the variable A2 (step S246).

Then, the magnitude relationship between A1 and A2 is compared (step S248). A1 in step S248
If> A2 is determined, the process proceeds to step S250 to further shift the pixels in the same direction. Step S25
At 0, the value of Shift is updated by subtracting 1 from the value of variable Shift. Then from the leftmost pixel column of the image, shift the variable
The image obtained by removing the pixel row for the shift amount indicated by is newly defined as image ′, while the image obtained by shifting the image to the right by the number of pixels indicated by the variable Shift is newly defined as image ′,
The difference between these image data is obtained, and the calculation result is set to the variable A1.
Are stored in association with (step S252).

After step S252, step S244
Then, the above process is repeated. If the determination of A1 ≤ A2 is obtained in step S248, it means that the shift amount that minimizes the difference between the pixel values is detected. Therefore, the process proceeds to step S254 and 1 is added to the value of the variable Shift to change the variable. Update the value of Shift. After step S254, the process proceeds to step S260, and the operation of converting the Shift value into the angle of view is performed.

The CPU 64 generates a control signal for driving the pan drive unit 52 by the angle of view obtained in step S260, and outputs this control signal.
Output to.

In the above description, the information of the entire two screens between fields (or between frames) is compared, but the processing speed can be increased by limiting the comparison area to a partial area in the center of the screen.

Next, another embodiment of the present invention will be described.

In the above embodiment to which the NTSC system is applied, images are captured at 1-field intervals (1/60 seconds) or 1-frame intervals (1/30 seconds). Image processing is delayed. Further, since there is a delay in the platform servo system, it is impossible to completely eliminate the image shake by the above-described embodiment (first example). Especially, the sway of high vibration frequency is not sufficiently removed.

In order to solve such a problem, in the second embodiment shown in FIG. 6, the pan head driving control and the video signal correction processing are used together.

FIG. 6 shows another configuration example (second embodiment) of the remote control platform system to which the present invention is applied. 6, those parts which are the same as or similar to those in the block diagram shown in FIG. 2 are designated by the same reference numerals, and a description thereof will be omitted.

The image processor 63 corrects the output video signal according to a command from the CPU 64.
That is, the read start point (read start point) of the video signal is shifted by the amount of shake (Shift amount) calculated. The image data thus corrected is stored in the image memory 6
2 is stored in the video encoder 2, converted into a predetermined video signal by the video encoder 66, and then output to the outside.

FIG. 7 is a conceptual diagram for explaining the shake amount correction according to the second embodiment. Image stabilization that controls pan / tilt operation of the pan head based on the amount of shake detected by image processing is called "hardware", and the output video signal is corrected based on the amount of shake detected by image processing. The measures to be taken are called "software measures".

FIG. 7A shows an image when no measures are taken against the image shake. In this case, a part of the screen becomes an unusable area (shaking area) D due to the vibration in the pan direction. On the other hand, if hardware measures are taken, the result is shown in FIG.
As shown in, the shaking area D is reduced by the effect of the pan head control. Then, by implementing the soft measures in addition to the hard measures, as shown in FIG. 7C, the shaking area can be removed.

FIG. 8 shows a flowchart thereof. In FIG. 8, the same steps as those in FIG. 3 are designated by the same step numbers, and the description thereof will be omitted. After the shake amount is obtained in step S120, the video signal is corrected based on the obtained shake amount in parallel with the process of controlling the platform 10 in steps S130 to S150.

However, if the shake is large, there is a problem that the left and right information (up and down with respect to the shake in the tilt direction) of the video signal is lost.

As a countermeasure against this, in the third embodiment shown in FIG. 9, a process of compensating for the missing area by the electronic zoom function is added. FIG. 9 shows another configuration example (third embodiment) of a remote control platform system to which the present invention is applied.
9, those parts which are the same as or similar to those in the block diagram shown in FIG. 6 are designated by the same reference numerals, and a description thereof will be omitted.

The image processor 63 corrects the output video signal in accordance with the instruction from the CPU 64.
That is, the read start point (read start point) of the video signal is shifted by the amount of shake (Shift amount) calculated. The image data thus corrected is stored in the image memory 6
Stored in 2. The image data stored in the image memory 62 is sent to the electronic zoom processing section 65, where the CPU 64
The enlargement processing is performed in accordance with

Based on the information from the image processor 63, the CPU 64 calculates the zoom magnification necessary for compensating for the video signal missing due to the software measure, and the electronic zoom processing section 65 receives the enlargement magnification information and interpolation calculation. Provide the necessary information to The image data enlarged by the electronic zoom processing unit 65 is converted into a predetermined video signal by the video encoder 66 and then output to the outside.

FIG. 10 is an explanatory diagram of a method of correcting a video signal by the electronic zoom function. FIG. 10A shows how an unusable data area is generated in a part of the screen when the software countermeasure is implemented. The video signal waveform in the horizontal (H) direction at this time is as shown in FIG. The left and right parts corresponding to the unusable data area do not contain effective image information components.

If such a video signal is directly input to an image output device such as a display, both sides of the display screen (reproduction screen) become non-display areas (for example, a display filled with black), and the effective screen becomes narrow. Become. In order to improve such a phenomenon, the video signal is enlarged by the electronic zoom function as shown in FIG. By doing this, the image will be displayed to the edge of the screen.

FIG. 11 shows the flowchart. Figure 1
The same step number is assigned to the process common to FIG. 9 in FIG. 1, and the description thereof is omitted. After the video signal is corrected based on the shake amount in step S132, the process proceeds to step S134, and the electronic zoom process is performed at an appropriate enlargement ratio so that the image is displayed up to the edge of the screen. Thus
It is possible to prevent image shake and improve the appearance of the screen.

In the above description, a video signal is acquired from one camera 40 and the amount of image shake is detected based on the video signal, but the scope of application of the present invention is not limited to this. For example, there is a mode in which a first camera as a means for detecting the amount of image shake and a second camera as an original photographing means are mounted on a common platform 10. By using a high-speed camera with a field frequency of 120 Hz or higher for the first camera, an image can be captured in a cycle shorter than a normal field cycle (1/60 seconds).
It is possible to realize more accurate image shake prevention. In addition, in the case where two cameras are used, when the correction of the video signal and the enlargement correction by the electronic zoom are added, the signal correction and the electronic zoom process are performed on the video signal obtained from the second camera. .

Further, the invention is not limited to the mode in which each of the first to third embodiments described above is applied independently, and a circuit configuration capable of realizing the first to third embodiments is provided in one image processing device 14. It is also preferable that the switching device is equipped with switching means (switching switch or the like) that allows the operator to select these functions as appropriate.

In the above embodiment, the pan head system equipped with the television camera is described, but the scope of application of the present invention is not limited to this, and the same applies to a digital video camera, a digital still camera with a moving image shooting function, and the like. Is applicable to.

[0057]

As described above, according to the image shake prevention apparatus of the present invention, the amount of image shake is detected by performing image processing on a video signal obtained from a camera mounted on an electric platform. Since the driving of the camera platform is controlled so as to move the camera in the direction of reducing the shake, it is possible to obtain a stable image with little shake.

In the present invention, since the pan head is driven according to the amount of shake, it is not necessary to add a special image shake preventing means such as a correction lens or a vibration isolation adapter to the camera body or the lens device side, and various types can be used. It can be applied to other cameras.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of a remote-controlled pan head system to which an image shake prevention device according to an embodiment of the present invention is applied.

FIG. 2 is a configuration example of a remote control platform system (first embodiment).
Block diagram showing

FIG. 3 is a flowchart showing a processing procedure relating to an image shake prevention function in the system configuration shown in FIG.

FIG. 4 is a flowchart showing a detailed procedure of processing for obtaining a shake amount by image processing.

FIG. 5 is an explanatory diagram showing a state in which two images are compared and the images are shifted.

FIG. 6 is a block diagram showing another configuration example (second embodiment) of a remote-controlled pan head system to which the present invention is applied.

FIG. 7 is a conceptual diagram for explaining correction contents in the system configuration shown in FIG.

8 is a flowchart showing a processing procedure relating to an image blur prevention function in the system configuration shown in FIG.

FIG. 9 is a block diagram showing another configuration example (third embodiment) of a remote-controlled pan head system to which the present invention is applied.

FIG. 10 is an explanatory diagram of a method of correcting a video signal with an electronic zoom function.

11 is a flowchart showing a processing procedure relating to an image shake prevention function in the system configuration shown in FIG.

[Explanation of symbols]

10 ... Platform, 12 ... Platform controller, 14 ... Image processing device, 16 ... Platform body, 18 ... Pan axis, 40 ... Camera,
42 ... Camera body, 44 ... Lens device, 51 ... Lens
Camera control unit, 52 ... Pan drive unit, 53 ... Tilt drive unit, 63 ... Image processing processor, 64 ... CPU, 65 ...
Electronic zoom processing unit

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/222 H04N 5/222 B 5/228 5/228 Z

Claims (3)

[Claims] 1. A camera for converting an optical image into an electric signal and outputting it as a video signal, a platform provided with an electric drive means for moving the photographing direction of the camera, and at least two screens taken in from the camera. A shake amount detection means for detecting the shake amount of the image from the image by image processing, and a control means for controlling the platform so as to move the camera in a direction to reduce the shake, based on the detected shake amount. An image shake prevention device comprising: 2. The image shake according to claim 1, further comprising signal correction means for correcting a video signal based on the shake amount so as to improve a poor-looking image due to the image shake. Prevention device. 3. The image blur prevention device according to claim 2, further comprising an electronic zooming unit that enlarges the image corrected by the signal correcting unit by signal processing.