JP2007116308A - Image information playback device - Google Patents

Abstract

An image blur is appropriately controlled by accurately determining the panning and tilting operations performed by a photographer.
An imaging device that captures a moving image, an angular velocity detection unit that detects at least one angular velocity in a Pann direction and a tilt direction attached to the imaging device, and the imaging according to the angular velocity detected by the angular velocity detection unit Image information superimposing density setting means (step S5) for setting a density for adding image information of adjacent image frames out of image frames captured by the apparatus;
Image information reproducing means (steps S6 and S7) for adding and reproducing the image information of adjacent image frames at the density set by the image information superimposing density setting means is provided.
[Selection] Figure 4

Description

The present invention detects image angular velocity in a panning and tilting direction of an imaging device that captures a moving image and suppresses discomfort to a viewer due to abrupt movement in the panning and tilting directions of a reproduced image. The present invention relates to a playback device.

As a conventional image information reproducing device, for example, a degree of change between successive frames of a video file, that is, a degree of change in motion of a video is calculated, and a change amount calculation means using the value as a change amount; The change amount calculated by the calculation means is displayed on the display screen of the display device, and the number of frames to be processed is specified by the specified number, the process is performed according to the processing request, and several frames worth An image editing apparatus having a time adjustment processing execution means for adjusting time is known (see, for example, Patent Document 1).

A moving speed detecting means for detecting a moving speed in the photographing direction by comparing the video signal of the current image with the video signal of the previous screen; and a sudden speed when the moving speed detected by the detecting means is equal to or higher than a certain speed. There is also known an electronic image pickup apparatus and a playback apparatus in which an image processing for obtaining a playback screen without moving an image, that is, a lens focus is adjusted to blur an image (see, for example, Patent Document 2).

Furthermore, it is a simulator in which different right and left deflection glasses are attached and a three-dimensional image is viewed by watching different video images displayed on the rear projection screen. By adding an image pattern in the reverse direction, the sense of speed is increased. A motion perception control device that prevents the occurrence of simulator sickness by adding an image pattern in the same direction is known (for example, see Patent Document 3).
Japanese Patent Laid-Open No. 10-98675 (first page, FIG. 1) Japanese Patent Laid-Open No. 4-312074 (first page, FIG. 1) JP-A-9-269723 (first page, FIG. 1, FIG. 4, FIG. 5)

However, in the conventional example described in Patent Document 1 above, if two or more moving objects are moving at different speeds and different vectors in the shooting screen, two reference points (sample points) Since the motion vector is predicted in (area), it is not always the case that the photographer is aware of the pan and tilt that the photographer is performing. However, there is an unsolved problem that it is impossible to cope with the rapid movement of the photographer in the pann and tilt directions.

In addition, in the conventional example described in the above-mentioned Patent Document 2, if two or more moving objects are moving at different speeds and different vectors in the photographing screen, a two-point reference signal (sample point area) ) Is not necessarily recognizing the panning and tilting movements performed by the photographer, and the focus is controlled, so the image is not blurred. There is an unsolved problem that the entire image is uniformly blurred, and a portion with little movement in the screen is blurred and the resolution is lost.

Furthermore, in the conventional example described in the above-mentioned Patent Document 3, since the stimulus that does not move is displayed separately from the motion screen and the sickness is suppressed by displaying the rear projection screen, the stimulus to be displayed is originally an image. There is an unresolved problem that the stimulation is not related, so there is a sense of incongruity, and in the case of video for viewing such as a movie, it is often disturbed.
Therefore, the present invention has been made paying attention to the unsolved problems of the conventional example described above, and appropriately controls the image blur by accurately determining the panning and tilting operations performed by the photographer. An object of the present invention is to provide an image information reproducing apparatus capable of performing the above.

An image information reproducing device according to a first invention includes an imaging device that captures a moving image, an angular velocity detection unit that detects at least one angular velocity in a Pann direction and a tilt direction attached to the imaging device, and the angular velocity detection unit. An image information superimposing density setting unit that sets a density for adding image information of adjacent image frames out of image frames captured by the imaging device according to the detected angular velocity, and a density set by the image information superimposing density setting unit And image information reproducing means for adding and reproducing image information of adjacent image frames.

In the first aspect of the invention, the angular velocity detection means detects at least one angular velocity in the Pann direction and the tilt direction of the imaging device, and among the image frames captured by the imaging device according to the detected angular velocity, the image of an adjacent image frame By setting the density to add the information, it is possible to give an appropriate blur to the movement in the pan and tilt directions, and to suppress sickness.
Further, in the image information reproducing apparatus according to the second invention, in the first invention, the angular velocity detection means detects two gyroscopes that individually detect angular velocities in the Pann direction and the tilt direction, and the two gyroscopes. It has at least a low-pass filter that removes noise components from the angular velocity.

In the second invention, the angular velocities in the panning direction and the tilt direction of the imaging device are detected by two gyros, and noise components are removed from the detected angular velocities by a low-pass filter. Can be detected.
Further, in the image information reproducing apparatus according to the third invention, in the first or second invention, the image information superimposed density setting means is a dead band width in the vicinity of a stationary state with respect to the angular velocity output from the two gyros. The image forming apparatus is characterized in that a dead band width setting unit is provided for setting a density at which image information is added for angular velocities exceeding the dead band width set by the dead band width setting unit.

In the third aspect of the invention, the gyro is often output as an analog signal and is easily affected by disturbance. Therefore, by providing a dead zone near the stationary state, the influence of the disturbance is removed and an accurate angular velocity is detected. be able to.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a perspective view showing a schematic configuration of an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes an imaging device such as a digital video camera for imaging a moving image, and an imaging lens 2 is provided on the front surface of a housing 1. In addition, piezoelectric gyros 3 and 4 serving as angular velocity sensors for individually detecting the angular velocities in the panning direction, that is, the yaw direction and the tilt direction, that is, the pitch direction, of the imaging apparatus 1 are disposed.

As shown in FIG. 2, the block diagram of the imaging apparatus 1 projects a subject image onto an imaging unit 6 composed of a CCD element via an imaging lens 2, and subject image data from the imaging unit 6 is frame by frame. The digital signal is converted by the A / D converter 7 and output to the image processing unit 8. The image processing unit 8 receives the angular velocity detection values in the Pann direction and the tilt direction output from the piezoelectric gyros 3 and 4 and receives the angular velocity ω in the Pann direction and the tilt direction detected by the angular velocity detection unit 9.
p and ωt are input, and the input angular velocities ωp and ωt are stored in the captured image memory 10 together with the image information of one frame, and at least three consecutive frames are stored in the captured image memory 10. The image information and the angular velocities ωp and ωt are stored, the image information for one frame stored in the memory of the captured image memory 10 and the image information for one frame before and after the image information are read out, and the corresponding image Based on the angular velocity ωp and ωt in the Pann direction and the tilt direction stored together with the information, a density for adding the image information of the previous and subsequent frames is set, and one frame before and after the current frame of image information with the set density. Reproduced image information is formed by adding together the image information of the minutes and output to the signal processing unit 11. In the signal processing unit 11, luminance signal processing and color signal processing are performed on the reproduction image information processed by the image processing unit 8, and the processed reproduction image information is output to and stored in the reproduction image memory 12. ,
After being converted into an analog signal by the D / A converter 13, it is output to the display device 14 constituted by a liquid crystal display, an organic EL display, or the like.

Here, as shown in FIG. 3, the angular velocity detection unit 9 receives the angular velocity detection values output from the piezoelectric gyros 3 and 4 and removes the influence of disturbance in the vicinity of the stationary state, and thus a low-pass filter 9 a.
And 9b, and A / D converters 9c and 9d for converting the filter output from the low-pass filters 9a and 9b, from which the influence of disturbance has been removed, into digital signals. The detected digital angular velocity values ωp and ωt output from the A / D converters 9 c and 9 d are supplied to the image processing unit 8. The sampling rate of the piezoelectric gyros 3 and 4 is preferably set to be the same as the sampling rate of the image frame or an integral multiple of the sampling rate so as to cause sampling at the same timing.

The image processing unit 8 executes captured image processing shown in FIG. 4 when a shooting switch (not shown) is turned on. In this captured image processing, first, in step S1, it is determined whether or not digital image information for one frame captured by the imaging unit 6 input from the A / D converter 7 is input, and digital for one frame is determined. When the image information is not input, it waits until it is input, and when the image information for one frame is input, the process proceeds to step S2, and the angular velocities ωp and ωt in the Pann direction and the tilt direction are obtained from the angular velocity detector 9. Then, the process proceeds to step S3, where the read digital image information for one frame is stored in the captured image memory 10 together with the angular velocities ωp and ωt, and then the process proceeds to step S4.

In this step S4, it is determined whether or not the number n of image information for one frame stored in the captured image memory 10 exceeds 1, and when n = 1, image correction processing cannot be performed. The process proceeds to step S7, which will be described later, and the image information for one frame is output to the signal processing unit 11 as it is. If n> 1, it is determined that the image correction process can be performed, and step S5 is performed. Migrate to

In this step S5, the previous 1 is based on the larger one of the angular velocities ωp and ωt stored together with the previous one frame of image information stored in the captured image memory 10.
The density of the image information for one adjacent frame to be added to the image information for the frame is set. The added density of the image information for one adjacent frame is set with reference to the added density calculation map shown in FIG. As shown in FIG. 5, the addition density calculation map has a dead band width −Δω ≦ ωi ≦ + Δω in which the angular velocity ωi is set to a value near “0”, that is, ± Δω.
Is set to “0”%, and when the angular velocity ωi exceeds the dead zone width and increases clockwise or counterclockwise, a characteristic line is set so that the addition concentration increases accordingly. In addition, the maximum value of the added density is set to 50%. Here, the added density represents the total value of the density of the image information for one frame before and after the adjacent one, and 50% is set to 25% for the density of the image information of the previous and next frame. The reason why the maximum value of the addition density is set to 50% in this way is that the density of the addition is abrupt when the image pickup apparatus 1 returns to a stationary state in a short time from a state where it is panned or tilted at a large angle. Since an afterimage remains when the change is made, the combined density is preferably about 50% at maximum in order to prevent this afterimage. In addition, the dead zones are often provided because the piezoelectric gyros 3 and 4 output an output signal as an analog voltage value, and as shown in FIG. In order to eliminate the influence of the above, a dead zone is provided near the stationary state.

Next, the process proceeds to step S6, and the previous one frame of image information stored in the captured image memory 10 is added to the previous and subsequent frames of image information based on the sum density set in step S5. The reproduction image information is formed by adding together, and then the process proceeds to step S7 to output the reproduction image information formed to the signal processing unit 11, and then the process proceeds to step S8.
In this step S8, it is determined whether or not the photographing switch is turned off and the photographing is finished. When the photographing switch is in the on state, it is judged that the photographing is continued, and the process returns to the step S1 to take the photographing. When the switch is in the OFF state, it is determined that the shooting has been completed, and the process is terminated.

In the process of FIG. 4, the process of step S5 corresponds to the image information superimposed density setting means,
Steps S6 and S7 correspond to the reproduction image information forming unit.
Next, the operation of the above embodiment will be described.
Now, it is assumed that the photographing switch of the imaging apparatus 1 is in an off state and photographing is stopped.
In this shooting stop state, image information and angular velocities ωp, ωt for at least three consecutive frames stored in the captured image memory 10 are not stored, and moving image information of the subject image is captured by the imaging unit 6. Absent.

When the shooting switch is turned on to shoot a moving image from this shooting stop state, the execution of the captured image processing of FIG. 4 is started.
For this reason, when the image information for the first frame is output from the imaging unit 6 and converted into digital image information by the A / D converter 7 and input to the image processing device 8, the image information for the one frame is output. The information and angular velocity detection values ωp and ωt in the pan direction and tilt direction at that time are read and stored in the captured image memory 10.

At this point in time, the image information for one frame stored in the captured image memory 10 is one and the number of frames n is “1”.
And is subjected to luminance signal processing, color signal processing, etc., and is stored in the reproduced image memory 12 as it is as reproduced image information, and is also output to the display device 14 via the D / A converter 13 for display. Is done.

Thereafter, when image information for the second one frame is input from the imaging unit 6, the image information and angular velocity detection values ωp and ωt in the pann and tilt directions at that time are stored in the captured image memory 10. The
Therefore, since the number n of image information stored in the captured image memory 10 is “2”, FIG.
In step S4, the process proceeds from step S4 to step S5, and the added density is calculated with reference to the added density map of FIG. 5 based on the larger one of the previous angular velocities ωp and ωt.

At this time, since the number n of the image information stored in the captured image memory 10 is “2”, the sum density is 1 from the sum density of 100% from the previous one frame of image information.
Reproduced image information is formed by adding the density obtained by subtracting / 2 and the density of 1/2 of the current image information for one frame, and this reproduced image information is stored in the reproduced image memory 12. At the same time, the analog signal is converted by the D / A converter 13 and then displayed on the display device 14.

Next, when the third image information is input from the imaging unit 6, either one of the angular velocities ωp and ωt is obtained by gathering the image information for one frame before and after the image information for the previous one frame. The sum density is set based on the large value, and based on this, the image information for one frame before and after the image information for the previous one frame is added.
Thereafter, by repeating this process, as shown in FIG. 7, the image information for one frame before and after the previous image information is added to the image information for the previous one frame.

Therefore, the angular velocities ω in the Pann direction and the tilt direction detected by the piezoelectric gyros 3 and 4
When the absolute value of p or ωt, whichever is larger, is larger than the dead band width, for example, the previous 1
The sum density for the frames is set to 80%, and the image information for one frame before and after the sum is added by 10% to form reproduced image information. Further, when a larger angular velocity is detected, for example, the total density for one frame is added to 50%, and image information for one frame before and after that is added by 25%. For this reason, when the imaging device 1 is moved in the Pann direction or the tilt direction in accordance with the moving subject speed, the moving subject becomes a clear image, and the surrounding landscape has 1 Since the position differs depending on the image information for the frame, the display becomes blurred when the image information for the previous and subsequent frames are added together, and the blurring becomes larger as the angular velocity increases, and the viewer feels sick. Can be suppressed.

As described above, according to the above embodiment, the angular velocities in the panning direction and the tilt direction of the photographing apparatus 1 can be accurately detected by the piezoelectric gyros 3 and 4, so that when fast panning or tilting is performed, The viewer may feel uncomfortable by adding the image information for the previous and subsequent frames to the previous frame of image information in accordance with the angular velocity and adding the image information for the previous and subsequent frames according to the angular velocity. It is possible to form reproduced image information that suppresses the above.

In addition, the gyroscope can accurately detect the angular velocity in the panning and tilting directions of the photographic device, and it can detect only the shooter's panning and tilting to perform accurate image correction, and calculate motion vectors. Since this is not the case, processing power can be reduced.
In the above-described embodiment, the case has been described in which the sum density of the image information for the previous and subsequent frames is set equal to the previous one frame of image information, but the present invention is not limited to this. The sum density may be different from the image information for one frame.

In the above embodiment, the case has been described where the image information for one frame before and after the image information to be added to the image information for one frame at a certain point in time is not limited thereto. Only the previous frame in time series with respect to the current frame is added with the addition density according to the angular velocity, or only the subsequent frame in time series with respect to the current frame is added with the addition density according to the angular velocity. The image information for a plurality of frames before and after the current frame may be added with the addition density corresponding to the angular velocity. Furthermore, when the frame rate is very high and there is almost no difference between adjacent frames, frames that are not adjacent in time series to the current frame may be added together with the addition density corresponding to the angular velocity. In this case, the processing can be reduced by processing every few frames, for example.

Furthermore, in the above-described embodiment, a case has been described in which image information addition processing according to angular velocity is performed in the imaging apparatus 1, but the present invention is not limited to this, and an image for one frame is captured by the imaging apparatus 1. Each time information is obtained, the current angular velocity is added and stored in a non-volatile memory such as a flash memory. This non-volatile memory is set in the image reproducing device, and the image reproducing device adds image information based on the angular velocity. A matching process may be performed.

Furthermore, in the above-described embodiment, the case where the dead zone is provided in the area where the angular velocity is near zero in the summed density calculation map has been described. However, the present invention is not limited to this, and the dead zone can be arbitrarily set. The dead zone may be omitted as shown in FIG.
Furthermore, in the above embodiment, the case where the angular velocity is detected by the piezoelectric gyro has been described. However, the present invention is not limited to this, and other types of gyros can be applied, and other angular velocity sensors can be applied. Can also be applied.

In the above embodiment, the angular velocities in the Pann direction and the tilt direction are individually detected,
Although the case where the sum density of the adjacent image frames is set based on the angular velocity of either of the two values has been described, the present invention is not limited to this, and the adjacent image frames are determined based on the average value of both angular velocities. You may make it set the addition density | concentration.
Furthermore, in the above embodiment, the case where the angular velocities in both the pann direction and the tilt direction are detected has been described. However, the present invention is not limited to this, and at least one angular velocity in the pann direction and the tilt direction is detected, The sum density of adjacent image frames may be set based on the detected angular velocity.

1 is a schematic configuration diagram of an imaging apparatus showing an embodiment of the present invention. FIG. 2 is a block diagram illustrating a circuit configuration of the imaging apparatus in FIG. 1. It is a block diagram which shows the specific structure of the angular velocity detection part of FIG. It is a flowchart which shows an example of the captured image processing procedure performed with an image process part. It is a characteristic diagram showing an addition density calculation map showing the relationship between the angular velocity and the addition density. It is a characteristic diagram which shows the output characteristic of a gyro. It is explanatory drawing which shows the setting condition of the addition density | concentration of the image information of the front and back frames according to angular velocity. It is a characteristic diagram which shows the other example of an addition density | concentration calculation map.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Imaging device, 2 ... Imaging lens, 3, 4 ... Piezoelectric gyro, 6 ... Imaging part, 7 ... A / D converter, 8 ... Image processing part, 9 ... Angular velocity detection part, 10 ... Memory for captured images, DESCRIPTION OF SYMBOLS 11 ... Signal processing part, 12 ... Memory for reproduced images, 13 ... D / A converter, 14 ... Display apparatus

Claims (3)

An imaging device that captures a moving image, an angular velocity detection unit that detects at least one angular velocity in a Pann direction and a tilt direction attached to the imaging device, and an image that is captured by the imaging device according to the angular velocity detected by the angular velocity detection unit An image information superimposing density setting means for setting a density for adding image information of adjacent image frames in the image frame, and adding the image information of the adjacent image frames at the density set by the image information superimposing density setting means An image information reproducing apparatus comprising: reproduced image information forming means for forming reproduced image information. The angular velocity detection means includes at least two gyros that individually detect angular velocities in the Pann direction and the tilt direction, and a low-pass filter that removes noise components from the angular velocities detected by the two gyros. The image information reproducing apparatus according to claim 1. The image information superimposing density setting means has a dead band width setting means for setting a dead band width in the vicinity of a stationary state with respect to the angular velocity output from the two gyros, and the dead band width set by the dead band width setting means. 3. The image information reproducing apparatus according to claim 1, wherein the image information reproducing apparatus is configured to set a density at which the image information is added for an angular velocity exceeding.

Images