JP3605870B2 - Panoramic still image creation device - Google Patents

Description

[0001]
[Industrial applications]
The present invention relates to an apparatus for creating a panoramic still image, which is a single wide-angle still image, by joining together images obtained by imaging a subject using an image sensor such as a CCD.
[0002]
[Prior art]
Conventionally, as this kind of panoramic still image creating apparatus, there has been one disclosed in Japanese Patent Application No. 4-297773 (Japanese Patent Application Laid-Open No. 6-284321) filed by the present applicant earlier. This panoramic still image creating apparatus uses, for example, as shown in FIG. 7, an image using motion vectors of images A1 to A8 captured while panning a camera-integrated video tape recorder (hereinafter referred to as “camcorder”). Are connected in the horizontal direction to create one wide-angle still image as shown in FIG.
[0003]
[Problems to be solved by the invention]
However, in the conventional panoramic still image creating apparatus, if the panning speed is uneven, the width of the images to be connected is also uneven due to a change in the motion vector, and the images are not connected well.
[0004]
Also, when the camcorder shakes up and down during panning, and a vertical movement of the captured image occurs, as shown in FIG. There were irregularities in the joints, and there were parts that seemed to be less beautiful as a panoramic still image. Therefore, it is conceivable to fill the unevenness, but if this is done, the vertical width of the panoramic still image becomes narrow as shown in FIG.
[0005]
The present invention has been made in view of such a problem, and a panoramic still image creating apparatus that can perform an appropriate panning operation by displaying a panning state at the time of panoramic shooting on a viewfinder or the like. The purpose is to provide.
[0006]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the present invention provides a panoramic still image by connecting a plurality of images formed by image signals created by imaging a subject while panning an imaging device based on the motion vector. An apparatus for creating an image, comprising: first means for detecting horizontal and vertical movements of the image pickup apparatus; and obtaining motion vector information based on an output of the first means; A panning speed (hereinafter, referred to as an appropriate value) suitable for creating a panoramic still image by joining the plurality of images by determining a panning speed (hereinafter, referred to as a panning speed) of the imaging device based on the information. Means for outputting information indicating the magnitude and direction of the displacement with respect to the image, and the panning speed of the imaging device with respect to the appropriate value based on the output of the second means. Characterized in that it comprises a third means for displaying the information indicating the magnitude and direction of the deviation of.
[0007]
Further, in the panoramic still image creation device according to the present invention, the second means determines vertical camera shake of the imaging device based on vertical motion vector information, and determines the magnitude of camera shake of the imaging device. And a function of outputting information indicating a direction and a magnitude of a camera shake of the imaging apparatus based on an output of the second means. Features.
[0008]
[Action]
According to the present invention, in the panoramic still image creation device, the first means detects the horizontal and vertical movements of the imaging device, and the second means determines the panning speed and the appropriate value based on the detected output. The relationship is determined, and the relationship between the panning speed and the appropriate value is displayed by the third means based on the determined output.
[0009]
Also, according to the present invention, the camera shake of the imaging device is determined by the second means based on the detection output of the means of the first means, and the camera shake of the imaging device is determined by the third means based on the determination output. Information indicating the size and direction of is displayed.
[0010]
【Example】
[1] Camcorder to which the Present Invention is Applied FIG. 1 is a diagram showing a system configuration of a camcorder to which the present invention is applied.
The video signal stored in the CCD imager 2 through the lens 1 is converted to a digital video signal by the A / D converter 4 via the S / H and AGC circuit 3. This digital video signal is input to a memory system block 6 after being subjected to camera signal processing such as gamma correction and white balance in a camera signal processing block 5.
[0011]
On the other hand, the angular velocity information of the camcorder main body obtained from the horizontal and vertical angular velocity sensors 7 is amplified after the DC component is cut off by the HPF and the amplifier 8, and the A / D converter 9 converts the digital angular velocity information into A / D. D conversion is performed. This digital angular velocity information is integrated by the microcomputer 10 and converted into angle information, which is further processed and sent to the memory system block 6 as a motion vector.
[0012]
Further, a control signal from a panorama mode control switch provided on the operation unit 11 is given to the memory system block 6 via the microcomputer 10. Then, the writing / reading of the memory is controlled based on this control signal and the above-described motion vector information, and the image is reduced and synthesized, and a panoramic still image is synthesized in the memory. Details of the panoramic still image synthesizing operation in the memory system block 6 will be described later.
[0013]
During the synthesis of the panoramic still image, the video signal of the memory system through is input, and after the synthesis is completed, the panoramic still image video signal read from the memory is input to the video signal processing block 12, where it is formed as a composite video signal. , To the synthesizing circuit 13.
[0014]
The combining circuit 13 combines the reference indicator created by the reference indicator creating circuit 14 at the time of panning shooting with the video signal. The output of the synthesis circuit 13 is converted into an analog video signal by the D / A converter 15 and sent to the synthesis circuit 16. The combining circuit 16 combines the panning indicator created by the panning indicator creating circuit 17 with the video signal during panning shooting.
[0015]
FIG. 2 shows an example of the reference indicator and the panning indicator. Here, the reference indicators Lx and Ly are respectively dotted lines in the horizontal and vertical directions passing through the center of the screen. The panning indicators C0 to C2 are circular indicators, and their positions indicate panning speed and blur. That is, when it is at the center of the screen as in C0, it indicates that the panning speed is appropriate and there is no blur. In addition, when it is in the panning direction as in C1, it indicates that the speed is too fast. Further, when the direction is opposite to the panning direction as in C2, it indicates that the speed is too slow. Then, although not shown, when the center of the circle is shifted upward or downward from the reference indicator Lx, it indicates that the camcorder is moving upward or downward.
[0016]
The reference indicator is created at the time of panning shooting as described above. This is because the microcomputer 10 detects that the user operates the panorama mode setting switch of the operation unit 11 to enter the panorama creation mode, and the microcomputer 10 sets the reference indicator. This is performed by sending a control signal to the creation circuit 14. The panning indicator indicates that the microcomputer 10 determines the magnitude and direction of the panning speed based on the horizontal motion vector information input from the A / D converter 9 and the magnitude of camera shake based on the vertical motion vector. The direction is determined by sending a control signal to the panning indicator generating circuit 17. Note that another microcomputer may control the creation of the panning indicator and the control of the creation of the reference indicator.
[0017]
A video signal obtained by combining the above indicators is output to a viewfinder or a liquid crystal monitor provided in a camcorder housing. Therefore, if the user during panning shooting looks at the viewfinder or the LCD monitor and operates the camcorder so that the panning indicator is at the center of the screen, panning shooting with constant speed and without blurring can be performed. . As a result, the created panoramic still image has a beautiful connection and a large vertical width.
[0018]
When the recording mode is set after the completion of the panoramic still image synthesis, the panoramic still image video signal is subjected to video signal processing such as emphasis of a luminance signal, FM modulation, and low-frequency conversion of a color signal in a video signal processing block 12. After being D / A converted by the / A converter 18, it is sent to the video head 20 via the R / P amplifier 19 and recorded on the video tape 21 by electromagnetic conversion.
[0019]
At the time of reproduction, a video signal reproduced from the video tape 21 by the video head 20 is converted into a digital signal by the A / D converter 22 through the R / P amplifier 19, and the low frequency band of the color signal is outputted by the video reproduction processing block 23. The video signal is subjected to inverse conversion of the conversion, video signal reproduction processing such as FM demodulation and de-emphasis of the luminance signal, and sent to the memory system block 6.
[0020]
In the memory system block 6, the video signal that has been subjected to the panoramic still image synthesizing process at the time of recording is output to the video signal processing circuit 12 as it is. The video signal processing circuit 12 outputs the input video signal to the synthesizing circuit 13. The video signal input to the synthesizing circuit 13 is output through the D / A converter 15 and the synthesizing circuit 16. At this time, since the panning shooting is not being performed, the creation and combination of the reference indicator and the panning indicator are not performed.
[0021]
[2] Configuration / operation of memory system [when synthesizing a panoramic still image]
FIG. 3 shows the configuration and operation of the memory system when synthesizing a panoramic still image. Here, the switches SW1 to SW3 are connected to the solid line side. A dotted line output from the memory controller 31 indicates a control signal.
[0022]
The video signal input from the camera signal processing block 5 of FIG. 1 is input to the SRAM / 4-point interpolation circuit 32 through the switches SW1 and SW2, and is passed through the switch SW3 to the outside for monitor output. Is output. As a result, the captured image can be viewed even during the synthesis of the panoramic still image. Also, an indicator can be combined with this image.
[0023]
The SRAM / 4-point interpolation circuit 32 uses the SRAM to reduce the size in the horizontal direction according to the panorama reduction rate, and the 4-point interpolation circuit uses horizontal and vertical 1 dot or less / 1 line or less corresponding to the motion vector. A phase shift is performed. The reason for not reducing the vertical direction before synthesizing the panoramic still image is to minimize the loss of image quality. When synthesizing a panoramic still image by panning, there is room in the memory in the vertical direction. Is reduced when the panoramic still image after the synthesis is read from the field memory 33, and the SRAM / 4-point interpolation circuit 32 shifts the phase by one line or less in the vertical direction.
[0024]
Since the composing field memory 33 cannot perform random access on a column basis, the reduced original image cannot be written to an arbitrary position. Therefore, in synthesizing the image, the video signal read from the field memory 33 and the video signal reduced / interpolated in the horizontal direction by the SRAM / 4-point interpolation circuit 32 are output to the feedback path by switching the bus switch 34, and each time the field signal is output. A method in which all data is overwritten on the memory 33 is adopted.
[0025]
That is, in the field memory 33, a composite image is read according to the motion vector. At this time, since the reduced original image has a fixed timing, the synthesized image side shifts the timing to perform the positioning, so that the timing is actually shifted in the direction opposite to the motion vector and read. The video signal read from the field memory 33 is input to C of the bus switch 34, and the reduced original video signal from the SRAM / 4-point interpolation circuit 32 is input to B, and the bus is linked to the effective screen size of the reduced original video signal. The switch 34 is switched to output a composite video signal of B and C from D.
[0026]
This synthesized video signal is overwritten again at the same position in the field memory 33 through the feedback path, and the synthesis for one field is completed. By repeating this process, a panoramic still image is formed on the field memory 33.
[0027]
The panoramic still image synthesis is controlled entirely by the memory controller 31. The memory controller 31 receives information from the microcomputer 10 of FIG. 1 for each field, processes the information on a line-by-line basis, and sends a control signal to each block in real time. .
[0028]
FIG. 4 shows the panoramic still image synthesizing operation described above. In this figure, (1) to (8) show processing of images corresponding to the images A1 to A8 in FIG. The input image A on the left shows an image input to the memory system of FIG. The horizontal reduced image B at the center indicates an image which is input to the C of the bus switch 34 by reducing and interpolating the input image A to 1/3 in the horizontal direction by the SRAM / four-point interpolation circuit 32. The right synthesized memory image D indicates an image synthesized on the field memory 33 from D of the bus switch 34 through a feedback path. At this time, for example, the reduced image C of (2) is combined with the combined memory image D of (1) to become the combined memory image D of (2). By repeating this operation up to (8), the combined image drawn at the right end of (8) in FIG. 5 is formed on the field memory 33, and the combination of the panoramic still image is completed.
[0029]
[3] Configuration / operation of memory system [When displaying a panoramic still image]
FIG. 5 shows the configuration and operation of the memory system when displaying a panoramic still image.
Since the panoramic still image formed in the field memory 33 is not reduced in the vertical direction, it is thinned out and read according to the reduction ratio in the vertical direction. At this time, thinning is performed so as to interlace according to whether data read from the field memory 33 is data of an odd field or data of an even field.
[0030]
The data read from the field memory 33 is subjected to vertical interpolation in the SRAM / 4-point interpolation circuit 32 when the reduction ratio is not a fraction of an integer or when interlacing cannot be accurately performed by thinning-out reading. In the case of performing interpolation by vertical reduction, data of two lines that are always close to each other are required, so that a two-port readout field memory is used.
[0031]
The video signal that has been subjected to the reduction processing in the vertical direction in this manner is sent from the switch SW3 to the video signal processing circuit. FIG. 6 shows an example of a panoramic still image synthesized by the field memory 33 and a panoramic still image vertically reduced by the SRAM / 4-point interpolation circuit 32.
[0032]
It should be noted that the present invention is not limited to the above-described embodiment, and various modifications are possible based on the gist of the present invention, and they are not excluded from the scope of the present invention.
For example, in the above embodiment, the reference indicator is a line indicating the center of an appropriate position, but an area having a predetermined width from the center is set as an allowable range of an appropriate position, and the reference indicator indicates the allowable range. Is also good.
[0033]
In the above embodiment, the reference indicator is synthesized with a digital signal, and the panning indicator is synthesized with an analog signal. However, both may be synthesized with an analog signal or a digital signal. Further, the indicator creation circuit may be provided in a path through which the video signal in the memory system passes.
[0034]
Further, in the above-described embodiment, the panning speed and the camera shake are displayed. However, for example, in the case of a camcorder having a function of correcting the camera shake, only the panning speed may be displayed.
[0035]
In the above embodiment, the present invention is applied to a camcorder, but may be applied to a video signal recording / reproducing apparatus using a rotary recording medium such as a magnetic disk or a magneto-optical disk as a recording medium.
[0036]
【The invention's effect】
As described above in detail, according to the present invention, the panning state (speed, camera shake) is displayed on the viewfinder or the like during panning shooting, so that the user can perform an appropriate panning operation while watching this display. Therefore, the created panoramic still image has a beautiful connection and a wide vertical width.
[Brief description of the drawings]
FIG. 1 is a diagram showing a system configuration of a camcorder to which the present invention is applied.
FIG. 2 is a diagram illustrating an example of an indicator.
FIG. 3 is a diagram showing the configuration and operation of a memory system when a panoramic still image is synthesized.
FIG. 4 is a diagram illustrating an operation of synthesizing a panoramic still image.
FIG. 5 is a diagram showing the configuration and operation of a memory system when displaying a panoramic still image.
FIG. 6 is a diagram illustrating an example of a panoramic still image synthesized by a field memory and a panoramic still image vertically reduced by an SRAM / 4-point interpolation circuit.
FIG. 7 is a diagram illustrating a conventional panoramic still image creation method.
FIG. 8 is a diagram showing an example of a display screen of a panoramic still image created by a conventional method.
FIG. 9 is a diagram showing another example of a display screen of a panoramic still image created by a conventional method.
[Explanation of symbols]
13, 16: synthesis circuit, 14: reference indicator creation circuit, 17: panning indicator creation circuit, Lx, Ly: reference indicator, C0 to C2: panning indicator

Claims (2)

An apparatus for creating a panoramic still image by connecting a plurality of images formed by image signals created by imaging a subject while panning an imaging apparatus based on the motion vector thereof,
First means for detecting horizontal and vertical movements of the imaging device;
Motion vector information is obtained based on the output of the first means, a panning speed of the image pickup apparatus (hereinafter, referred to as a panning speed) is determined based on the horizontal motion vector information, and the plurality of images are connected. Second means for outputting information indicating the magnitude and direction of a shift with respect to a panning speed (hereinafter referred to as an appropriate value) suitable for creating a panoramic still image;
A panoramic still image creating apparatus, comprising: third means for displaying information indicating a magnitude and a direction of a shift of the panning speed of the imaging apparatus with respect to the appropriate value based on an output of the second means. The second means further has a function of determining vertical camera shake of the imaging device based on vertical motion vector information, and outputting information indicating the magnitude and direction of the camera shake of the imaging device,
2. The panoramic still image according to claim 1, wherein said third means further has a function of displaying information indicating a magnitude and a direction of camera shake of said imaging device based on an output of said second means. Creation device.

Images