JP2000101895A - Image signal photographing device and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow the user to easily photograph a plurality of partial pictures used for generated a panoramic picture. SOLUTION: A fixed mark 32 shown in a square is displayed in the middle of a screen frame 31 of a viewfinder 18. Then a target frame 33 shown in dotted lines is set in the inside of the screen frame 31 and a partial picture is photographed by one frame when a movable mark 34 overlaps the fixed mark 32. When the partial picture is photographed, the target frame 33 and the movable mark 34 are moved in a horizontal or vertical direction to photograph a succeeding partial picture. Then the user moves a viewfinder while observing the viewfinder 18 so as to display the target frame 33 and the movable mark 34 in a way of approaching the screen frame 31 and the fixed mark 32 and the succeeding partial picture is photographed when the target frame 33 is contained in the screen frame 31 and the movable mark 34 overlaps the fixes mark 32.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera and a digital VCR or a digital still camera which are used to shoot a plurality of partial images for generating a panoramic image so that adjacent partial images can be easily shot. The present invention relates to an image signal imaging apparatus and method described above.

[0002]

2. Description of the Related Art Conventionally, for example, a digital V
In order to obtain a panoramic image using CR, recording is performed while moving the photographing area little by little so that a partial image for each frame (or field) can be obtained continuously from the photographing start point. Then, with respect to the recorded partial images, the overlapping portions of the partial images adjacent to each other in position are extracted, and a predetermined operation is performed on the overlapping portions to seamlessly combine the respective partial images. And as a whole 1
Image processing for obtaining a still image is performed. Such image processing is performed by software processing by a personal computer, for example.

[0003] By applying such a photographing method, a panoramic image can be obtained by a digital VCR integrated with a camera. The user shoots while moving the camera in the horizontal direction. For example, 360 ° around the user is continuously photographed. By connecting the obtained partial images for each frame while applying appropriate processing to the overlapping portion, a horizontally long panoramic image can be obtained.

[0004]

However, when a user shoots a partial image for synthesizing a panoramic image, it is difficult to check whether or not adjacent partial images overlap sufficiently. There was a problem of scarcity.

Accordingly, it is an object of the present invention to provide an image signal imaging apparatus and method capable of easily photographing adjacent partial images for generating a panoramic image.

[0006]

According to a first aspect of the present invention, there is provided an image signal imaging apparatus for capturing a plurality of partial images for generating a panoramic image, comprising: a display unit for displaying a captured image; Having a motion detection means for detecting motion, displaying a fixed mark at the center of the screen of the display unit, displaying a movable mark at the same position as the fixed mark, and after the partial image is taken, the movable mark is next Move to the position of the partial image to be captured, move the movable mark according to the detection result of the motion detection means, and capture the next partial image to be captured when the fixed mark and the movable mark overlap An image signal imaging device characterized by the following.

According to a seventh aspect of the present invention, in the image signal imaging method for capturing a plurality of partial images to generate a panoramic image, a fixed mark is displayed at the center of a screen of a display unit for displaying the captured image. The movable mark is displayed at the same position as the fixed mark, and after the partial image is shot, the movable mark moves to the position of the next partial image to be shot, detects the movement of the main body, and responds to the detected movement. An image signal imaging method characterized in that when a movable mark moves and a fixed mark and a movable mark overlap each other, a partial image to be captured next is captured.

[0008] A fixed mark indicated by a square is displayed at the center of the screen frame of a viewfinder (display unit) provided in the camera-integrated digital VCR. Then, when the target frame indicated by the dotted line fits within the screen frame and the movable mark overlaps the fixed mark, the user captures a partial image of one frame.
When a partial image is shot, to shoot the next partial image,
The target frame and the movable mark move in the horizontal or vertical direction. Then, the user moves the camera-integrated digital VCR while looking at the viewfinder, bringing the target frame and the movable mark closer to the screen frame and the fixed mark,
When the target frame falls within the screen frame and the movable mark overlaps the fixed mark, the next partial image is shot. in this way,
Since the target frame and the movable mark are displayed at the positions of the adjacent partial images, the preset parameters, for example, the overlap rates do not greatly differ. Therefore, a panoramic image can be easily synthesized from a plurality of captured partial images.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a block diagram of an embodiment of a camera-integrated digital VCR to which the present invention is applied. The lens system 11 can adjust an optical system such as iris, focus, and zoom. These adjustments are respectively controlled by the lens system drive circuit 22. This control is automatically performed based on the control of the system controller 20 described later. The adjustment of the optical system is not limited to this, and can be performed manually. Based on this control, control information is output from the lens system drive circuit 22. In this example, respective values of iris, focus, and zoom are output. This control information is supplied to the system controller 20 and the recording medium 14.

An object is CCD (Cha) through the lens system 11.
rge Coupled Device) 12. The illuminated subject is converted by the CCD 12 into an electric signal. Although not shown, the CCD 12 has a predetermined signal processing circuit, and further converts the converted electric signal into digital image data and outputs it. The number of pixels of the CCD 12 is
It corresponds to the image size of the moving image area. From the CCD 12,
Image data is output at the timing of the field.

The output of the CCD 12 is supplied to a REC terminal of a switch circuit 17 and is also supplied to a compression circuit 13. The motion detection circuit 23 is connected to the compression circuit 13. The motion detection circuit 23 obtains a motion vector by comparing image data between adjacent frames in time series, and thereby detects a motion component of the image data.
In the compression circuit 13, this motion vector and DCT (D
The supplied image data is compressed using, for example, iscrete cosine transform. The motion component output from the motion detection circuit 23 is supplied to a mixing (MIX) circuit 24. The mixing circuit 24 mixes the motion information from the angular velocity sensor 25 described later and the motion component from the motion detection circuit 23, and supplies the mixing result to the recording medium 14 and the system controller 20.

The recording medium 14 is supplied with the compressed image data from the compression circuit 13, the control information from the lens system drive circuit 22, and the mixing result from the mixing circuit 24, and mixes the control information with the control information for each compressed image data. The result is recorded as subcode data. The recording medium 14 may be any recording medium as long as it can record digital signals, such as a hard disk, a magneto-optical disk,
VD (Digital Versatile Disc), MD (Mini Disk),
A recording medium such as a semiconductor memory and a magnetic tape can be used. The compressed image data and subcode data read from the recording medium 14 are supplied to a decompression circuit 15.

The decompression circuit 15 decompresses the compression encoding applied to the compressed image data at the time of recording, and outputs it as reproduced image data. The reproduced image data output from the decompression circuit 15 is supplied to the PB terminal of the switch circuit 17. The sub-code data extracted from the recording medium 14 is supplied to the data output terminal 16.

In photographing (recording), the switch circuit 17 selects the terminal REC, so that C
An image being captured by the CD 12 can be displayed on the viewfinder 18. This image can be output via the video output terminal 19 and displayed on an external video monitor or the like.

At the time of reproduction, the PB terminal is selected by the switch circuit 17, and reproduced image data is output via the video output terminal 19. At the same time, the reproduced image data is supplied to the viewfinder 18. When the PB terminal is selected by the switch circuit 17 during reproduction, the reproduced image from the recording medium 14 is displayed on the viewfinder 1.
8 can be displayed.

The system controller 20 comprises a CPU, a RAM and a ROM, and controls the operation of the camera-integrated digital VCR. The operation key unit 21 is provided with a switch for performing various settings of the camera-integrated digital VCR and a switch for switching ON / OFF of the panoramic photographing. Setting information from these various setting keys is supplied from the operation key unit 21 to the system controller 20 and stored in, for example, a RAM.

For this camera-integrated digital VCR, an angular velocity sensor 25 composed of, for example, a gyro sensor is provided. As the angular velocity sensor 25, a sensor mounted on a camera-integrated digital VCR for a so-called camera shake prevention function can be used. The output of the angular velocity sensor 25 is integrated by the integration circuit 26 to obtain motion information.
In the integration circuit 26, an average movement is generated. That is, the integration circuit 26 generates motion information while suppressing fine motion. This motion information is supplied to the mixing circuit 24.

By using the output of the angular velocity sensor 25 as motion information, even when the motion detection circuit 23 does not work well, such as when photographing a flat subject or when photographing in an image or in darkness. The relative position of the captured image can be known. The angular velocity sensor 25 and the integrating circuit 26 can be omitted.

In the embodiment shown in FIG.
It is also possible to appropriately switch between the motion component from the motion detection circuit 23 and the motion information from the angular velocity sensor 25. That is, when the motion vector based on the output of the angular velocity sensor 25 is used when the motion vector is not correctly obtained by the motion detection circuit 23 in a flat portion of the subject, etc. Uses the motion component from the motion detection circuit 23 with higher accuracy. This switching is performed manually. The present invention is not limited to this. For example, automatic switching can be performed by detecting a pattern of image data.

When the panorama photographing switch provided on the operation key section 21 is turned on, the system controller 2 displays the fixed mark, the target frame, and the movable mark on the viewfinder 18 as described later.
The control signal is supplied from 0 to the viewfinder 18. The target frame and the movable mark move on the screen of the viewfinder 18 according to the mixing result from the mixing circuit 24. That is, according to the movement of the camera-integrated digital VCR, the target frame and the movable mark move in the horizontal direction and / or in the horizontal direction.
Alternatively, the viewfinder 18 may be moved in the vertical direction.
Will be displayed.

When the target frame is accommodated in the screen frame and the movable mark overlaps the fixed mark, the user presses the shutter to photograph a partial image of one frame. When a partial image is shot, the target frame and the movable mark move to the position of the next partial image to be shot, and the user moves the camera-integrated digital VCR to the moved position, and the next partial image is shot. Is photographed by the user. That is, the user moves the camera-integrated digital VCR so that the target frame fits in the screen frame and the movable mark overlaps the fixed mark while watching the viewfinder 18, and the target frame is set in the screen frame. Further, when the movable mark overlaps the fixed mark, a partial image is shot.

The image data thus photographed and recorded is reproduced and supplied to a device for performing predetermined image processing, for example, a personal computer (not shown) equipped with corresponding image processing software. In addition to the supply of the image data, the sub-code data is output from the data output terminal 16. This subcode data is also supplied to the personal computer together with the image data. A seamless synthesizing process is performed on the supplied image data by the image processing software, and a panoramic image can be obtained. At this time, by using the motion information and various shooting information such as zoom, iris, and focus supplied as the subcode data, it is possible to perform the synthesizing process with higher accuracy.

For example, the positional relationship of each frame image data can be known based on the motion information and / or motion component included in the subcode data. As a result, an overlapping portion between the respective frame image data is obtained, and by performing a predetermined operation on the overlapping portion, it is possible to automatically synthesize the respective frame image data. Also,
Performs image correction based on various optical system information, such as zoom, iris, and focus, included in the subcode data, and automatically synthesizes still images after homogenizing each frame image data. It is said.

For example, based on zoom information, conversion of frame image data into a plane or conversion into a cylindrical surface or a spherical surface can be performed. Further, the brightness of each frame image data can be corrected based on the iris information. Further, frame image data with different focus may be obtained even in the same photographing area. Based on the focus information, it is possible to select the sharpest frame image data in pixel units, whereby an image with a deep depth of focus can be obtained.

Using the camera-integrated digital VCR having such a configuration, the user turns on a panorama shooting ON / OFF setting switch provided on the operation key section 21 to start shooting for a panorama image. You. Then, the plurality of captured partial images are transferred to a personal computer equipped with, for example, the predetermined image processing software as described above, and are subjected to seamless processing.
One panoramic image can be completed.

An example of parameters necessary for performing a seamless process and generating a panoramic image is shown below.

1) Zoom magnification or horizontal angle of view 2) Aspect ratio 3) Auto focus value 4) Auto iris value 5) Information indicating direction of optical axis by angular velocity sensor 6) Motion detection data during compression a) Average of top, bottom, left and right B) Rotation in image plane c) Magnification rate of image (upper and lower horizontal magnifications may differ depending on the angle with the subject. Similarly, left vertical magnification and right horizontal magnification 7) Imaging time (when frame period or field period is not constant) These are small amounts of data obtained from a large amount of motion detection data at the time of compression, and are used as compressed image data. And recorded on the recording medium 14 as subcode data.

Here, a plurality of partial images are stored in the viewfinder 1.
An example of an algorithm for generating a panoramic image from a plurality of partial images shot by the user while viewing the image 8 is shown in the flowchart of FIG. Although the number of panoramic images to be combined and the overlap ratio are set to default values (initial values), they can be freely set in advance as long as the user can combine them.

In step S1, panorama shooting ON / O
The setting switch of the FF is turned on. When the panoramic shooting is turned on, a fixed mark 32, a target frame 33, and a movable mark 34 are displayed on the viewfinder 18 as shown in FIG. 3A. The screen frame 31 shows a screen of the viewfinder 18. The fixed mark 32 is displayed as a square in the center of the screen frame 31. The target frame 33 is displayed by a dotted line having substantially the same size as the screen frame 31 so as to fit inside the screen frame 31. The movable mark 34 is displayed as a triangle at the same position as the fixed mark 32. That is, the movable mark 34 is displayed at the center of the target frame 33.

In step S 2, a still image of one frame is captured and recorded on the recording medium 14 while the camera-integrated digital VCR is stationary at the subject. In this example, as shown in FIG.
Is stored in the screen frame 31 and when the movable mark 34 overlaps the fixed mark 32, a partial image of one frame is pressed by the user and photographed.

In step S3, as shown in FIG. 3B,
The target frame 33 and the movable mark 34 are moved to the position of the next partial image to be shot. In FIG. 3, the target frame 33 protruding from the screen frame 31 is left for ease of explanation, but only the target frame 33 in the screen frame 31 is actually displayed on the screen of the viewfinder 18. The target frame 33 that extends beyond the screen frame 31 is the viewfinder 18.
Is not displayed on the screen. Similarly, the portion of the movable mark 34 that protrudes from the screen frame 31 is not displayed on the screen of the viewfinder 18.

In step S4, the user moves the camera-integrated digital VCR in the horizontal direction while looking at the viewfinder 18, as shown in FIG. That is, by moving the camera-integrated digital VCR,
Since the optical axis moves, the view finder 18
As shown in C, a state until the target frame 33 fits into the screen frame 31 is displayed, and similarly, a state until the movable mark 34 overlaps the fixed mark 32 is displayed. At this time, the moving distance of the camera-integrated digital VCR is measured in real time based on the motion component from the motion detecting circuit 23 and / or the motion information from the angular velocity sensor 25 as described above. With this, the camera-integrated digital VC
The target frame 33 and the movable mark 3 are moved by the distance R has moved.
4 is displayed on the viewfinder 18 so as to approach the screen frame 31 and the fixed mark 32.

In step S5, as shown in FIG. 3D, when the target frame 33 falls within the screen frame 31 and the movable mark 34 overlaps the fixed mark 32, the shutter is pressed and a one-frame still image (partial image) Is taken in.
The captured partial image is recorded on the recording medium 14.

In step S6, it is determined whether or not all partial images for generating a panoramic image have been shot. If it is determined in step S6 that all the partial images have been captured, the control proceeds to step S7, and if it is determined that all the partial images have not been captured, the control returns to step S3.

In step S7, one panoramic image is generated from a plurality of captured partial images and output. In this example, FIG. 3A and FIG.
A panoramic image shown in F is generated. Such processing is synthesized by, for example, automatic panorama synthesis software by a personal computer, and a panoramic image is output.

As described above, the target frame and the movable mark of the partial image to be photographed next are displayed on the viewfinder 18, so that the user moves the camera-integrated digital VCR while looking at the viewfinder 18, and then the photograph is taken. Even in the case of a plurality of partial images, a preset parameter, for example, an overlapping rate does not greatly differ. Therefore,
Even with a plurality of partial images taken by the user, a panoramic image can be easily synthesized.

In this embodiment, the description has been made using the camera-integrated digital VCR. However, the present invention can be similarly applied to a digital still camera.

In this embodiment, the target frame and the movable mark are moved so that the horizontally adjacent partial images overlap, and the camera-integrated digital VCR is moved to photograph the horizontally adjacent partial images. However, it is also possible to move the target frame and the movable mark so that the adjacent partial images in the vertical direction overlap, and move the camera-integrated digital VCR to photograph the adjacent partial images in the vertical direction. It is also possible to move the target frame and the movable mark in the horizontal direction and the vertical direction in order to capture adjacent horizontal and vertical partial images.

In this embodiment, the camera-integrated digital VCR is moved by the user, but may be fixedly moved on a tripod or the like.

In this embodiment, when the target frame 33 is accommodated in the screen frame 31 and the movable mark 34 overlaps the fixed mark 32, the user presses the shutter to capture a partial image. However, when the target frame 33 falls within the screen frame 31 and the movable mark 34 overlaps the fixed mark 32, the shutter may be automatically pressed to capture a partial image.

Further, an optical axis conversion unit, for example, an active prism type or a mirror block type may be provided before the lens system 11, and a plurality of partial images may be photographed by changing the optical axis.

In this embodiment, the motion detection circuit 23 detects a motion component of image data based on a motion vector obtained by comparing image data between frames adjacent in time series. However, a motion component may be obtained by using a motion detection process used in image compression. For example, MPEG (Moving Pic
Nature Experts Group) or H.E. The motion component may be obtained using the motion compensation used in H.261.

[0043]

According to the present invention, the position of the next partial image to be photographed is displayed on the viewfinder, so that the partial image for synthesizing the panoramic image can be very easily photographed and a plurality of photographed partial images can be obtained. Since the partial images do not greatly differ from the preset parameters, the time for synthesizing the panoramic image is reduced, and the accuracy of the synthesis is further improved.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of a camera-integrated digital VCR of the present invention.

FIG. 2 is a flowchart of an embodiment of a process for generating a panoramic image to which the present invention is applied.

FIG. 3 is a schematic diagram for explaining a panoramic image to which the present invention is applied;

FIG. 4 is a schematic diagram for explaining shooting of a partial image to which the present invention is applied;

[Explanation of symbols]

11 lens system, 12 CCD, 13 compression circuit, 14 recording medium, 15 decompression circuit, 1
7: switch circuit, 18: view finder,
Reference numeral 20: system controller, 21: operation key unit, 22: lens system drive circuit, 23: motion detection circuit, 24: mixing circuit, 25: angular velocity sensor,
26 ... Integration circuit

Claims (8)

[Claims] 1. An image signal imaging apparatus for capturing a plurality of partial images to generate a panoramic image, comprising: a display unit for displaying a captured image; and a motion detecting means for detecting a motion of a main body. A fixed mark is displayed at the center of the display screen, a movable mark is displayed at the same position as the fixed mark, and after the partial image is shot, the movable mark moves to the position of the next partial image to be shot. An image characterized in that the movable mark moves in accordance with the detection result of the movement detecting means, and the partial image to be captured next is captured when the fixed mark and the movable mark overlap each other. Signal imaging device. 2. The image signal imaging apparatus according to claim 1, wherein the motion detection means detects a motion component from an image taken and adjacent in time series. 3. The motion detection device according to claim 1, wherein the motion detection means includes an angular velocity sensor, calculates a rotation speed by integrating a signal from the angular velocity sensor, and obtains motion information according to the calculated rotation speed. An image signal imaging device characterized in that: 4. The image signal imaging apparatus according to claim 1, wherein said motion detecting means calculates a motion component using a motion detection process used in image compression. 5. The display device according to claim 1, further comprising: displaying a target frame having substantially the same size as that of the display unit on the display unit; And the target frame is moved according to the detection result of the motion detecting means, and the next partial image is photographed when the target frame fits on the display unit. Image signal imaging device. 6. The image signal imaging apparatus according to claim 5, wherein at least a part of the target frame is displayed on a screen of the display unit when the target frame moves. 7. The image signal imaging apparatus according to claim 1, wherein the plurality of captured partial images are combined to generate a panoramic image. 8. An image signal imaging method for capturing a plurality of partial images to generate a panoramic image, wherein a fixed mark is displayed at the center of a screen of a display unit for displaying the captured image, and the same position as the fixed mark is displayed. A movable mark is displayed on the image, and after the partial image is captured, the movable mark moves to the position of the next partial image to be captured, a movement of the main body is detected, and the movable mark is detected in accordance with the detected movement. Is moved, and when the fixed mark and the movable mark overlap each other, the next partial image to be photographed is photographed.