JP2001057652A - Image input device and image input method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a digital still camera that can easily photograph an object with a simple means. SOLUTION: The image input device is provided with an image pickup element 24 that photographs an object. A frame memory 7 stores image data of a global image photographed by an image pickup element 24 so that the entire area of the object is contained within an image frame. A display device 8 displays the global image. The frame memory 7 stores image data by a plurality of division image photographed by the image pickup element 24 in a way that each part of the object is in duplicate with each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image input device and an image input method for photographing a plurality of still images, such as a digital still camera, and more particularly, to photographing a part of a subject so that the parts partially overlap each other. The present invention relates to a technique for generating a composite image of an entire region of a subject by laminating a plurality of images.

[0002]

2. Description of the Related Art With the rapid progress of computer networks, business modes have become diversified, and it has become necessary to obtain important information on the spot in various situations. Accordingly, there is an increasing demand for simple and highly accurate input of a planar document or photograph using a portable input device. Therefore, conventionally, using an image input device such as a digital still camera, image information to be collected is promptly input / transferred and utilized for work and entertainment.

However, since the number of pixels of an image pickup device such as a CCD (charge coupled device) used in the above image input device is generally finite, an image of required quality cannot be obtained due to insufficient resolution. There was a defect.

On the other hand, in order to compensate for the lack of resolution of the image input device, a technique of creating a high resolution and high definition image by devising image processing has been developed. For example, a conventional technique of obtaining an image having a sufficient resolution over a wide range without restriction on the number of pixels by combining images obtained by photographing a plurality of the same subject and combining a single panoramic image. There is. Specifically, the depth of the subject is measured, and if the subject can be regarded as spherical (distant scenery, etc.) or planar, a part of the subject is divided using a geometric correction formula such as affine transformation or projective transformation. This is a technique for attaching captured images, and an overview of these techniques is described in, for example, "Computer Vision-Technical Review and Future Perspective-/ Takashi Matsuyama et al. / New Technology Communications". In order to combine images that have been divided and photographed using such a technique, it is necessary to photograph the subject in a divided manner such that a part of the images overlaps each other.

[0005] However, it is very difficult for a user to take a divided image of a subject while holding the image input device in his hand while appropriately setting the size of the area where the divided images overlap each other.

In view of this point, in Japanese Patent Application Laid-Open No. H5-161050, an image that has been input and an image that is currently being input are thinned out and reduced, and displayed on adjacent portions of a display unit. There has been disclosed a digital electronic still camera capable of confirming whether a current shooting position is appropriate when obtaining a panoramic image.

In Japanese Patent Application Laid-Open No. 9-266561,
In a camera that creates continuous images, a part of the previously captured image is displayed according to the moving direction of the camera, and it is displayed side by side with the currently input image (for example, if the camera rotates clockwise, The right end portion of the obtained image is cut out by a predetermined area and is projected on the left end of the display unit provided on the back of the camera, and the image of the current input is displayed immediately next to the right side) and the two images are connected. A digital still camera is disclosed in which, when a shutter is pressed in a state where the shutter is pressed, a combined continuous image is stored in a panoramic memory.

Further, Japanese Patent Application Laid-Open No. 9-261520 discloses that the position and orientation of a photographing unit are measured, and based on the measurement result, an overlap between a previously captured previous image and a currently photographed image is obtained. A motion amount between each photographed image is obtained based on a technique of calculating a lap amount and moving image data sequentially output from the photographing unit, and a motion amount between the previous image is calculated by adding this motion amount. Thus, there is disclosed an invention for calculating an amount of overlap between a previous image and a currently captured image.

In addition, in Japanese Patent Application Laid-Open No. Hei 10-186551, a partial area of a divided image (previous image) already stored and a divided image (current image) newly received by a camera unit are disclosed.
Are stored as divided images, a predetermined process such as comparing the two divided images and inputting the divided image as a divided image in an appropriate direction when the two images coincide with each other is performed. Is disclosed.

[0010]

However, the above-mentioned Japanese Patent Application Laid-Open No.
In the technology disclosed in Japanese Patent No. 161050, it is difficult for a user to see the displayed low-resolution thinned-out image and check the size of the connection or overlap between the images. This is remarkable when it is composed of periodic and fine textures.

In Japanese Patent Application Laid-Open No. Hei 9-261520, it is necessary to describe the position and orientation of the photographing unit by using six degrees of freedom information of positions on three orthogonal axes and angles around the three axes. In order to accurately obtain information of all six degrees of freedom, various detectors are required, and there is a problem that the apparatus becomes large-scale. Also, when calculating the overlap amount from the motion amount in the moving image, it is difficult to detect the motion amount in the moving image unless the camera is slowly moved from the time of capturing the previous image, so the user must carefully operate the camera. Must be restricted.

In the technique disclosed in Japanese Patent Application Laid-Open No. H10-186551, a careful camera operation is required to match two divided images while the user holds the camera, and a restriction is imposed on the shooting operation. receive.

An object of the present invention is to provide an image input apparatus and an image input method which can easily take a picture by simple means.

Another object of the present invention is to provide an image input apparatus and an image input method which allow the user to intuitively know the shooting range of a divided image to be shot next.

Another object of the present invention is to provide an image input device and an image input method which can easily confirm whether or not an overlap area of an appropriate size exists between both divided images.

It is another object of the present invention to provide an image input apparatus and an image input method capable of quickly confirming an area where a divided image needs to be taken in the future.

Another object of the present invention is to provide an image input apparatus and an image input method capable of quickly confirming whether or not a currently captured divided image is appropriate.

Another object of the present invention is to provide an image input apparatus and an image input method capable of quickly confirming whether or not the size of an overlapping area of a divided image is appropriate.

It is another object of the present invention to provide an image input apparatus and an image input method that can appropriately set the size of a divided image to be shot and prevent unnecessary divided images from being shot.

[0020]

According to a first aspect of the present invention, there is provided an image pickup device and image data of a global image which is an image photographed by the image pickup device so that the entire area of a subject falls within a screen frame. A first memory, a display, and display means for displaying the general image on the display;
A second memory for storing image data of a plurality of divided images, which are a plurality of images taken by the image sensor so that a part of the subject partially overlaps each other; and a plurality of divided images and the general image. An image input apparatus comprising: an image processing unit that performs predetermined image processing using at least the former of the images.

Therefore, a general image is photographed in advance, the image is displayed on a display, and a plurality of divided images are photographed while quickly confirming the area of the subject corresponding to each divided image while viewing the displayed image. Therefore, it is possible to easily capture a divided image by simple means.

According to a second aspect of the present invention, in the image input device according to the first aspect, the display means is configured to display the global image stored in the first memory and the image currently being captured by the image sensor. The divided images are displayed on the display simultaneously or in a time-division manner.

Therefore, the global image and the already-captured divided image can be displayed on the display simultaneously or in a time-division manner, so that the photographing range of the next divided image to be photographed can be intuitively known.

According to a third aspect of the present invention, in the image input device according to the first or second aspect, the display means is currently photographing the divided image stored in the second memory and the image sensor. Are displayed on the display simultaneously or in a time-division manner.

[0025] Therefore, the already captured divided image and the currently captured divided image can be displayed on the display simultaneously or in a time-division manner, so that there is an overlapping area of an appropriate size between the two divided images. Can be easily confirmed.

According to a fourth aspect of the present invention, in the image input device according to any one of the first to third aspects, the second image processing apparatus is provided.
First image correspondence detection means for detecting which range of the global image stored in the first memory corresponds to the divided image stored in the first memory, and wherein the display means comprises: Based on the detection result of the image correspondence detection means, the area of the divided image stored in the second memory corresponding to this image or the other area corresponding to the general image stored in the first memory Is displayed on the display.

Therefore, since the area of the divided image which has been already taken or the other area is displayed on the display with respect to the general image, the area which needs to be taken in the future can be quickly confirmed.

According to a fifth aspect of the present invention, in the image input device according to any one of the first to fourth aspects, the divided image currently being photographed by the imaging device is stored in the first memory. Second image correspondence detection means for detecting which range of the global image corresponds to the first image, and the display means stores the first image in the first memory based on a detection result of the second image correspondence detection means. With respect to the stored global image, an area of the divided image currently being captured by the image sensor corresponding to this image or an area other than the divided image is displayed on the display.

Therefore, since the area of the currently captured divided image or the other area is displayed on the display with respect to the general image, it is possible to quickly confirm whether the currently captured divided image is appropriate. .

According to a sixth aspect of the present invention, in the image input device according to any one of the first to fifth aspects, the divided image currently being photographed by the image pickup device and the second image are stored in the second memory. An overlapping area detecting unit for detecting an overlapping area with the divided image, and a first notifying unit for notifying the size of the detected overlapping area.

Therefore, it is possible to know the size of the overlapping area between the currently captured divided image and the already-captured divided image, and it is possible to quickly confirm whether or not the overlapping area between the two divided images is appropriate. be able to.

According to a seventh aspect of the present invention, in the image input apparatus according to any one of the first to fifth aspects, the divided image currently being photographed by the image pickup device and the second image are stored in the second memory. Overlapping area detecting means for detecting an overlapping area with the divided image, and, when the detected overlapping area is in a predetermined state set in advance, notifying that the image currently being captured is inappropriate. Second notification means.

Therefore, it is possible to know that the overlapping area between the currently captured divided image and the already-captured divided image is in a predetermined inappropriate state. It is possible to prevent the photographing of a complicated divided image.

According to an eighth aspect of the present invention, there is provided a step of photographing a general image, which is an image in which the entire area of a subject falls within a screen frame, by an image sensor, and storing image data of the photographed image in a first memory. And a step of displaying this image on a display, and a step of photographing a divided image which is an image of a part of the subject and a plurality of images photographed by an image sensor so that a part thereof overlaps with each other. Storing the image data of the plurality of divided images in the second memory, and performing a predetermined image processing using at least the former among the plurality of divided images and the global image. This is an image input method.

Therefore, a general image is photographed in advance, the image is displayed on a display, and a plurality of divided images are photographed while quickly checking the area of the subject corresponding to each divided image while viewing the displayed image. Therefore, it is possible to easily capture a divided image by simple means.

According to a ninth aspect of the present invention, in the image input method according to the eighth aspect, the display step includes the step of displaying the global image stored in the first memory and the image currently being captured by the image sensor. The divided images are displayed on the display simultaneously or in a time-division manner.

Therefore, the global image and the already-captured divided images can be displayed on the display simultaneously or in a time-division manner, so that the photographing range of the next divided image to be photographed can be intuitively known.

According to a tenth aspect of the present invention, in the image input method according to the eighth or ninth aspect, the display step includes:
The divided image stored in the second memory and the divided image currently being photographed by the image sensor are displayed on the display simultaneously or in a time-division manner.

Therefore, the already captured divided image and the currently captured divided image can be displayed simultaneously or in a time-division manner on the display, so that there is an overlapping area of an appropriate size between the two divided images. Can be easily confirmed.

The invention according to claim 11 is the invention according to claims 8 to 1.
0, wherein the divided image stored in the second memory is the first image.
Detecting which range of the general image stored in the memory corresponds to the general image. The displaying step includes the step of detecting the general image stored in the first memory based on the detection result. In the image, the area of the divided image stored in the second memory corresponding to the image or an area other than the divided image is displayed on the display.

Therefore, since the area of the divided image which has been already taken or the other area is displayed on the display with respect to the general image, an area which needs to be taken in the future can be quickly confirmed.

The twelfth aspect of the present invention is the eighth aspect of the present invention.
The image input method according to any one of claims 1 to 3, wherein a step of detecting which range of the global image stored in the first memory corresponds to the divided image currently being captured by an image sensor. The display step includes, based on the detection result, the area of the divided image currently being captured by the image sensor corresponding to the general image stored in the first memory or the other area. Are displayed on the display in the same manner.

Therefore, since the area of the currently captured divided image or the other area is displayed on the display with respect to the general image, it is possible to quickly confirm whether the currently captured divided image is appropriate. .

The thirteenth aspect of the present invention is the eighth aspect of the present invention.
2. In the image input method according to any one of (2), a step of detecting an overlapping area between the global image currently being captured by an image sensor and the divided image stored in the second memory; Notifying the size of the overlapping area.

Therefore, it is possible to know the size of the overlapping area between the currently captured divided image and the already captured divided image, and it is possible to quickly check whether or not the overlapping area between the two divided images is appropriate. be able to.

The invention according to claim 14 is the invention according to claims 8 to 1.
2. In the image input method according to any one of (2), a step of detecting an overlapping area between the global image currently being captured by an image sensor and the divided image stored in the second memory; Notifying that the currently captured image is inappropriate when the overlapping area is in a predetermined state set in advance.

Therefore, it is possible to know that the overlapping area between the currently captured divided image and the already captured divided image is in a predetermined inappropriate state, so that the size of the divided image to be captured is made appropriate and unnecessary. It is possible to prevent the photographing of a complicated divided image.

[0048]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments of the present invention in which the image input device of the present invention is applied to a digital still camera will be described below. As shown in FIG. 1, the digital still camera 1 is first used to shoot an image so that the entire area of the object falls within a screen frame, and then to take a part of the object so as to be divided so as to partially overlap each other. This will be described using an example of performing the operation. Then, the image taken earlier is a global image,
An image obtained by dividing and photographing later is called a divided image.

[First Embodiment of the Invention] FIG. 2 is a block diagram showing an electrical connection of a digital still camera 1 according to a first embodiment of the present invention.

As shown in FIG. 1, the digital still camera 1 is provided with a photographing means 2, a signal processing means 3 for performing predetermined processing on an image signal obtained by the photographing means 2, and an output of the signal processing means 3. Memory control means 4 to be operated, and main control means 5 for centrally controlling each part of the digital still camera 1
An interface (hereinafter abbreviated as I / F) 6, a frame memory 7 for storing image signals according to instructions from the memory control unit 4, a display 8 for displaying image signals via the I / F 6, An internal storage unit 9 used for temporarily storing data and the like necessary for signal processing, and an external storage unit 10 for reading and writing various signals such as image signals via the I / F 6. It has. In addition, a photographing mode setting unit 11 for switching a photographing mode, and a photographing instruction unit 1 for instructing a user to photograph a general image of a subject first and then photograph a divided image.
2 is provided.

The details of each section will be described below. Photographing means 2
Is a lens 21, an aperture 22, a shutter 23, an image sensor 2
4 and a preprocessing means 25. As the imaging device 24, for example, a charge coupled device (CCD) is used. The preprocessing means 25 includes a preamplifier and an AG.
An analog signal processing circuit such as C (auto gain control) and an A / D converter are provided. The analog video signal output from the image sensor 24 is subjected to pre-processing such as amplification and clamping, and then the analog signal is processed. Converts video signals to digital video signals.

The signal processing means 3 is constituted by a digital signal processor (DSP processor) or the like.
Various image processing such as color separation, white balance adjustment, and γ correction are performed on the digital video signal obtained by the photographing unit 2. The memory control means 4 stores the image signal thus processed in the frame memory 7 and reads out the image signal stored in the frame memory 7. Main control means 5
Is constituted by a microcomputer or the like.

The frame memory 7 can store at least two images, and is composed of a VRAM, an SRAM, a DRAM and the like. The recording of the image signal read from the frame memory 7 is performed by performing signal processing such as image compression on the image signal in the signal processing unit 3 and then via the I / F 6 to the internal storage unit 9 or the external storage unit 10. This is done by saving. First and second memories are configured by the frame memory 7 and the like.

The internal storage means 9 comprises a VRAM, an SRAM, a DRAM and the like, like the frame memory 7. The external storage means 10 can use an IC memory card, a magneto-optical disk, or the like. However, the image signal can be directly transmitted to a remote recording medium via a network using a modem card or an ISDN card. Good. vice versa,
Reading of the image signal recorded in the internal storage means 9 is performed according to I
This is performed by transmitting an image signal to the signal processing unit 3 via / F6 and performing image expansion in the signal processing unit 3. On the other hand, the external storage means 10 and the frame memory 7
The display of the image signal read out from the device is performed by performing signal processing such as D / A conversion and amplification on the image signal in the signal processing unit 3 and then transmitting the image signal to the display 8 via the I / F 6. Will be The display 8 is composed of, for example, a liquid crystal display device provided in a housing of the digital still camera 1.

The photographing mode setting means 11 is constituted by a push button switch, a slide switch and the like, and is used for a normal photographing mode for photographing a snapshot by a user's operation of the switch, and for creating a high-resolution image later. Therefore, it is possible to manually switch between the divided shooting modes for split shooting of the same subject.

The photographing instructing means 12 is activated only when the divided photographing mode is selected, and when the divided photographing mode is set, the first photograph of the subject is photographed for the user and the second photograph is taken. Thereafter, an instruction is given to take a divided image of the subject.

In response to this instruction, the user sets the optical system of the photographing means 2 to the wide-angle side for a distant subject in which the subject image is hardly deformed by the parallel movement of the digital still camera 1, and A high-resolution divided image may be shot by shooting a general image so that the entire image falls within the screen frame and setting the optical system of the shooting unit 2 on the telephoto side. For a subject at a short distance where the subject image is deformed by the parallel movement of the digital still camera 1, the entire image of the subject is set by setting the optical system of the photographing unit 2 to the wide angle side or moving the photographing unit 2 away from the subject. May be taken by setting the optical system of the photographing means 2 on the telephoto side or by moving the photographing means 2 close to the subject so as to capture a high-resolution divided image.

The photographing instruction is to display a text indicating the photographing procedure on the screen of the display 8 as an overlay, to explicitly notify the photographing procedure by voice, or to use photographing means when it is necessary to photograph a general image. Various modes such as an implicit one in which the lens of the photographing unit 2 is forcibly shifted to the telephoto side when it is necessary to forcibly shift the second lens to the wide-angle side and to shoot a divided image. Can be taken.

The general image and the divided images taken in the above procedure are displayed on the display 8 simultaneously or in a time-division manner. The photographed divided image and the photographed divided image are also displayed simultaneously or time-divisionally. In order to display images simultaneously or in a time-division manner, the screen of the display 8 is divided into two parts, and a general image or a divided image being captured is captured in one of the divided areas and the other area is captured. It may take a form such as displaying already-divided images. Further, in order to display the divided images in a time-division manner, the general image or the photographed divided image and the photographed divided image may be alternately displayed at regular time intervals.

FIG. 3 is a flowchart showing a series of procedures for photographing. Here, it is assumed that the shooting mode is set in advance to the division shooting mode. As shown in FIG.
First, a subject image is photographed over a wide area, and the obtained image is stored as a global image (step S1). Next, the subject is photographed in a photographing range narrower than the general image, and the obtained image is stored as a divided image (step S4). If there is a divided image that has already been photographed before this processing (Y in step S2), the photographed divided image and the currently photographed divided image are displayed on the display 8 simultaneously or in a time-division manner ( Step S3). After step S4,
The general image and the divided image are displayed simultaneously or in a time-division manner (step S5). Unless the split shooting mode is turned off, shooting of the split image is continued (Y in step S6),
The operation from step S2 to step S5 is repeated. When the divided photographing mode is switched to OFF, a series of images from the global image to the last photographed divided image is associated and stored as a set of images photographing the same subject (step S7), and the processing is performed. finish. Step S3, S
5 realizes the display means of the present invention.

According to the digital still camera 1 described above, a general image is photographed in advance, the image is displayed on the display 8, and while viewing the displayed image, the area of the subject corresponding to each divided image can be quickly determined. Since a plurality of divided images can be photographed while checking, it is possible to easily photograph the divided images by simple means.

Further, since the general image and the already-captured divided image can be displayed on the display 8 simultaneously or in a time-division manner, the photographing range of the next divided image to be photographed can be intuitively known.

Since the already captured divided image and the currently captured divided image can be displayed simultaneously or in a time-division manner on the display, it is determined whether or not there is an overlapping area of an appropriate size between the two divided images. It can be easily confirmed.

Note that a plurality of photographed divided images are combined to generate a composite image of the entire region of the subject. Specific means for that are described in, for example, "Computer Vision-Technical Review and Future Perspective-/ Takashi Matsuyama et al. / New Technology Communications".

[Second Embodiment of the Invention] FIG. 4 is a block diagram showing an electrical connection of a digital still camera 1 according to a second embodiment of the present invention.

In FIG. 4, members having the same reference numerals as those in FIG.
Since the members are the same as those of the digital still camera 1 according to the first embodiment of the invention, detailed description will be omitted. The digital still camera 1 includes an image correspondence detection unit 13 for detecting which region the currently captured divided image corresponds to in the previously captured general image, and changes each of the general image and the divided image. Image normalizing means 14 for normalizing the global image and the size of the divided image
A photographed area notifying unit 15 for notifying an already photographed object area is provided.

As shown in FIG. 5, the image correspondence detecting means 13 detects the correspondence between the currently photographed or already photographed divided images and the photographing range in the general image. Here, a method of calculating the correlation between the density of the divided image and the global image and detecting the correspondence will be described. First, as shown in FIG. 6, the divided image is scaled to the size of “(2N + 1) (2P + 1)”, and its density pattern is extracted to create a correlation window. Subsequently, a portion of the global image that substantially matches the shading pattern of the correlation window is detected by a correlation operation. For example,
The cross-correlation value S between the scaled divided image of “(2N + 1) (2P + 1)” and the point (x, y) in the global image
Is calculated by the following equation.

[0068]

(Equation 1)

Therefore, by finding a point where the maximum value of the cross-correlation value S is equal to or greater than a predetermined threshold value, the coordinates (x, y) in the global image corresponding to the center coordinates (x _c , y _c ) of the divided image are obtained. ) Is required, so that the correspondence between the divided image and the global image can be detected. The detected correspondence is, as shown in FIG. 5, “the divided image“ a ”is a global image where the image coordinates (x ₁₁ , y ₁₁ ) are the upper left vertex, (x ₁₂ ,
₁₂ ) The divided image b corresponding to the rectangular area having the lower right vertex
Is output in the format of "..." And recorded in the internal storage means 9 and the external storage means 10 as necessary.

In general, it is known that the density matching based on the correlation operation represented by the equation (1) is likely to cause an error unless the size of the subject image in two target images is substantially the same. Before the image correspondence detecting means 13 detects the correspondence between the divided image and the global image, the image normalizing means 14 determines the size of the subject image shown in the divided image,
For normalizing the size of the subject image shown in the global image, for example, as shown in FIG. 7, a photographing parameter detecting unit 26, a subject distance detecting unit 27, an enlargement coefficient detecting unit 28, and an image scaling unit 29.

The photographing parameter detecting means 26 detects the focal length of the photographing means 2 and is realized, for example, by reading the lens position of the photographing means 2 with an encoder. The subject distance detecting means 27 is provided by the photographing means 2
To detect the distance from to the subject, for example,
The distance can be obtained from the focal position of the photographing means 2. The enlargement coefficient detecting means 28 is provided with a focal length and subject distance detecting means 2 output from the photographing parameter detecting means 26.
Based on the distance to the subject output in step 7, the enlargement coefficients of the global image and the divided image are calculated.

This operation will be described with reference to FIG. As shown in FIG. 8A, the general image is obtained by moving a subject having a length L in the horizontal direction from a focal length f to the optical center o of the photographing means 2 from the subject.
It is assumed that the image was taken in a state where the distance to D was D. 8B, it is assumed that the same subject is photographed in a state where the focal length is f ′ and the distance from the subject to the optical center o of the photographing means 2 is D ″ ′ as shown in FIG. 8B. At this time, when the optical system of the photographing unit 2 is represented by a central projection model as shown in FIG. 8, the length l of the subject image in the general image and the length l ′ of the subject image in the divided image are as follows. Can be calculated as follows.

[0073]

(Equation 2)

Accordingly, in order to make the size of the global image and that of the subject image captured in the divided image uniform, the magnification of the global image is k, the magnification of the divided image is k ′, and the ratio between k and k ′ is as follows. What is necessary is just to determine like a formula.

[0075]

(Equation 3)

According to the formula (4), the size of the subject image in these images can be normalized based on the focal length and the subject distance at the time of capturing the general image and the divided image. The image scaling means 29 performs scaling processing by scaling the global image and the divided image, respectively, using the enlargement coefficients k and k 'calculated by the equation (4). The scaled image is sent to the image correspondence detection unit 13, and the image correspondence detection unit 13 detects the correspondence between the scaled divided image and the scaled global image in the procedure described above. .

The photographed area notifying means 15 receives the output of the image correspondence detecting means 13 and detects an area (already photographed area) in the general image that has already been divided and photographed. For example, the photographed area can be detected by performing an OR operation on the area corresponding to each divided image transmitted from the image correspondence detection unit 13 in the general image. Further, the photographed area notifying means 15 sends a signal to the main control means 5 to display the detected photographed area on the display 8, and in response to this, the main control means 5 transmits the signal to the main controller 5 through the I / F 6 as shown in FIG. As shown in the figure, on the display 8, rectangular frame lines 30 and 31 indicating the photographed area are displayed in an overlay manner on the global image.

As shown in FIG. 10, an unphotographed area notifying means 16 may be provided in place of the photographed area notifying means 15. The unphotographed area notification means 16 receives the output of the image correspondence detection means 13 and detects an area in the general image that has not been photographed yet (unphotographed area). For example,
The non-photographed area can be detected by performing a NOR operation on an area corresponding to each divided image transmitted from the image correspondence detection unit 13 in the general image. Further, the non-photographed area notifying means 16 sends a signal to the main control means 5 to display the detected non-photographed area on the display 8, and the main control means 5 receives the signal via the I / F 6, as shown in FIG. As shown, on the display 8, for example, the frame lines 30, 3
1 is masked, and only the unphotographed area is displayed brightly.

The user uses the photographed area notifying means 15 or the non-photographed area notifying means 16 to check the area of the object already photographed or the area of the object to be divided and photographed in the future, and continues the divided photographing. At this time, the subject image input by the photographing means 2 is projected on the display 8, and at the same time, the image correspondence detecting means 13 detects the correspondence between the currently captured divided image and the general image. When the user instructs to display the shooting range of the currently captured divided image (the means for instructing this is not shown), FIG.
As shown in (1), it is convenient to superimpose and display a frame line 32 indicating the shooting range of the currently captured divided image on the general image on the display 8. That is, by switching between the screen shown in FIG. 12 and the screen shown in FIG. 9 or FIG. 11 on the display 8, the user can intuitively determine whether or not the shooting range of the currently captured divided image is appropriate. You can judge.

FIG. 13 is a flowchart showing a series of photographing procedures. Here, it is assumed that the shooting mode is set in advance to the split shooting mode. As shown in FIG. 13, first, a subject image is photographed in a wide range, and the obtained image is stored as a global image (step S11). Next, the subject is photographed in a photographing range narrower than the general image, and the obtained image is stored as a divided image (step S1).
2). Subsequently, based on the equation (4), using the focal length and the subject distance at the time of capturing the divided image and the general image, scaling is performed so that the size of the subject image in each image becomes substantially the same (step S13), the correspondence of the photographing range between the divided image and the general image is detected (step S14). When the correspondence is detected, the area of the subject already photographed or the area of the subject not yet photographed is displayed (step S15). Unless the split shooting mode is turned off, the shooting of the split image is continued (step S1).
6) The operation from step S2 to step S5 is repeated. When the split shooting mode is switched to OFF, a series of images from the global image to the last shot
The same subject is stored as a set of captured images (step S17), and the process ends. Step S14 implements first and second image correspondence detection means, and step S15 implements display means.

According to the digital still camera 1 described above, the area of the divided image already taken or the other area is superimposed on the global image and displayed on the display, so that the area which needs to be taken in the future can be quickly identified. You can check.

Further, since the area of the currently captured divided image or the other area is superimposed on the global image and displayed on the display 8, it is possible to quickly confirm whether or not the currently captured divided image is appropriate. Can be.

[Third Embodiment of the Invention] FIG. 14 is a block diagram showing an electrical connection of a digital still camera 1 according to a third embodiment of the present invention.

In FIG. 14, members having the same reference numerals as those in FIGS. 4 and 10 are the same members as those of the digital still camera 1 according to the second embodiment of the present invention, and a detailed description thereof will be omitted. The digital still camera 1 includes an overlapping area detecting unit 17 for detecting the size and position of an area where a previously captured divided image and a currently captured divided image overlap.
And a warning means 18 for issuing a warning when it is determined that the currently captured divided image is inappropriate.

The overlapping area detecting means 17 receives the information indicating the correspondence between each divided image and the global image from the image correspondence detecting means 13 and, based on the information, the divided image which is currently photographed and which is first photographed. An area where the divided images overlap each other is detected. The operation will be described with reference to FIG. In FIG. 15A, it is assumed that the divided image a and the divided image b are already stored, and the divided image c is currently being photographed. As shown in FIG. 15B, the image correspondence detection means 13 outputs a region corresponding to the three divided images in the general image in the form of image coordinates in the general image. As a result, an area where the divided image c overlaps with the divided image a and the divided image b based on the image coordinates in the global image can be easily detected. Further, the overlapped area detection unit 17 further divides the overlapped area into divided areas. Image c
Is converted to image coordinates based on. As shown in FIG. 16, when the coordinate system of the global image is an xy coordinate system and the coordinate system based on the divided image c is a uv coordinate system, the image coordinates (x, y) in the xy coordinate system and the image coordinates in the uv coordinate system
The relationship with (u, v) is represented by the following conversion equation.

[0086]

(Equation 4)

In order to calculate the three variables a to c shown in equation (5), it is necessary that two or more sets of the relationship between the coordinates in the global image and the coordinates in the divided image c are known. The coordinates of the four corners are known. Therefore, it is possible to detect an overlapping area between the currently captured divided image and the already captured divided image based on the currently captured divided image. For example, as shown in FIG. 17, the detected overlap region is displayed on the display 8 by displaying a rectangular frame 33 representing an already captured region or a text 34 representing the ratio of the overlap region to the size of the divided image. Are displayed by being superimposed on the currently captured divided image.

The warning means 18 receives the output of the image correspondence detection means 13 and the output of the overlapping area detection means 17, determines whether or not the currently captured divided image is appropriate, and issues a warning when the divided image is determined to be inappropriate. . For example, when the following event occurs, it is determined that “the currently captured divided image is inappropriate”. (1) The correspondence between the currently captured divided image and the global image cannot be detected. (2) The overlapping area between the currently captured divided image and a previously captured divided image is too large.

(1) In the image correspondence detecting means 13,
For example, the appearance of the subject in the divided image is significantly different from the appearance of the subject in the global image,
This occurs when the correlation value between the divided image and the global image is calculated based on Expression (1) and the correlation value does not reach the threshold value. (2) can be easily determined from the output of the above-described overlapping region detecting means 17, and for example, the size of the overlapping region between the currently captured divided image and a previously captured certain divided image is 1/1 of the image area. If it is 2 or more, it may be determined to be inappropriate. The output of the warning means 18 is transmitted to the display 8,
The user is instructed to re-take the divided image, for example, by overlaying a text indicating a warning on the screen of the display 8.

FIG. 18 is a flowchart showing a series of photographing procedures. It is assumed that the shooting mode is set in advance to the split shooting mode. First, as shown in FIG.
The subject image is photographed over a wide area, and the obtained image is stored as a global image (step S21). Next, a divided image of the subject is photographed in a photographing range narrower than the general image (step S22). Subsequently, the correspondence between the shooting range of the divided image and the general image is detected (step S23), and the size of the area where the currently shot divided image and each of the previously shot divided images overlap is determined. It is detected (step S24), and the size is displayed on the display 8 (step S25). Then, it is determined whether the currently captured image is appropriate (step S26). If it is determined that the image is inappropriate, a warning is issued to the user to retake the divided image (step S27), and if it is determined that the image is not appropriate, the warning is not issued and the photographing of the divided image is continued. Unless the split shooting mode is turned off, the shooting of the split image is continued (step S28), and the operation from step S22 to step S27 is repeated. When the divided photographing mode is switched to OFF, a series of images from the global image to the last photographed divided image is stored as a group of images photographing the same subject (step S29), and the process is terminated. . Step S24 implements the area detecting means, and step S25 implements the first notifying means.
27 implements second notification means.

According to the digital still camera 1 described above, it is possible to know the size of the overlapping area between the currently captured divided image and the already captured divided image. Can be quickly checked to see if it is appropriate.

Further, since it is possible to know that the overlapping area between the currently photographed divided image and the already photographed divided image is in a predetermined inappropriate state, the size of the divided image to be photographed is made appropriate and unnecessary. It is possible to prevent the photographing of a complicated divided image.

Needless to say, the description of each embodiment described above is merely an example of the embodiment of the present invention, and does not limit the present invention. For example, in the above description, the subject has been described as a document page, but it is needless to say that the subject may be another object such as a landscape or a person. Regarding the means for detecting the correspondence between the divided image and the global image, means other than the above description, for example, cut out the shading pattern of the small area at the four corners in the divided image as a correlation window, and use the respective correlation windows in the global image. Other means such as finding a corresponding area may be used.

In the description at the beginning, the technique of combining the images obtained by dividing and photographing a part of the subject is described as an application example. However, it goes without saying that the technique can be applied to other applications. For example, processing that requires a correspondence relationship between the positions of each divided image and the entire image of the subject (such as partial high resolution of the image), and a positional relationship between the divided images and an overlapping area between the divided images are required. It is also applicable to simple processing (such as reconstruction of a three-dimensional shape of a subject using a continuous image captured by a monocular camera).

[0095]

According to the first aspect of the present invention, a general image is taken in advance, and the image is displayed on a display.
Since a plurality of divided images can be photographed while quickly checking the area of the subject corresponding to each divided image while viewing the display image, the divided images can be easily photographed by simple means.

According to a second aspect of the present invention, in the image input device according to the first aspect, the global image and the already captured divided image can be displayed on the display simultaneously or in a time-division manner. It is possible to intuitively know the shooting range of the divided image to be obtained.

According to a third aspect of the present invention, in the image input device according to the first or second aspect, it is possible to simultaneously or time-dividely display a divided image that has already been captured and a currently captured divided image on a display. Since it is possible, it is possible to easily confirm whether or not there is an overlapping area of an appropriate size between the two divided images.

According to a fourth aspect of the present invention, in the image input device according to any one of the first to third aspects, an area of a divided image which has already been shot or an area other than the divided image is indicated with respect to a global image. Since the information is displayed on the display, it is possible to quickly confirm an area that needs to be photographed in the future.

According to a fifth aspect of the present invention, in the image input device according to any one of the first to fourth aspects, the area of the currently captured divided image or the other area is indicated for the global image. Since the image is displayed on the display, it is possible to quickly confirm whether or not the currently captured divided image is appropriate.

According to a sixth aspect of the present invention, in the image input device according to any one of the first to fifth aspects, the size of the overlapping area between the currently captured divided image and the already captured divided image is determined. Since it is possible to know, it is possible to quickly confirm whether or not the size of the overlapping area of the two divided images is appropriate.

According to a seventh aspect of the present invention, in the image input device according to any one of the first to fifth aspects, the overlapping area between the currently photographed divided image and the already photographed divided image has a predetermined area. Since it is possible to know that the image is in an appropriate state, the size of the divided image to be photographed can be made appropriate, and unnecessary divided images can be prevented from being photographed.

According to an eighth aspect of the present invention, a general image is photographed in advance, the image is displayed on a display, and while viewing the displayed image, each divided image quickly confirms the area of the subject corresponding to the divided image. Since a plurality of divided images can be taken, the divided images can be easily taken by simple means.

According to the ninth aspect of the present invention, in the image input method according to the eighth aspect, the global image and the already captured divided image can be displayed on the display simultaneously or in a time-division manner. It is possible to intuitively know the shooting range of the divided image to be obtained.

According to a tenth aspect of the present invention, in the image input method according to the eighth or ninth aspect, it is possible to simultaneously or time-divisionally display the already captured divided image and the currently captured divided image. Since it is possible, it is possible to easily confirm whether or not there is an overlapping area of an appropriate size between the two divided images.

The invention according to claim 11 is the invention according to claims 8 to 1.
In the image input method according to any one of the above, the area of the already captured divided image or the other area is shown on the display with respect to the global image, so that the area which needs to be captured in the future is quickly confirmed. can do.

The invention according to claim 12 is the invention according to claims 8 to 1.
In the image input method according to any one of the first to third aspects, the area of the currently captured divided image or the other area is displayed on the display with respect to the global image, so that the currently captured divided image is appropriate. You can quickly check whether or not.

The invention according to claim 13 is the invention according to claims 8 to 1.
In the image input method according to any one of the above two aspects, the size of the overlapping area between the currently captured divided image and the already captured divided image can be known, so that the size of the overlapping area between the two divided images can be determined. Can be quickly checked to see if it is appropriate.

The invention according to claim 14 is the invention according to claims 8 to 1
2. In the image input method according to any one of (2) and (4), it is possible to know that the overlapping area between the currently captured divided image and the already captured divided image is in a predetermined inappropriate state. The size of the image can be made appropriate so that unnecessary divided images are not shot.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram illustrating a relationship between a global image and a divided image according to each embodiment of the present invention.

FIG. 2 is a block diagram illustrating electrical connection of the digital still camera according to the first embodiment of the present invention;

FIG. 3 is a flowchart showing a series of procedures of photographing by the digital still camera.

FIG. 4 is a block diagram illustrating electrical connection of a digital still camera according to Embodiment 2 of the present invention;

FIG. 5 is an explanatory diagram of a process of detecting a correspondence relationship of each divided image currently being photographed or already photographed by the digital still camera to which photographing range corresponds to a general image.

FIG. 6 is an explanatory diagram of a process of detecting a correspondence relationship between each divided image currently being photographed or photographed by the digital still camera and which photographing range corresponds to a general image.

FIG. 7 is a block diagram showing a configuration of an image normalizing means of the digital still camera.

FIG. 8 is an explanatory diagram illustrating a process of calculating an enlargement coefficient of a global image and a divided image by an enlargement coefficient detection unit of the digital still camera.

FIG. 9 is a plan view showing a display example of a general image showing a photographed area on a display of the digital still camera.

FIG. 10 is a block diagram showing electrical connections of a modified example of the digital still camera according to the second embodiment of the present invention.

FIG. 11 is a plan view showing a display example of a general image showing an unphotographed area on the display of the digital still camera.

FIG. 12 is a plan view showing a display example of a general image showing a shooting range of a currently shot divided image on a display of the digital still camera.

FIG. 13 is a flowchart showing a series of procedures of photographing by the digital still camera.

FIG. 14 is a block diagram showing electrical connection of a digital still camera according to Embodiment 3 of the present invention.

FIG. 15 is an explanatory diagram illustrating a process of detecting an area where a divided image currently captured by the digital still camera and each of the previously captured divided images overlap each other.

FIG. 16 is an explanatory diagram illustrating a relationship between a global image and a divided image captured by the digital still camera.

FIG. 17 shows a rectangular frame representing an area of a divided image already captured by the digital still camera and a text representing a ratio of an overlapping area to the size of the divided image, on the top of the currently captured divided image. FIG. 10 is a plan view showing a display example in which a display is superimposed on the display.

FIG. 18 is a flowchart showing a series of procedures of photographing by the digital still camera.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image input device 7 1st and 2nd memory 8 Display 24 Image sensor

 ────────────────────────────────────────────────── ─── Continued on the front page F term (reference) 5B047 AA30 BB01 BC15 CA14 CA23 5C022 AA13 AC01 AC18 AC42 AC69 5C023 AA11 AA31 AA37 AA38 BA11 CA03 DA04 5C053 FA08 FA14 FA27 JA21 KA03 LA01 LA06

Claims (14)

[Claims] An image sensor; a first memory for storing image data of a general image which is an image photographed by the image sensor so that the entire area of the subject falls within a screen frame; a display; A display unit for displaying the general image; and a second memory for storing image data of a divided image, which is a plurality of images captured by the image sensor so that a part of the subject is partially overlapped with each other. An image input device comprising: 2. The display means displays the global image stored in the first memory and the divided image currently being captured by the image sensor simultaneously or in a time-division manner on the display. Item 2. The image input device according to Item 1. 3. The display means displays the divided image stored in the second memory and the divided image currently being photographed by the image sensor simultaneously or in a time-division manner on the display. Item 3. The image input device according to item 1 or 2. 4. A first image correspondence detecting means for detecting which range of the global image stored in the first memory corresponds to the divided image stored in the second memory, The display means is configured to, based on a detection result of the first image correspondence detection means, generate the divided image stored in the second memory corresponding to the general image stored in the first memory. 4. An image area or other area is shown and displayed on the display.
The image input device according to any one of the above. 5. A second image correspondence detecting means for detecting which range of the global image stored in the first memory corresponds to the divided image currently being photographed by the image sensor, A display unit configured to display, based on a detection result of the second image correspondence detection unit, the divided image currently being captured by the imaging device corresponding to the general image stored in the first memory by the image sensor corresponding to the image; An area or another area is indicated and displayed on the display.
5. The image input device according to any one of 4. 6. An overlap area detecting means for detecting an overlap area between the divided image currently being photographed by the image sensor and the divided image stored in the second memory, and a size of the detected overlap area. The image input device according to any one of claims 1 to 5, further comprising first notification means for notifying the user. 7. An overlap area detecting means for detecting an overlap area between the divided image currently being photographed by the image sensor and the divided image stored in the second memory, and the detected overlap area is set in advance. And a second notifying unit for notifying that the image currently being photographed is inappropriate when in the predetermined state.
The image input device according to any one of the above. 8. A step of photographing a general image, which is an image in which the entire area of the subject falls within a screen frame, by an image sensor; a step of storing image data of the photographed image in a first memory; Displaying the image on a display; and photographing a plurality of divided images, each of which is an image of a part of the subject, the plurality of images being taken by an image sensor so that a part thereof overlaps each other. And storing the image data in a second memory. 9. The display step includes displaying the general image stored in the first memory and the divided image currently being captured by the image sensor on the display simultaneously or in a time-division manner. The image input method according to claim 8. 10. The display step displays the divided image stored in the second memory and the divided image currently being captured by the image sensor on the display simultaneously or in a time-division manner. Item 10. The image input method according to Item 8 or 9. 11. A step of detecting which range of the global image stored in the first memory corresponds to the divided image stored in the second memory, the display comprising: The step is based on the detection result, and the region of the divided image stored in the second memory corresponding to the general image stored in the first memory or another region is stored in the second memory based on the detection result. 11. The image input method according to claim 8, wherein the image is displayed on the display. 12. A step of detecting which range of the global image stored in the first memory corresponds to the divided image currently being captured by an image sensor, and the display step includes: Based on the detection result, the general image stored in the first memory is displayed on the display by indicating an area of the divided image currently being captured by the image sensor corresponding to the image or another area. The image input method according to any one of claims 8 to 11, wherein the image is input. 13. A step of detecting an overlapping area between the general image currently being captured by the image sensor and the divided image stored in the second memory, and a step of notifying a size of the detected overlapping area. The image input method according to any one of claims 8 to 12, comprising: 14. A step of detecting an overlapping area between the global image currently being photographed by an image sensor and the divided image stored in the second memory, and a step of determining the detected overlapping area in advance. The method according to any one of claims 8 to 12, further comprising a step of notifying that the image currently being photographed is inappropriate when in the state of (1).