JP2012165405A - Imaging apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and a program capable of acquiring an image of wide field angle easily.SOLUTION: In an imaging apparatus, a CPU 11 allows a CCD (image pickup device) 5 to shoot a plurality of images continuously in a prescribed cycle (time-interval) while a user moves the shooting direction of a digital camera 1 so as to cover the desired scene, allows an image processing part 8 to synthesize reduced images of the plurality of shot images so that some parts are overlapping, and displays the reduced and synthesized images in an image display part 15 in real time. At that time, the CPU 11 stores an original image (high picture quality image) of the reduced image used for the synthesis, and when the shooting is finished, the CPU 11 synthesizes the original image stored so that some parts are overlapping, and finally generates an image of wide field angle which would not be able to obtain with one shot.

Description

  The present invention relates to an imaging apparatus such as a digital camera or a mobile phone having an imaging function, and a program.

  In a digital camera, a mobile phone having an imaging function, and the like, the limit of the imaging angle of view depends on hardware specifications provided in the apparatus main body, such as the focal length of the lens and the size of the imaging element.

  For such a problem of the limit of the imaging angle of view, a conversion lens for wide-angle imaging or the like is mounted in front of an existing lens in the imaging apparatus (for example, Patent Documents 1, 2, 3), a plurality of lenses are provided in advance, and this is dealt with by switching the lenses according to the imaging purpose (for example, see Patent Document 4).

JP 2004-191897 A Japanese Patent Laying-Open No. 2005-027142 JP 2005-057548 A JP 2007-081473 A

  However, in the above-described prior art, in order to perform wide-angle imaging, it is necessary to attach a conversion lens for wide-angle imaging one by one or switch the lens according to the imaging purpose. There was a problem. In addition, there is a problem that it is difficult to obtain a wide-angle image desired by the photographer even if these conversion lenses for wide-angle imaging and switchable lenses are used.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus and a program that can easily obtain a wide-angle image.

  In order to achieve the above object, the invention described in claim 1 is based on display means, imaging means for imaging at a first angle of view, imaging control means for performing imaging a plurality of times by the imaging means, and the imaging control means. Generating means for generating a composite image that reproduces an image captured at a second angle of view wider than the first angle of view by combining a plurality of images obtained by a plurality of times of imaging; and the generating means And a display control means for displaying the composite image generated by the display means on the display means.

  Further, as a preferred aspect, for example, as in claim 2, in the imaging apparatus according to claim 1, the imaging direction of the captured image changes greatly between imaging operations in a plurality of imaging operations by the imaging control unit. A first determination unit that determines whether or not the image capturing direction has changed significantly by the first determination unit; and a first notification unit that notifies the fact when the first determination unit determines that the imaging direction has changed significantly. It is characterized by.

  Further, as a preferred aspect, for example, as in claim 3, in the imaging apparatus according to claim 2, the first notification means determines that the imaging direction has changed significantly by the first determination means. In such a case, a message to that effect is displayed on the display means.

  Further, as a preferable aspect, for example, as in claim 4, in the imaging device according to claim 2 or 3, based on a change in center coordinates of images sequentially captured by a plurality of times of imaging by the previous imaging control means. Then, it is determined whether or not the imaging direction has changed significantly.

  Further, as a preferred aspect, for example, as in claim 5, in the imaging device according to claim 2 or 3, the first determination unit is an image sequentially captured by a plurality of times of imaging by the imaging control unit. In the meantime, it is determined whether or not the imaging direction has largely changed based on whether or not the same area as the image captured immediately before is a predetermined area or less.

  Further, as a preferable aspect, for example, as in claim 6, in the imaging apparatus according to any one of claims 1 to 5, the image captured between the imaging operations of a plurality of times of imaging by the imaging control unit is recorded. When it is determined by the second determination means that the imaging direction is outside the processing area of the image synthesis by the generation means, and when the second determination means determines that the imaging direction is outside the processing area. And a second informing means for informing.

  As a preferred mode, for example, as in claim 7, in the imaging device according to claim 6, when the second notification means is determined to be outside the processing region by the second determination means. In addition, the display means displays that effect.

  Further, as a preferred aspect, for example, as in claim 8, in the imaging apparatus according to claim 6 or 7, the second determination unit is an image sequentially captured by a plurality of times of imaging by the imaging control unit. In the meantime, based on whether or not there is the same region as the image captured immediately before, it is determined whether or not it is outside the processing region.

  Further, as a preferable aspect, for example, as in claim 9, in the imaging device according to any one of claims 1 to 8, the generation unit includes a plurality of images sequentially captured by a plurality of times of imaging by the imaging control unit. Are sequentially combined to generate a composite image having a second field angle wider than the first field angle, and the display control unit is configured to sequentially combine the images of the second field angle. A synthesized image is displayed on the display means.

  Further, as a preferred aspect, for example, as in claim 10, in the imaging device according to any one of claims 1 to 9, the image synthesized immediately before from the synthesized image generated by the generating means is centered. Trimming means for trimming a predetermined area is further provided, and the display control means displays an image of the predetermined area centered on the immediately previous synthesized image trimmed by the trimming means on the entire screen of the display means. It is characterized by that.

  Further, as a preferred aspect, for example, as in claim 11, in the imaging apparatus according to any one of claims 1 to 10, the display control means further includes direction information indicating a direction to be imaged by the imaging means. Is displayed on the display means.

  Further, as a preferable aspect, for example, as in claim 12, in the imaging apparatus according to claim 11, the display control unit has already generated a composite image having a second angle of view by the generation unit. Position information indicating the position of the captured image and the position of the image that has not yet been captured is displayed on the display means.

  Further, as a preferred aspect, for example, as in claim 13, in the imaging apparatus according to any one of claims 1 to 12, the imaging control means performs imaging a plurality of times continuously in a predetermined cycle. The imaging means is controlled.

  In order to achieve the above object, the invention described in claim 14 is an image pickup control in which an image pickup device including an image pickup means for picking up an image at a first angle of view and a display means is picked up by the image pickup means. Generating a composite image that reproduces an image picked up at a second angle of view wider than the first angle of view by combining a plurality of images picked up by a plurality of times of shooting by the function and the image pickup control function A program for executing a display control function for displaying a composite image of the function and the second angle of view on the display means.

  According to the present invention, there is an advantage that an image with a wide angle of view can be easily obtained.

It is a block diagram which shows the structure of the digital camera by 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the wide angle imaging mode of the digital camera 1 by this 1st Embodiment. It is a conceptual diagram which shows the relationship between the field angle of a lens, and the synthesized image obtained by wide-angle imaging mode in the digital camera 1 by this 1st Embodiment. It is a schematic diagram for demonstrating user operation in the wide angle imaging mode of the digital camera 1 by this 1st Embodiment. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment. It is a schematic diagram for demonstrating the image synthesis | combination in the wide angle imaging mode of the digital camera by this 1st Embodiment. It is a flowchart for demonstrating operation | movement of the digital camera by this 2nd Embodiment. It is a schematic diagram which shows the example of a display of the image display part of the digital camera by this 2nd Embodiment. It is a schematic diagram which shows the operation | movement of the digital camera by this 2nd Embodiment, and the example of a display of an image display part. It is a flowchart for demonstrating operation | movement of the digital camera by this 3rd Embodiment. It is a schematic diagram which shows the operation | movement of the digital camera by this 3rd Embodiment, and the example of a display of an image display part. It is a flowchart for demonstrating operation | movement of the digital camera by this 4th Embodiment. It is a schematic diagram which shows the operation | movement of the digital camera by the 4th Embodiment, and the example of a display of an image display part. It is a schematic diagram which shows the modification of this 4th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First embodiment A-1. Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment of the present invention. In the figure, a digital camera 1 includes an imaging lens 2, a lens driving unit 3, a diaphragm / shutter 4, a CCD 5, a TG (Timing Generator) 6, a unit circuit 7, an image processing unit 8, a CPU 11, a DRAM 12, a memory 13, and a flash memory 14. , An image display unit 15, a key input unit 16, a card I / F 17, and a memory card 18.

  The imaging lens 2 includes a focus lens, a zoom lens, and the like, and a lens driving unit 3 is connected thereto. The lens driving unit 3 includes a motor for driving the focus lens and the zoom lens constituting the imaging lens 2 in the optical axis direction, a focus motor driver for driving the focus motor and the zoom motor according to a control signal from the CPU 11, and a zoom motor driver. It is configured.

  The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 11. The diaphragm 4 controls the amount of light that enters from the imaging lens 2. The CCD (imaging device) 5 converts the light of the subject projected through the imaging lens 2 and the diaphragm 4 into an electrical signal and outputs it to the unit circuit 7 as an imaging signal. The CCD 5 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The unit circuit 7 includes a CDS (Correlated Double Sampling) circuit that holds the imaging signal output from the CCD 5 by correlated double sampling, an AGC (Automatic Gain Control) circuit that performs automatic gain adjustment of the imaging signal after the sampling, It is composed of an A / D converter that converts an analog imaging signal after automatic gain adjustment into a digital signal. The imaging signal of the CCD 5 is sent to the image processing unit 8 as a digital signal through the unit circuit 7. The unit circuit 7 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The image processing unit 8 performs image processing (pixel interpolation processing, γ correction, luminance color difference signal generation, white balance processing, exposure correction processing, etc.) of image data sent from the unit circuit 7, and compression / decompression of image data ( For example, JPEG format, M-JPEG format, or MPEG format compression / expansion processing, processing of combining a plurality of captured images, and the like are performed. The image processing unit 8 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The CPU 11 is a one-chip microcomputer that controls each part of the digital camera 1. In particular, in the first embodiment, the CPU 11 continuously shoots a plurality of images at a predetermined cycle (time interval), and the plurality of captured images are partially overlapped (for example, using an α blend). ) The respective components are controlled so as to generate a single composite image that is synthesized and captured at a wide angle. Details of the image composition will be described later.

  The DRAM 12 is used as a buffer memory for temporarily storing image data sent to the CPU 11 after being imaged by the CCD 5 and also as a working memory for the CPU 11. The memory 13 stores a program necessary for controlling each part of the digital camera 1 by the CPU 11 and data necessary for controlling each part, and the CPU 11 performs processing according to this program. The flash memory 14 and the memory card 18 are recording media for storing image data captured by the CCD 5.

  The image display unit 15 includes a color LCD and its driving circuit, and displays the subject imaged by the CCD 5 as a through image when in the imaging standby state, and when the recorded image is reproduced, the storage flash memory 14, The recorded image read from the card 23 and expanded is displayed. In the first embodiment, a composite image obtained by sequentially combining a plurality of images continuously shot in the wide-angle imaging mode is displayed. The key input unit 16 includes a plurality of operation keys such as a shutter SW, a zoom SW, a mode key, a SET key, and a cross key, and outputs an operation signal corresponding to the user's key operation to the CPU 11. A memory card 18 is detachably attached to the card I / F 17 through a card slot (not shown) of the digital camera 1 main body.

  FIG. 2 is a conceptual diagram for explaining the wide-angle imaging mode of the digital camera 1 according to the first embodiment. For example, assume that the digital camera 1 captures a scene as shown in FIG. The scenery to be imaged has a wider angle of view than the angle of view S of the imaging system of the digital camera 1. Therefore, it is not possible to capture all desired scenes with a single imaging.

  Therefore, in the first embodiment, while the user moves the imaging direction of the digital camera 1 so as to cover a desired scene, a predetermined period (time) for a predetermined time or a predetermined number of images is taken. Provided is a wide-angle imaging mode in which a plurality of images are continuously captured at intervals, and a plurality of captured images are combined so that a part of them is overlapped, so that a wide-angle image can be easily obtained.

  In the following description, in order to clarify the imaging range, the imaging angle of view, and the like, the scenery in FIG. 2 is described as a diagram as shown in FIG. In FIG. 3, the angle of view S1 is the size (angle of view) of the image that is finally generated, and the outside thereof is outside the region that is not left as the final image even if it is captured.

  In the first embodiment, an image writing array is secured on the memory (DRAM 12). For convenience, this is referred to as a canvas in the first embodiment. In the first embodiment, the first captured image continuously shot in the wide-angle imaging mode is set as a reference image (corresponding to an image having an angle of view S in FIG. 3), and the vertical and horizontal dimensions of the reference image are, for example, twice as large. (Image pickup area S1 in FIG. 3), the reference image is pasted to the center of the canvas. Note that the size of the canvas may be other than twice the height and width. Then, the plurality of captured images are aligned and combined so that the reference image partially overlaps, and overwritten on the canvas. As a positioning method, for example, a technique such as block matching can be considered. In addition, in the case of overwriting, a method of performing projection conversion or the like and superimposing using a method such as α blending is conceivable.

  FIG. 4 is a schematic diagram for explaining a user operation in the wide-angle imaging mode of the digital camera 1 according to the first embodiment. The user moves the digital camera 1 so as to draw a circle as indicated by an arrow with respect to a desired scene, for example, in a state where the shutter SW is pressed (half pressed → full pressed) at the center. However, it is difficult for the user to know how to move the digital camera 1 or whether the required image has been reliably obtained.

  Therefore, in the first embodiment, when the user presses the shutter SW (half-press → full-press), as described above, the predetermined period (time interval) is maintained for a predetermined time or a predetermined number of images. ), Each time a plurality of images are captured, a reduced image (low resolution) is generated in real time, and the reference image (or composite image) is partially overlapped with the composite image. The image is displayed on the image display unit 15. At this time, the original image (high quality image) of the reduced image used for the synthesis is stored.

  Then, when imaging for a predetermined time or a predetermined number of images is completed, a part of the original image (high-quality image) is stored using the stored original image (high-quality image) in the same manner as the synthesis performed using the reduced image. The images are combined so that they overlap, and finally, a wide-angle image that cannot be obtained by one-time imaging is generated. During continuous shooting, the reduced image obtained by combining the images is displayed on the image display unit 15, so that the user can easily confirm in which direction the digital camera should be directed.

A-2. Operation of First Embodiment Next, the operation of the first embodiment described above will be described.
FIG. 5 is a flowchart for explaining the operation of the digital camera according to the first embodiment. FIGS. 6A and 6B are schematic diagrams for explaining image composition in the wide-angle imaging mode of the digital camera according to the first embodiment.

  First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S10). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S10. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S12), and it is determined whether or not the shutter SW is fully pressed (step S14). If the shutter SW is not fully pressed, steps S10 and S12 are repeated.

  On the other hand, when the shutter switch SW is fully pressed, the captured image is captured, a reduction (thinning) process is executed, and a reduced image is generated (step S16). Next, the image position for superimposition is calculated using the reduced image (step S18). The calculation of the image position for superimposition is, for example, calculating the center position (coordinates) of the reduced image and, if there is already a reference image (or composite image), the reduced image and reference image (or composite image) of the current frame. This means that the position of the reduced image of the current frame is calculated in the canvas and the like so as to partially overlap with the image. Next, based on the center position of the reduced image and the position in the canvas, it is determined whether or not the center position of the reduced image is in the processing area (in the canvas) (step S20).

  If the center position of the reduced image is within the processing area, the captured image (high resolution) captured is saved as an effective image (step S22), and the reduced image is overwritten on the blank portion that is an unacquired portion ( Step S24). That is, when the center position of the reduced image of the current frame is within the processing region, the reduced image of the current frame and the reference image (or the synthesized image) are combined so that they overlap, and the canvas 40 is overwritten. (In the case of the first captured image, the center portion of the canvas 40 is overwritten as a reference image). In the example shown in FIG. 6A, since the center position of the reduced image 31 of the current frame is within the processing area 40, the reduced image 31 of the current frame and the reference image 30 are combined so that they partially overlap, The canvas 40 is overwritten. Then, the composite image 32 is displayed on the image display unit 15 (step S26).

  Next, it is determined whether or not all necessary images have been acquired (for example, whether or not images for a predetermined time or a predetermined number of images have been acquired) (step S28). If all the necessary images have not been acquired, the process returns to step S16, and the same processing is repeated for the captured image of the next frame. As a result, whenever the image is captured, if the center position of the captured image is within the processing region, the image is sequentially combined with the reference image (or the combined image), and the combined image is displayed on the image display unit 15 each time. Become.

  On the other hand, when the center position of the reduced image of the current frame is not within the processing region, the process returns to step S16, and the same processing is repeated for the next captured image. For example, as shown in FIG. 6B, when the center position of the reduced image 31 of the current frame is not within the processing area 40, image synthesis is not performed.

  Once all the necessary images have been acquired, the saved effective image, which is the original image of the reduced image used for the composition, is aligned so that part of it overlaps, as in the composition performed using the reduced image. Finally, a wide angle image as shown in FIG. 2 is generated (step S30).

  According to the first embodiment described above, a reduced image obtained by combining images is displayed on the image display unit 15 in real time each time images are taken during continuous shooting. Alternatively, it is possible to easily recognize the direction that has already been imaged. As a result, the user can know where the digital camera should be pointed next, so that an image with a wide angle of view can be obtained easily and efficiently.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
In the second embodiment, when the outside of the processing area (outside the canvas) is imaged, or when the digital camera moves too quickly (changes in the imaging direction), the user is notified of that fact. Thus, it is characterized in that the speed at which the digital camera should be moved and in which direction the digital camera should be pointed are notified. The configuration of the digital camera 1 is the same as that shown in FIG.

  FIG. 7 is a flowchart for explaining the operation of the digital camera 1 according to the second embodiment. 8 and 9 are schematic diagrams showing display examples of the image display unit 15 of the digital camera 1 according to the second embodiment. First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S30). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S30. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S32), and it is determined whether the shutter SW is fully pressed (step S34). If the shutter SW is not fully pressed, steps S30 and S32 are repeated.

  On the other hand, when the shutter switch SW is fully pressed, the captured image is captured, a reduction (thinning) process is executed, and a reduced image is generated (step S36). Next, the image position for superimposition is calculated using the reduced image (step S38). The calculation of the image position for superimposition is, for example, calculating the center position (coordinates) of the reduced image and, if there is already a reference image (or composite image), the reduced image and reference image (or composite image) of the current frame. Position the image so that it partially overlaps the image, and calculate the position of the reduced image of the current frame in the canvas, the distance from the center position (coordinates) of the image (reduced image) that was captured immediately before, etc. Means. Next, based on the center position of the reduced image and the position in the canvas, it is determined whether or not the center position of the reduced image is in the processing area (in the canvas) (step S40).

  If the center position of the reduced image is not within the processing area, an area excess mark is displayed on the image display unit 15 (step S42). For example, as shown in FIG. 8A, when the center of the reduced image 31 of the current frame is outside the processing area canvas 40, as shown in FIG. An arrow 50 indicating the direction opposite to the moved direction is displayed on the image display unit 15 as an area excess mark so as to return to the area. As a result, the user can know that the imaging direction of the digital camera 1 has gone up too much, and therefore, by moving the digital camera 1 in the direction of the arrow 50, the angle of view to be captured can be returned to the processing area. It becomes. Thereafter, the process returns to step S36, and the above-described processing is performed on the captured image of the next frame. That is, in this case, image composition is not performed.

  On the other hand, when the center position of the reduced image is within the processing region, it is determined whether or not the distance between the center position of the reduced image of the previous frame and the center position of the reduced image of the current frame is smaller than a predetermined threshold ( Step S44). If the distance is equal to or greater than the predetermined threshold, an overspeed mark is displayed on the image display unit 15 (step S46). Thereafter, the process returns to step S36, and the above-described processing is performed on the captured image of the next frame. That is, in this case, image composition is not performed.

  In the present invention, alignment is realized by a method such as block matching. However, if the moving speed of the digital camera 1 is high, two images for alignment (a reduced image of the previous frame and a reduced image of the current frame) are displayed. Among them, the area of the same image is reduced, and the alignment is inaccurate. Therefore, it is necessary to suppress the moving speed of the digital camera 1 below a certain fixed speed. Therefore, the user is explicitly notified of the moving speed of the digital camera 1 (change speed in the imaging direction).

  For example, as shown in FIG. 9A, when the distance between the center position of the reduced image 31a of the previous frame and the center position of the reduced image 31b of the current frame is equal to or greater than a predetermined threshold value, Since the overlapping area becomes small and there is a high possibility that the alignment will be inaccurate, an overspeed mark 60 is displayed on the image display unit 15 as shown in FIG.

  As the overspeed mark 60, as shown in FIG. 9C, a tachometer-shaped mark 61 that changes the position of the meter needle and the color of the meter according to the moving distance per unit time of the image. Or, a mark 62 that changes the arc-shaped area and the color of the arc according to the size of the moving distance per unit time of the image can be considered. Alternatively, it may be a bar graph in which the length and color are changed according to the size of the moving distance per unit time of the image (not shown).

  On the other hand, if the reduced image of the current frame is within the processing area and the moving distance is smaller than the predetermined threshold, sufficiently accurate alignment can be performed for combining the images. (High resolution) is stored as an effective image (step S48), and the reduced image is overwritten on the blank portion which is an unacquired portion (step S50). That is, when the center position of the reduced image of the current frame is within the processing region, the reduced image of the current frame and the reference image (or the synthesized image) are combined so that they overlap, and the canvas 40 is overwritten. (In the case of the first captured image, the center portion of the canvas 40 is overwritten as a reference image). Then, the composite image is displayed on the image display unit 15 (step S52).

  Next, it is determined whether or not all necessary images have been acquired (for example, whether or not images for a predetermined time or a predetermined number of images have been acquired) (step S54). If all the necessary images have not been acquired, the process returns to step S36, and the same processing is repeated for the captured image of the next frame. As a result, whenever the image is captured, if the center position of the captured image is within the processing region, the image is sequentially combined with the reference image (or the combined image), and the combined image is displayed on the image display unit 15 each time. Become.

  Once all the necessary images have been acquired, the saved effective image, which is the original image of the reduced image used for the composition, is aligned so that part of it overlaps, as in the composition performed using the reduced image. Finally, a wide angle image as shown in FIG. 2 is generated (step S56).

  According to the second embodiment described above, when the user captures an image outside the processing area in real time every time an image is captured, or when the movement of the digital camera 1 (change in the imaging direction) is too fast, etc. In addition, since the user is notified of this, the user can know how fast the digital camera 1 should be moved and in which direction it should be directed. A corner image can be obtained.

C. Third Embodiment Next, a third embodiment of the present invention will be described.
In the third embodiment, captured images (reduced images) that are continuously shot are not simply combined and displayed, but from a combined image so that the reduced image of the current frame is positioned at the center of the image display unit 15. A predetermined range is trimmed and displayed on the image display unit 15. The configuration of the digital camera 1 is the same as that shown in FIG.

  FIG. 10 is a flowchart for explaining the operation of the digital camera according to the third embodiment. First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S60). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S60. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S62), and it is determined whether or not the shutter SW is fully pressed (step S64). If the shutter SW is not fully pressed, steps S60 and S62 are repeated.

  On the other hand, when the shutter switch SW is fully pressed, a captured image is captured, a reduction (thinning) process is executed, and a reduced image is generated (step S66). Next, the image position for superimposition is calculated using the reduced image (step S68). The calculation of the image position for superimposition is, for example, calculating the center position (coordinates) of the reduced image and, if there is already a reference image (or composite image), the reduced image and reference image (or composite image) of the current frame. This means that the position of the reduced image of the current frame is calculated in the canvas and the like so as to partially overlap with the image. Next, based on the center position of the reduced image and the position in the canvas, it is determined whether or not the center position of the reduced image is in the processing area (in the canvas) (step S70). If not in the processing area, the process returns to step S66, and the above-described processing is performed on the captured image of the next frame. In this case, image composition is not performed.

  On the other hand, if the center position of the reduced image is within the processing region, the captured image (high resolution) captured is saved as an effective image (step S72), and the reduced image is overwritten on the blank portion that is an unacquired portion ( Step S74). In other words, if the center position of the reduced image is within the processing region, the reduced image of the current frame and the reference image (or composite image) are aligned and combined so that they partially overlap, and the canvas is overwritten. (In the case of the first captured image, the reference image is overwritten on the center of the canvas).

  Next, the composite image is trimmed with the display size of the image display unit 15 around the reduced image of the current frame (step S76), and the trimmed composite image is displayed on the image display unit 15 (step S78). Next, it is determined whether or not all necessary images have been acquired (for example, whether or not images for a predetermined time or a predetermined number of images have been acquired) (step S80).

  If all the necessary images have not been acquired, the process returns to step S66, and the same processing is repeated for the next captured image. As a result, every time an image is captured, if the center position of the captured image is within the processing region, the image display unit 15 is sequentially combined with the reference image (or the combined image), and each time, the reduced image of the current frame is the center. The composite image is trimmed with the display size of, and the trimmed composite image is displayed on the image display unit 15.

  Once all the necessary images have been acquired, the saved effective image, which is the original image of the reduced image used for the composition, is aligned so that part of it overlaps, as in the composition performed using the reduced image. Finally, a wide angle image as shown in FIG. 2 is generated (step S82).

  FIG. 11 is a schematic diagram illustrating an operation of the digital camera according to the third embodiment and a display example of the image display unit. First, after the reference image 30 that is the first captured image is acquired, the second captured image (reduced image) 31 is acquired. If the center position of the second reduced image 31 is at the image acquisition position, the second reduced image 31 is synthesized by superimposing a part of the first reference image 30. Next, the composite image 32 is trimmed with the display size of the image display unit 15 around the reduced image 31 of the current frame, and the trimmed composite image 32 a is displayed on the image display unit 15.

  According to the third embodiment described above, every time captured images are sequentially combined, the reduced image of the current frame is displayed on the image display unit 15 so as to be positioned at the center of the image display unit 15. That is, a user who is taking an image while looking at the image display unit 15 can view an image centered on a direction in which the digital camera is facing in real time, and in order to fill a blank portion that has not yet been taken, Next, it is possible to intuitively and easily recognize in which direction the digital camera should be directed. As a result, an image with a wide angle of view can be obtained easily and efficiently.

  In the above-described series of processing, the reference image moves on the screen of the image display unit 15. Therefore, in order to let the user know which part of the composite image is the reference image, a frame of a predetermined color is used. You may make it surround with.

D. Fourth Embodiment Next, a fourth embodiment of the present invention will be described.
In the fourth embodiment, during continuous shooting, how to move the digital camera while clearly indicating to the user the direction that has not yet been captured or the direction that has already been captured, or which It is characterized by presenting a guide for guiding whether the digital camera should be directed in the direction. The configuration of the digital camera 1 is the same as that shown in FIG.

  FIG. 12 is a flowchart for explaining the operation of the digital camera according to the fourth embodiment. FIGS. 13A to 13E are schematic diagrams illustrating an operation of the digital camera according to the fourth embodiment and a display example of the image display unit.

  First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S90). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S90. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S92), and it is determined whether or not the shutter SW is fully pressed (step S94). If the shutter switch is not fully pressed, steps S90 and S92 are repeated.

  On the other hand, when the shutter SW is fully pressed, for example, as shown in FIG. 13A, a spiral guide 70 is displayed on the lower right of the image display unit 15 (step S96). Next, the captured image is captured, and a reduction (thinning-out) process is executed to generate a reduced image (step S98). Next, the image position for superimposition is calculated using the reduced image (step S100). The calculation of the image position for superimposition is, for example, calculating the center position (coordinates) of the reduced image and, if there is already a reference image (or composite image), the reduced image and reference image (or composite image) of the current frame. This means that the position of the reduced image of the current frame is calculated in the canvas and the like so as to partially overlap with the image.

  Next, based on the center position of the reduced image and the position in the canvas, it is determined whether or not the center position of the reduced image is in the processing area (in the canvas) (step S102). If not in the processing area, the process returns to step S96, and the above-described processing is performed on the captured image of the next frame. In this case, image composition is not performed.

  On the other hand, when the center position of the reduced image is within the processing region, the captured image (high resolution) captured is saved as an effective image (step S104), and the reduced image is overwritten on the blank portion that is an unacquired portion ( Step S106). In other words, if the center position of the reduced image is within the processing area, the reduced image of the current frame and the reference image (or composite image) are aligned and combined so that a part of the image overlaps, and the canvas 40 is overwritten. (In the case of the first captured image, it is overwritten on the center of the canvas 40 as a reference image). Next, the composite image is displayed on the image display unit 15 (step S108).

  Next, in order to show that the combined portion of the reduced image of the current frame has been acquired, a part of the color of the guide 70 corresponding to the combined portion (in the illustrated example, it is indicated by the difference in line type) ) And the guide 70 displayed on the image display unit 15 is updated (step S110). For example, FIG. 13B shows a display example of the image display unit 15 after the first reference image 30 is captured. At this time, in the guide 70, the color corresponding to the position of the first reference image from the spiral center changes (in the illustrated example, the line type of the guide 70 changes). The user may look at the guide 70 and move the digital camera 1 along the spiral.

  After the synthesis of the second reduced image, as shown in FIG. 13C, the color from the spiral center to the location corresponding to the position of the second captured image (shown in the figure) in the guide 70. In the example, the line type of the guide 70 is changed. The user may look at the guide 70 and move the digital camera 1 along the spiral.

  Next, it is determined whether or not all necessary images have been acquired (for example, whether or not images for a predetermined time or a predetermined number of images have been acquired) (step S112). If all the necessary images have not been acquired, the process returns to step S96, and the same processing is repeated for the next captured image. As a result, each time the reduced images are sequentially combined, a part of the color of the guide 70 corresponding to the position of the reduced image to be combined changes. That is, every time reduced images are sequentially combined, the state shown in FIG. 13 (d) is obtained. Finally, as shown in FIG. 13 (e), the entire guide 70 changes, and the combined image of the entire screen is displayed. It will be shown that it has been acquired.

  Then, when all the necessary images are obtained, the saved effective image that is the original image of the reduced image used for the synthesis is synthesized so as to partially overlap, as in the synthesis performed using the reduced image. Specifically, an image with a wide angle of view as shown in FIG. 2 is generated (step S114).

  FIGS. 14A to 14C are schematic views showing modifications of the fourth embodiment. In the fourth embodiment described above, the guide 70 has a spiral shape and is displayed at the right corner of the image display unit 15. However, the present invention is not limited to this. For example, as shown in FIG. May be displayed on the entire screen of the image display unit 15 so as to overlap with the composite image, or as shown in FIG. 14B, the guide 72 may be Z-shaped meandering from the upper left of the screen. In the case of FIG. 14B, the first reference image is the upper left corner that is the starting point of the guide 72. Further, as shown in FIG. 14C, a circular guide 73 is arranged at a position to be imaged, and when the image is captured, the color is changed to determine whether the image has been captured or not captured. You may do it. That is, the guide 70 may have any shape as long as it has a shape corresponding to how many times the angle of view (area) of the composite image is generated with respect to the reference image.

  According to the fourth embodiment described above, how should the digital camera be moved while clearly indicating to the user the direction that has not yet been captured or the direction that has already been captured during continuous shooting? Or a guide for guiding in which direction the digital camera should be directed is displayed, so that the user who is taking an image while looking at the image display unit 15 then moves the digital camera. It is possible to intuitively and easily recognize which should be performed or in which direction the digital camera should be directed. As a result, an image with a wide angle of view can be obtained easily and efficiently.

  In addition, the above-described first to fourth embodiments can be combined, for example, the region excess mark and the overspeed mark display processing according to the second embodiment, the trimming processing according to the third embodiment, Or / and a guide display according to the fourth embodiment may be added. In addition, for example, in the above-described first to fourth embodiments, an acceleration sensor that detects the movement of the digital camera is further provided, and when the captured images sequentially obtained by continuous shooting are overlapped, The image position for superimposition may be calculated in consideration of the movement detected by the acceleration sensor.

  In the first to fourth embodiments described above, the digital camera has been described as the imaging device. However, the present invention is not limited to this, and any electronic device having an imaging function can be applied to, for example, a mobile phone. is there.

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Imaging lens 3 Lens drive part 4 Shutter combined shutter 5 CCD
6 TG
7 Unit Circuit 8 Image Processing Unit 11 CPU
12 DRAM
13 Memory 14 Flash memory 15 Image display section 16 Key input section 17 Card I / F
18 Memory card

Claims (14)

Display means;
Imaging means for imaging at a first angle of view;
Imaging control means for performing imaging a plurality of times by the imaging means;
Generating means for generating a composite image that reproduces an image picked up at a second angle of view wider than the first angle of view by combining a plurality of images obtained by a plurality of times of image pickup by the image pickup control means. When,
An image pickup apparatus comprising: a display control unit configured to display a composite image generated by the generation unit on the display unit. First determination means for determining whether or not the imaging direction of the captured image has changed greatly between imaging operations in a plurality of imaging operations by the imaging control means;
2. The imaging apparatus according to claim 1, further comprising: a first notification unit that notifies the fact when the first determination unit determines that the imaging direction has changed significantly. The first notification means includes
The imaging apparatus according to claim 2, wherein when the first determination unit determines that the imaging direction has changed significantly, the display unit displays the fact. The first determination means includes
4. The method according to claim 2, wherein it is determined whether or not the imaging direction has largely changed based on a change in center coordinates of images sequentially captured by a plurality of times of imaging by the imaging control unit. 5. Imaging device. The first determination means includes
Whether or not the imaging direction has changed greatly based on whether or not the same area as the image captured immediately before is less than or equal to a predetermined area between images sequentially captured by the imaging control means multiple times. The imaging device according to claim 2, wherein the imaging device is determined. A second determination unit that determines whether or not an imaging direction of a captured image is outside an image synthesis processing region by the generation unit between a plurality of imaging operations by the imaging control unit;
6. The imaging according to claim 1, further comprising: a second notifying unit that notifies that when the second determining unit determines that it is outside the processing region. apparatus. The second notification means includes:
The imaging apparatus according to claim 6, wherein when it is determined that the second determination unit is outside the processing region, the display unit displays the fact. The second determination means includes
Determining whether or not the processing area is outside based on whether or not there is the same area as the image captured immediately before among the images sequentially captured by the imaging control means a plurality of times. The imaging apparatus according to claim 6, wherein: The generating means sequentially generates a composite image having a second angle of view wider than the first angle of view by sequentially combining a plurality of images sequentially captured by the imaging control means a plurality of times;
The display control means includes
9. The imaging apparatus according to claim 1, wherein a composite image of the second angle of view sequentially synthesized by the generation unit is displayed on the display unit. Trimming means for trimming a predetermined area centered on the image synthesized immediately before from the synthesized image generated by the generating means;
The display control means includes
The imaging according to any one of claims 1 to 9, wherein an image of a predetermined area centering on the immediately previous synthesized image trimmed by the trimming means is displayed on the entire screen of the display means. apparatus. The display control means includes
11. The imaging apparatus according to claim 1, further comprising: direction information indicating a direction to be imaged by the imaging unit is displayed on the display unit. The display control means includes
When generating the composite image having the second angle of view by the generation unit, the position information indicating the position of the image that has already been captured and the position of the image that has not yet been captured is displayed on the display unit. The imaging apparatus according to claim 11, wherein the imaging apparatus is characterized. The imaging control means includes
The imaging apparatus according to any one of claims 1 to 12, wherein the imaging unit is controlled to perform imaging a plurality of times continuously at a predetermined cycle. In a computer of an imaging apparatus comprising an imaging means for imaging at a first angle of view and a display means
An imaging control function for performing imaging a plurality of times by the imaging means;
A generation function for generating a composite image in which a plurality of images picked up by a plurality of times of image pickup by the image pickup control function are combined to reproduce an image picked up at a second angle of view wider than the first angle of view;
A program for executing a display control function for displaying a composite image of the second angle of view on the display means.

Images