JP5892211B2 - Imaging apparatus and program - Google Patents

Description

  The present invention relates to an imaging apparatus such as a digital camera or a mobile phone having an imaging function, and a program.

  In a digital camera, a mobile phone having an imaging function, and the like, the limit of the imaging angle of view depends on hardware specifications provided in the apparatus main body, such as the focal length of the lens and the size of the imaging element.

  Therefore, when acquiring a wide-angle image that exceeds the hardware specifications, such as panoramic shooting, there is a technique for generating a wide-angle image by continuously shooting while moving the imaging device in a certain direction and combining the obtained images. (For example, refer to Patent Document 1).

JP 2002-027312 A

  In the case of Patent Document 1 described above, it is effective in the case of widening the angle of view in one direction, but when generating a wide-angle image that widens in two directions, it is possible to grasp which direction should be taken during continuous shooting. There is a problem that it is difficult and it is difficult to obtain a desired image.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus and a program that can easily and efficiently obtain an image necessary for generating a wide-angle image.

In order to achieve the above object, the invention according to claim 1 is an image generation unit, a display unit, and a generation unit that generates an image having a wider angle than the captured image based on images sequentially captured by the image capture unit. An imaging range necessary for generating a wide-angle image by the generation unit , a preview image as a through image displayed in the imaging range and sequentially captured by the imaging unit, and the imaging First display control means for displaying on the display means a mark indicating one of each of the plurality of imaging directions of the means; the preview image displayed on the display means by the first display control means; determination means for determining the positional relationship between the serial landmarks, on the basis of the determination result by said determination means, and second display control means for displaying on said display means by changing the display mode of the previous SL landmarks, An imaging apparatus, characterized in that it comprises.

In order to achieve the above object, the invention described in claim 6 is directed to generating means for generating a wider-angle image than the captured image based on images sequentially captured by the imaging means. an imaging range required to produce an image of the preview image as a through image corresponding to the sequentially captured images by the imaging unit, shows the one of each of a plurality of imaging directions of the imaging means first display control means for displaying on the display means and the eye mark, determination means for determining a positional relationship between the preview image and the previous SL marker displayed on said display means by said first display control means, said based on the determination result by the determination means, before Symbol program characterized by change the table示態like mark to function as a second display control means for displaying on said display means That.

  According to the present invention, there is an advantage that an image necessary for generating a wide-angle image can be obtained easily and efficiently.

It is a block diagram which shows the structure of the digital camera by 1st Embodiment of this invention. It is a conceptual diagram for demonstrating the wide angle imaging mode of the digital camera 1 by this 1st Embodiment. It is a conceptual diagram which shows the relationship between the field angle of a lens, and the synthesized image obtained by wide-angle imaging mode in the digital camera 1 by this 1st Embodiment. It is a schematic diagram for demonstrating user operation in the wide angle imaging mode of the digital camera 1 by this 1st Embodiment. 4 is a flowchart for explaining the operation of the digital camera according to the first embodiment. It is a schematic diagram which shows the operation | movement of the digital camera by this 1st Embodiment, and the example of a display of an image display part. It is a flowchart for demonstrating operation | movement of the digital camera by this 2nd Embodiment. It is a schematic diagram which shows the operation | movement of the digital camera by this 2nd Embodiment, and the example of a display of an image display part.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First embodiment A-1. Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment of the present invention. In the figure, a digital camera 1 includes an imaging lens 2, a lens driving unit 3, a diaphragm / shutter 4, a CCD 5, a TG (Timing Generator) 6, a unit circuit 7, an image processing unit 8, a CPU 11, a DRAM 12, a memory 13, and a flash memory 14. , An image display unit 15, a key input unit 16, a card I / F 17, and a memory card 18.

  The imaging lens 2 includes a focus lens, a zoom lens, and the like, and a lens driving unit 3 is connected thereto. The lens driving unit 3 includes a motor for driving the focus lens and the zoom lens constituting the imaging lens 2 in the optical axis direction, a focus motor driver for driving the focus motor and the zoom motor according to a control signal from the CPU 11, and a zoom motor driver. It is configured.

  The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 11. The diaphragm 4 controls the amount of light that enters from the imaging lens 2. The CCD (imaging device) 5 converts the light of the subject projected through the imaging lens 2 and the diaphragm 4 into an electrical signal and outputs it to the unit circuit 7 as an imaging signal. The CCD 5 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The unit circuit 7 includes a CDS (Correlated Double Sampling) circuit that holds the imaging signal output from the CCD 5 by correlated double sampling, an AGC (Automatic Gain Control) circuit that performs automatic gain adjustment of the imaging signal after the sampling, It is composed of an A / D converter that converts an analog imaging signal after automatic gain adjustment into a digital signal. The imaging signal of the CCD 5 is sent to the image processing unit 8 as a digital signal through the unit circuit 7. The unit circuit 7 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The image processing unit 8 performs image processing (pixel interpolation processing, γ correction, luminance color difference signal generation, white balance processing, exposure correction processing, etc.) of image data sent from the unit circuit 7, and compression / decompression of image data ( For example, JPEG format, M-JPEG format, or MPEG format compression / expansion processing, processing of combining a plurality of captured images, and the like are performed. The image processing unit 8 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The CPU 11 is a one-chip microcomputer that controls each part of the digital camera 1. In particular, in the first embodiment, the CPU 11 continuously shoots a plurality of images at a predetermined cycle (time interval), and the plurality of captured images are partially overlapped (for example, using an α blend). ) The respective components are controlled so as to generate a single composite image that is synthesized and captured at a wide angle. Details of the image composition will be described later.

  The DRAM 12 is used as a buffer memory for temporarily storing image data sent to the CPU 11 after being imaged by the CCD 5 and also as a working memory for the CPU 11. The memory 13 stores a program necessary for controlling each part of the digital camera 1 by the CPU 11 and data necessary for controlling each part, and the CPU 11 performs processing according to this program. The flash memory 14 and the memory card 18 are recording media for storing image data captured by the CCD 5.

  The image display unit 15 includes a color LCD and its driving circuit, and displays the subject imaged by the CCD 5 as a through image when in the imaging standby state, and when reproducing the recorded image, the flash memory 14 or the memory card 23. The recorded image read out and decompressed is displayed. In the first embodiment, in the wide-angle imaging mode, a composite image obtained by sequentially synthesizing a plurality of continuously shot images, a mark for guiding the imaging direction to the user, and the like are displayed. The key input unit 16 includes a plurality of operation keys such as a shutter SW, a zoom SW, a mode key, a SET key, and a cross key, and outputs an operation signal corresponding to the user's key operation to the CPU 11. A memory card 18 is detachably attached to the card I / F 17 through a card slot (not shown) of the digital camera 1 main body.

  FIG. 2 is a conceptual diagram for explaining the wide-angle imaging mode of the digital camera 1 according to the first embodiment. For example, assume that the digital camera 1 captures a scene as shown in FIG. The scenery to be imaged has a wider angle of view than the angle of view S of the imaging system of the digital camera 1. Therefore, it is not possible to capture all desired scenes with a single imaging.

  In the following description, in order to clarify the imaging range, the imaging angle of view, and the like, the scenery in FIG. 2 is described as a diagram as shown in FIG. In FIG. 3, the angle of view S1 is the size (view angle) of the wide-angle image that is finally generated, and the outside of the image is outside the region that is not left as the final image even if it is captured.

  In the first embodiment, an image writing array is secured on the memory (DRAM 12). For convenience, this is referred to as a canvas in the first embodiment. In the first embodiment, the first captured image captured in the wide-angle imaging mode is set as a reference image (corresponding to an image having an angle of view S in FIG. 3), and the reference image is vertically and horizontally, for example, twice as large ( The reference image is pasted to the center of the imaging area S1) and canvas of FIG. Note that the size of the canvas may be other than twice the height and width.

  Further, the reference image is not necessarily the center of the canvas, and may be, for example, the upper left corner or the upper right corner. Then, the plurality of captured images are aligned and combined so that the reference image (or combined image) partially overlaps, and overwritten on the canvas. As a positioning method, for example, a technique such as block matching can be considered. In addition, in the case of overwriting, a method of performing projection conversion or the like and superimposing using a method such as α blending is conceivable.

  FIG. 4 is a schematic diagram for explaining a user operation in the wide-angle imaging mode of the digital camera 1 according to the first embodiment. For example, when the user presses the shutter SW at the center (half press → full press) for a desired scene, the digital camera 1 captures an image for obtaining a wide-angle composite image at a predetermined timing. I will do it. The user needs to move to change the imaging direction of the digital camera 1 so as to draw a circle, as shown by the arrows in the figure, in order to capture all the images required to obtain a wide-angle composite image. There is. However, it is difficult for the user to know how to move the digital camera 1 or whether the required image has been reliably obtained.

  Therefore, in the first embodiment, in the wide-angle imaging mode, when the shutter SW is pressed by the user, a mark indicating which direction the digital camera 1 should be directed is displayed on the image display unit 15. It is designed to guide the user. When the digital camera 1 detects that the user has turned the imaging direction of the digital camera 1 in the direction indicated by the mark, the digital camera 1 sequentially captures images at that timing, and a wide image that cannot be obtained by a single imaging. It is possible to acquire a plurality of images required for synthesizing the corner images.

  More specifically, based on the first captured image (reference image), a mark indicating the imaging direction for the second and subsequent sheets for generating an image with a wide angle of view is displayed on the image display unit 15. When the imaging direction of the digital camera 1 is directed in the direction indicated by, an image (high quality image) for obtaining an image with a wide angle of view is captured. Then, every time an image is taken, images (high quality images) for obtaining an image with a wide angle of view are taken in sequence while moving and displaying the mark in the next imaging direction. For this reason, in the first embodiment, in order to obtain an image with a wide angle of view, it is sufficient to capture the minimum necessary image, and the memory capacity for storing the image can be efficiently used without squeezing the memory capacity. Can do.

  In the wide-angle imaging mode, an image formed on the CCD 5 in the current imaging direction of the digital camera 1 is displayed as a preview image (low resolution) on the image display unit 15 and is synthesized using the preview image. The synthesized image thus displayed is displayed on the image display unit 15. When all the images (high-quality images) for obtaining a wide-angle image have been captured, the plurality of captured images (high-quality images) are finally combined so that the same region substantially matches. To generate a wide-angle composite image. As described above, during the wide-angle imaging mode, the reduced image synthesized image is displayed on the image display unit 15 and the mark indicating the imaging direction to be imaged next is displayed. Can be easily recognized.

A-2. Operation of First Embodiment Next, the operation of the first embodiment described above will be described.
FIG. 5 is a flowchart for explaining the operation of the digital camera according to the first embodiment. FIGS. 6A to 6D are schematic diagrams illustrating an operation of the digital camera according to the first embodiment and a display example of the image display unit. During the processing described below, the digital camera 1 captures an image formed on the CCD 5 at a predetermined time interval (several tens of frames / second) and displays an image as a preview image (low resolution). Through display is shown in the section 15.

  In addition, the display position of the preview image on the image display unit 15 is the same as the center of the image display unit 15 and the center of the preview image in normal imaging (the subject image in the imaging direction is the center of the image display unit). However, in the wide-angle imaging mode according to the first embodiment, the position of the preview image on the screen of the image display unit 15 according to the imaging direction of the digital camera 1 with the imaging position of the reference image as a reference. Is supposed to move.

  For example, when the user moves the image capturing direction of the digital camera 1 to the right (relative to the reference image capturing position) after capturing the reference image, the preview image is also relative to the reference image on the screen of the image display unit 15. When the user moves to the right and moves the imaging direction to the left, the preview image also moves to the left with respect to the reference image. The same applies to the movement in the vertical direction. Hereinafter, imaging / generation of a wide angle of view will be described on the premise of such preview image position control.

  First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S10). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S10. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S12), and it is determined whether or not the shutter SW is fully pressed (step S14). If the shutter SW is not fully pressed, steps S10 and S12 are repeated.

  On the other hand, when the shutter SW is fully pressed, first, the first captured image (high resolution) is captured as a reference image and stored in the DRAM 12 (step S16). As shown in FIG. A preview image of the reference image is displayed in the center of the display unit 15 (step S18). Next, the CPU 11 specifies a blank portion that is an unimaged portion based on the reference image (step S20), and displays a mark M indicating the imaging direction on the first blank portion on the image display unit 15 (step S20). S22). When the user confirms the mark M, the user moves the digital camera 1 so that the imaging direction of the digital camera 1 is directed in the direction of the mark M. When the user moves the imaging direction to the left with the mark M as a target, the preview image 31 also moves to the left with respect to the reference image 30 as shown in FIG.

  Next, the digital camera 1 captures a preview image (step S24), and calculates an overlay image position (step S26). The calculation of the overlay image position is, for example, calculating the center position (coordinates) of the preview image and aligning the preview image and the reference image (or composite image) so that they overlap, This means that the position of the preview image in the canvas (or the relative position with respect to the reference image) is calculated.

  Next, based on the center position C of the preview image and the position in the canvas, it is determined whether or not the center position C of the preview image has come near the mark M (step S28). If the center position of the preview image is not near the mark M, the process returns to step S24, and the same processing is repeated for the next preview image.

  On the other hand, when the center position C of the preview image comes near the mark M, an image (high resolution) is captured and saved as an effective image in the DRAM 12 (step S30), and the preview image is displayed as a blank portion displaying the mark M. Is overwritten (step S32). In other words, the preview image and the reference image (or composite image) are combined so that a part of them overlaps, and the canvas is overwritten (in the case of the first captured image, the center image is overwritten as the reference image) To do). In the example shown in FIG. 6B, since the center position C of the preview image 31 is in the vicinity of the mark M, the preview image 31 and the reference image 30 are combined so that a part thereof overlaps, and FIG. ), The composite image 32 is displayed on the image display unit 15 (step S34).

  Next, it is determined whether or not all necessary images have been acquired (step S36). If all the necessary images have not been acquired, the process returns to step S20 to identify the next blank part, display the mark M in the next blank part, and capture it as shown in FIG. 6 (d). The same process is repeated for the preview image. As a result, each time an image is captured, the mark M is displayed in the next blank portion, and the user is directed toward the mark M by the image capturing direction of the digital camera 1. In this case, an image (high resolution) is captured and saved as an effective image, and the preview image is sequentially combined with a reference image (or composite image), and the composite image is displayed in the image display unit 15 each time. Display.

  Then, when all the necessary images are acquired, the saved effective images are aligned and combined so as to partially overlap, as in the combination performed using the preview image, and finally, as shown in FIG. An image with a wide angle of view is generated (step S38).

  According to the first embodiment described above, the composite image is displayed on the image display unit 15 in real time, and the mark is displayed on the unimaged blank portion corresponding to the imaging direction in which the digital camera 1 should be directed. Therefore, since the user only has to move the digital camera 1 so as to follow the mark M, a plurality of images required for generating a wide-angle image that cannot be obtained by one imaging can be easily and efficiently performed. Therefore, it is possible to easily capture a wide angle of view.

  Further, only when the digital camera 1 is directed in the direction of a blank portion that has not been imaged, that is, only when the center position of the preview image comes near the mark M, a high resolution for synthesizing a wide-angle image. Since only the effective image is captured and stored, it can be used efficiently without reducing the memory capacity.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
In the second embodiment, when the reference image is captured, by dividing the canvas blank part into a plurality of areas, in a predetermined position of each region in advance a plurality of landmarks (e.g., white) image display unit 15, a mark indicating the center position (for example, yellow) is displayed at the center of the preview image in the direction in which the digital camera 1 faces, and the user displays a yellow mark indicating the center of the preview image on the white mark. By sequentially moving the digital camera 1 so as to approach each other, an effective image (high resolution) necessary for synthesizing images with a wide angle of view is captured and stored. The configuration of the digital camera 1 is the same as that shown in FIG.

  FIG. 7 is a flowchart for explaining the operation of the digital camera 1 according to the second embodiment. FIGS. 8A to 8D are schematic diagrams illustrating an operation of the digital camera according to the second embodiment and a display example of the image display unit.

  First, the CPU 11 determines whether or not the shutter SW is half-pressed (step S40). If the shutter SW is not half-pressed, the CPU 11 repeatedly executes step S40. On the other hand, when the shutter SW is half-pressed, AF (autofocus) processing is executed (step S42), and it is determined whether the shutter SW is fully pressed (step S44). If the shutter SW is not fully pressed, steps S40 and S42 are repeatedly executed.

  On the other hand, when the shutter SW is fully pressed, first, the first captured image (high resolution) is captured as a reference image and stored in the DRAM 12 (step S46). As shown in FIG. A preview image of the reference image is displayed in the center of the display unit 15 (step S48). Next, the CPU 11 displays marks M <b> 1 to M <b> 8 (for example, white) indicating the imaging direction at predetermined positions of a blank portion that is an unimaged portion present around the reference image (step S <b> 50).

  The display positions of the marks M1 to M8 are relative positions of the center of the second and subsequent images with respect to the center of the reference image captured on the first image, and are ideal for combining wide-angle images. Position. That is, based on the angle of view of the reference image, the canvas having the final angle of view is divided into a plurality of areas, and in each area, the mark M1 is positioned at an ideal position when a wide-angle image is synthesized. ~ M8 is displayed.

  Next, a preview image is captured (step S52), and the position of the overlay image is calculated (step S54). The calculation of the overlay image position is, for example, calculating the center position (coordinates) of the preview image and aligning the preview image and the reference image (or composite image) so that they overlap, This means that the position of the preview image in the canvas (or the relative position with respect to the reference image) is calculated.

  Next, as shown in FIG. 8A, a mark (for example, yellow) CM is displayed at the center of the preview image (step S56). When the user confirms the marks M1 to M8, for example, the user moves the digital camera 1 so that the imaging direction of the digital camera 1 is directed to the direction of the mark M1. When the user moves the imaging direction to the left with the mark M1 as a target, the preview image 31 also moves to the left with respect to the reference image 30, as shown in FIG.

  Next, it is determined whether or not the center position of the preview image 31 is in the vicinity of the mark Mi (i = 1 to 8) (step S58). If the center position of the preview image 31 is not near the mark Mi, the process returns to step S52, and the same processing is repeated for the next preview image 31. That is, as the user moves to the digital camera 1, the preview image on which the mark CM indicating the center is displayed moves on the screen of the image display unit 15.

  On the other hand, when the center position of the preview image 31 is in the vicinity of the mark Mi, the image (high resolution) is captured and saved as an effective image in the DRAM 12 (step S60), as shown in FIG. The color of the corresponding mark Mi is changed to green (step S62), and the preview image is overwritten on the blank portion displaying the mark Mi (step S64). In other words, the preview image and the reference image (or composite image) are combined so that a part of them overlaps, and the canvas is overwritten (in the case of the first captured image, the center image is overwritten as the reference image) To do). Next, as shown in FIG. 8C, the composite image 32 is displayed on the image display unit 15 (step S66).

  Next, it is determined whether or not all necessary images have been acquired (step S68). If all the necessary images have not been acquired, the process returns to step S52, and the same processing is repeated for the next preview image.

  That is, when the mark Mi changes to green, the user moves the digital camera 1 so that the mark CM indicating the center of the preview image approaches the next mark M2, as shown in FIG. Change). When the center position of the preview image comes near the mark M2, the image (high resolution) is saved as an effective image, and the color of the mark M2 is changed as shown in FIG. The images are sequentially synthesized with the reference image (or synthesized image), and each time the synthesized image is displayed on the image display unit 15. This operation is continued until all the marks M1 to M8 are changed to green. The user may move the digital camera 1 so as to follow the marks M1 to M8 displayed on the image display unit 15.

  When all the necessary images are acquired, the saved effective images are aligned and combined so as to partially overlap as in the combination performed using the preview image, and finally, as shown in FIG. An image with a wide angle of view is generated (step S70).

  According to the second embodiment described above, the composite image is displayed on the image display unit 15 in real time, the mark is displayed on the unimaged blank portion corresponding to the imaging direction in which the digital camera 1 should be directed, and the imaging is performed. Since the landmarks that have already been used and the landmarks that have not been imaged have different colors, the user can easily know which direction the digital camera should be pointed next. An image required for generation can be captured easily and efficiently, and an image with a wide angle of view can be generated easily and efficiently.

  Further, only when the digital camera 1 is directed in the direction of a blank portion that has not been imaged, that is, only when the center position of the preview image comes near the mark M, a high resolution for synthesizing a wide-angle image. Since the effective image is captured and stored, it can be used efficiently without reducing the memory capacity.

  In the first and second embodiments described above, the movement of the digital camera 1 (change of the imaging direction) is determined based on the relative positional relationship of the preview image with respect to the reference image. In addition, Sensor means that can directly detect the movement of the digital camera 1 such as an acceleration sensor may be used as an auxiliary. By using such a sensor means, in particular, when capturing a subject image for which the position of the review image cannot be obtained accurately, for example, a subject image having no contrast characteristics, it is detected by an acceleration sensor or the like. Based on the movement of the digital camera 1, the position of the preview image with respect to the reference image can be specified.

  Further, the mark indicating the imaging direction may be other than the color described above. Further, it may be accompanied by visual effects such as blinking, and the shape may be a star shape other than a point or a heart shape. In particular, instead of changing the color of the corresponding mark after capturing as an effective image, the shape may be changed. Alternatively, the corresponding mark may be erased after being captured as an effective image.

  In the first and second embodiments described above, the digital camera has been described as the imaging device. However, the present invention is not limited to this, and any electronic device having an imaging function can be applied to, for example, a mobile phone.

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Imaging lens 3 Lens drive part 4 Shutter combined shutter 5 CCD
6 TG
7 Unit Circuit 8 Image Processing Unit 11 CPU
12 DRAM
13 Memory 14 Flash memory 15 Image display section 16 Key input section 17 Card I / F
18 Memory card

Claims (6)

Imaging means;
Display means;
Generating means for generating a wider-angle image than the captured image based on the images sequentially captured by the imaging means;
An imaging range necessary for generating a wide-angle image by the generation unit , a preview image as a through image displayed in the imaging range and sequentially captured by the imaging unit, and the imaging unit First display control means for displaying on the display means a mark indicating one of each of the plurality of imaging directions;
Determination means for determining a positional relationship between said display means to the preview image and the previous SL mark displayed by said first display control means,
Based on the determination result by said determination means, and second display control means for displaying on said display means by changing the display mode of the previous SL landmarks,
An imaging apparatus comprising: The first display control means causes the display means to display the mark so as to superimpose information indicating an imaging direction of the imaging means on a region of an unimaged portion in the imaging range.
The imaging apparatus according to claim 1. When the determination unit determines that the preview image and the mark are in a positional relationship closer than a predetermined distance, the second display control unit displays a mark corresponding to the imaging direction in the area of the unimaged portion. The display mode of the information indicating the imaging direction is changed and displayed on the display means by moving to a position indicating a new imaging direction in
The imaging apparatus according to claim 2. A calculation means for calculating a center position of the preview image;
The determining means determines whether the preview image and the mark are in a close positional relationship based on whether or not the position of the mark and the center position of the preview image calculated by the calculating means substantially match. to decide,
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. The first display control means displays a mark indicating the center position at the center position of the preview image calculated by the calculation means.
The imaging apparatus according to claim 4. Computer
Generating means for generating an image having a wider angle than the captured image based on the images sequentially captured by the imaging means;
An imaging range required to produce a wide-angle image by the generation unit, and the preview image as a through image corresponding to the sequentially captured images by the imaging means, of each of the plurality of imaging directions of the imaging means first display control means for displaying the one on the display means and indicates to th mark,
Determination means for determining a positional relationship between said display means to the preview image and the previous SL mark displayed by said first display control means,
Based on the determination result by the determination means, second display control means for displaying on said display means by changing the table示態like before Symbol landmarks,
A program characterized by functioning as

Images