JP5641352B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus such as a digital camera or a mobile phone having an imaging function, an image processing method, and a program.

  In a digital camera, a mobile phone having an imaging function, and the like, the limit of the imaging angle of view depends on hardware specifications included in the apparatus main body, such as the focal length of the lens and the size of the imaging element. For such a problem of the limit of the imaging angle of view, a conversion lens for wide-angle imaging or the like is mounted in front of an existing lens in the imaging apparatus (for example, Patent Documents 1, 2, 3), a plurality of lenses are provided in advance, and this is dealt with by switching the lenses according to the imaging purpose (for example, see Patent Document 4).

JP 2004-191897 A Japanese Patent Laying-Open No. 2005-027142 JP 2005-057548 A JP 2007-081473 A

  However, in the above prior art, in order to perform wide-angle imaging, it is necessary to attach a conversion lens for wide-angle imaging one by one, or switch the lens according to the imaging purpose, and in terms of operability and cost There was a problem. In addition, there is a problem that it is difficult to obtain a wide-angle image desired by the photographer even if these conversion lenses for wide-angle imaging and switchable lenses are used.

  Therefore, an object of the present invention is to provide an image processing apparatus, an image processing method, and a program that can easily and efficiently obtain an image necessary for generating a wide-angle image without exchanging lenses. .

The present invention provides an acquisition unit that acquires a plurality of panoramic images that are each generated by combining a plurality of images and that is long in one direction, and a size of an imaging range between the plurality of panoramic images acquired by the acquisition unit. As a result of the comparison by the comparison means and the comparison means, the panorama images are arranged in a direction orthogonal to the one direction so that the panorama images have a size matching the smaller panorama image of the imaging range. And a wide-angle image generating means for generating a single wide-angle image by combining with the image processing apparatus.

  According to the present invention, there is an advantage that an image necessary for generating a wide-angle image can be obtained easily and efficiently without exchanging lenses.

It is a block diagram which shows the structure of the digital camera by 1st Embodiment of this invention. It is a conceptual diagram for demonstrating normal imaging mode. It is a conceptual diagram for demonstrating the panorama imaging mode with the digital camera 1 by this 1st Embodiment. It is a conceptual diagram which shows the motion (how to move by a user) of the digital camera 1 in the panorama imaging mode with the digital camera 1 by this 1st Embodiment. 4 is a flowchart for explaining an operation of the digital camera 1 according to the first embodiment. 4 is a flowchart for explaining an operation of continuous shooting processing in the digital camera 1 according to the first embodiment. It is a flowchart for demonstrating the operation | movement of the digital synthetic | combination process by this 1st Embodiment. It is a conceptual diagram for demonstrating the operation | movement of the digital synthetic | combination process by this 1st Embodiment. It is a flowchart for demonstrating operation | movement of the continuous shooting process in the digital camera 1 by this 2nd Embodiment. It is a conceptual diagram for demonstrating operation | movement of the continuous shooting process in the digital camera 1 by this 2nd Embodiment. It is a flowchart for demonstrating operation | movement of the continuous shooting process in the digital camera 1 by this 3rd Embodiment. It is a conceptual diagram for demonstrating operation | movement of the continuous shooting process in the digital camera 1 by this 3rd Embodiment. It is a conceptual diagram which shows the other example of how to move the digital camera 1 at the time of panoramic photography. It is a conceptual diagram which shows the imaging frame which shows an imaging | photography range, and the other example of a display of a moving direction.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

A. First embodiment A-1. Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a digital camera according to a first embodiment of the present invention. In the figure, a digital camera 1 includes an imaging lens 2, a lens driving unit 3, a diaphragm / shutter 4, a CCD 5, and a TG (Timing).
Generator) 6, unit circuit 7, image processing unit 8, CPU 11, DRAM 12, memory 13, flash memory 14, image display unit 15, key input unit 16, card I / F 17, and memory card 18.

  The imaging lens 2 includes a focus lens, a zoom lens, and the like, and a lens driving unit 3 is connected thereto. The lens driving unit 3 includes a motor for driving the focus lens and the zoom lens constituting the imaging lens 2 in the optical axis direction, a focus motor driver for driving the focus motor and the zoom motor according to a control signal from the CPU 11, and a zoom motor driver. It is configured.

  The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 11. The diaphragm 4 controls the amount of light that enters from the imaging lens 2. The CCD (imaging device) 5 converts the light of the subject projected through the imaging lens 2 and the diaphragm 4 into an electrical signal and outputs it to the unit circuit 7 as an imaging signal. The CCD 5 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

The unit circuit 7 is a CDS (Correlated) that holds the imaging signal output from the CCD 5 by correlated double sampling.
Double Sampling) circuit, AGC (Automatic) for automatic gain adjustment of the imaged signal after sampling
(Gain Control) circuit and an A / D converter for converting an analog image pickup signal after the automatic gain adjustment into a digital signal. The imaging signal of the CCD 5 is sent to the image processing unit 8 as a digital signal through the unit circuit 7. The unit circuit 7 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The image processing unit 8 performs image processing (pixel interpolation processing, γ correction, luminance color difference signal generation, white balance processing, exposure correction processing, etc.) of image data sent from the unit circuit 7, and compression / decompression of image data ( For example, JPEG format, M-JPEG format, or MPEG format compression / expansion processing, processing of combining a plurality of captured images, and the like are performed. The image processing unit 8 is driven according to a timing signal having a predetermined frequency generated by the TG 6.

  The CPU 11 is a one-chip microcomputer that controls each part of the digital camera 1. In particular, in the first embodiment, while the user moves the digital camera 1, the CPU 11 continuously shoots a plurality of images at a predetermined cycle (time interval), and some of the captured images are partially captured. The components are combined so as to overlap each other (for example, using an α blend), and each unit is controlled so as to generate a single composite image captured at a wide angle. Details of the image composition will be described later.

  The DRAM 12 is used as a buffer memory for temporarily storing image data sent to the CPU 11 after being imaged by the CCD 5 and also as a working memory for the CPU 11. The memory 13 stores a program necessary for controlling each part of the digital camera 1 by the CPU 11 and data necessary for controlling each part, and the CPU 11 performs processing according to this program. The flash memory 14 and the memory card 18 are recording media for storing image data captured by the CCD 5.

  The image display unit 15 includes a color LCD and its driving circuit, and displays the subject imaged by the CCD 5 as a through image when in the imaging standby state, and when reproducing the recorded image, the flash memory 14 or the memory card 23. The recorded image read out and decompressed is displayed. The key input unit 16 includes a plurality of operation keys such as a shutter SW, a zoom SW, a mode key, a SET key, and a cross key, and outputs an operation signal corresponding to the user's key operation to the CPU 11. A memory card 18 is detachably attached to the card I / F 17 through a card slot (not shown) of the digital camera 1 main body.

  FIG. 2 is a conceptual diagram for explaining a normal imaging mode. When the digital camera 1 shoots in the normal imaging mode, it can shoot only at the angle of view S of the imaging system of the digital camera 1 as shown in FIG.

  FIG. 3 is a conceptual diagram for explaining a panoramic imaging mode in the digital camera 1 according to the first embodiment. FIG. 4 is a conceptual diagram showing movement of the digital camera 1 in the panoramic imaging mode (how to move by the user) in the digital camera 1 according to the first embodiment.

  The user holds the digital camera 1 in the vertical direction so that the longitudinal direction of the angle of view is the vertical direction with respect to the desired scenery, and depresses the shutter SW at the upper left end (half press → full press). First, the shutter SW is pressed down and moved in the horizontal direction (state # 1: see FIG. 4), moved in the vertical direction at a predetermined position (state # 2: see FIG. 4), and further horizontal. Move in the direction (state # 3: see FIG. 4). The digital camera 1 sequentially captures images at a predetermined timing.

  In the digital camera 1, a first panoramic image is synthesized from a plurality of images taken in the state # 1 moving in the horizontal direction, and a second one is obtained from the plurality of images taken in the same state # 3 moving in the horizontal direction. The panorama images are synthesized, the first panorama image and the second panorama image are synthesized, and finally a desired wide-angle image (lower side in FIG. 3) is generated. In the state # 2 in which the image is moved in the vertical direction, it is not necessary for generating a panoramic image, and thus no image is captured.

A-2. Operation of First Embodiment Next, the operation of the first embodiment described above will be described.
FIG. 5 is a flowchart for explaining the operation of the digital camera 1 according to the first embodiment. First, when the shutter SW is half-pressed (step S10), the CPU 11 executes an AF (autofocus) process (step S12). When the shutter SW is fully pressed (step S14), the CPU 11 performs a predetermined cycle (time). A plurality of images are continuously shot at intervals (step S16). Details of the continuous shooting process will be described later.

  At this time, as shown in FIG. 3, the user holds the digital camera 1 in the vertical direction so that the longitudinal direction of the field of view is the vertical direction with respect to the desired scenery, and presses the shutter SW in the upper left (half way). 3), and as shown by the arrow shown in FIG. 3, the digital camera 1 is first moved in the horizontal direction (from the left end to the right) by pressing the shutter SW (state # 1: see FIG. 4) Are moved in the vertical direction (downward) at the position (state # 2: see FIG. 4) and further moved in the horizontal direction (leftward) (state # 3: see FIG. 4). Then, it is determined whether or not the continuous shooting is finished, that is, whether or not the panorama shooting is finished (step S18). If the continuous shooting is not finished, the process returns to step S16 and the continuous shooting process is continued.

  On the other hand, when the continuous shooting is finished, that is, when panoramic photography is finished (YES in step S18), the first panoramic image is taken from a plurality of images taken in the state # 1 moving rightward from the left end. Combining and combining the second panoramic image from the plurality of images taken in the state # 3 moving from the right end to the left direction, combining the first panoramic image and the second panoramic image, finally, A desired wide-angle image is generated (step S20). Details of the synthesis process will be described later.

  FIG. 6 is a flowchart for explaining the operation of the continuous shooting process in the digital camera 1 according to the first embodiment. First, the CPU 11 aligns the previous captured image with the current captured image (step S30), and determines whether the current state is any of states # 1, # 2, and # 3 (step S30). S32).

  When the user moves the digital camera 1 in the horizontal direction (from the left end to the right) # 1 (state S1 in step S32), the captured image is saved as an image for creating a panorama. (Step S34). Next, it is determined whether or not the digital camera 1 has reached a predetermined position (in this case, the right end that is the final position in the state # 1) (step S36). If the predetermined position has not been reached (NO in step S36), the process ends without changing the current state # 1, and the process returns to the main routine shown in FIG.

  Thereafter, until the digital camera 1 reaches a predetermined position (in this case, the right end which is the end position in the state # 1), the captured image is stored as an image for creating a panorama in step S34. When the digital camera 1 reaches a predetermined position (YES in step S36), the state changes from state # 1 to state # 2, so the current state is changed to state # 2 (step S38).

  Next, when the user shifts to the state # 2 in which the digital camera 1 is moved in the vertical direction (downward) (state # 2 in step S32), the captured image is not stored as a panorama creation image. (Step S40), it is determined whether or not the digital camera 1 has reached a predetermined position (in this case, the lower right end which is the final position in the state # 2) (Step S42). If the predetermined position has not been reached (NO in step S42), the process ends without changing the current state # 2, and the process returns to the main routine shown in FIG.

  Thereafter, until the digital camera 1 reaches a predetermined position (in this case, the lower right end, which is the end position in the state # 2), the digital camera 1 continues to operate without being saved as a panorama creation image. When the position is reached (YES in step S42), since the state # 2 is shifted to the state # 3, the current state is changed to the state # 3 (step S44).

  Next, when the user moves to the state # 3 in which the digital camera 1 is moved in the horizontal direction (from right to left) (state # 3 in step S32), the captured image is stored as a panorama creation image. (Step S46). Then, it is determined whether or not the digital camera 1 has reached a predetermined position (in this case, the left end that is the end position in the state # 3) (step S48). If the predetermined position has not been reached (NO in step S48), the process ends without changing the current state # 3, and the process returns to the main routine shown in FIG.

  Thereafter, until the digital camera 1 reaches a predetermined position (in this case, the left end which is the end position in the state # 3), the captured image is stored as an image for creating a panorama in step S46. When the digital camera 1 reaches a predetermined position (YES in step S48), the continuous shooting is terminated (step S50).

  As a result of the above-described operation, the user moves the digital camera 1 from the left end in the right direction # 1 and a plurality of images taken in the state # 1 and the digital camera 1 moves from the right to the left direction # 3 And a plurality of images picked up by. Next, a method for obtaining a final wide-angle image using the plurality of captured images will be described.

  FIG. 7 is a flowchart for explaining the operation of the digital synthesizing process according to the first embodiment. FIG. 8 is a conceptual diagram for explaining the operation of digital synthesis processing according to the first embodiment. First, the CPU 11 acquires an image for creating a panorama (step S60), and determines the state at the time of shooting the image (step S62). If the state at the time of shooting the image is state # 1, the composition processing is performed as panorama # 1 (step S64).

  Thereafter, it is determined whether or not the panorama synthesis has ended (step S68). If not, the process returns to step S60. Hereinafter, as shown in FIG. 8A, a plurality of images FR1 to FR6 photographed in the state # 1 are combined so as to partially overlap (for example, using an α blend) to create a panorama # 1. To do.

  On the other hand, when the acquired panorama creation image is an image captured in the state # 2, the composition process is performed as the panorama # 2 (step S66). Thereafter, it is determined whether or not the panorama synthesis has ended (step S68). If not, the process returns to step S60. Hereinafter, as shown in FIG. 8B, a plurality of images FL1 to FL6 photographed in the state # 2 are combined so as to partially overlap (for example, using an α blend) to create a panorama # 2. To do.

  Next, when the panorama composition is completed (YES in step S68), as shown in FIG. 8C, a predetermined area on the lower side of the panorama # 1 and a predetermined area on the upper side of the panorama # 2 are A part of the images is overlapped (for example, using an α blend) to create one wide-angle image (step S70).

  In the first embodiment described above, when a wide-angle image capturing range is designated in advance from the key input unit, a panoramic image having a size based on the designated capturing range may be generated. In addition, when the panorama images are combined in the image processing unit 8, the imaging ranges of the panoramic images to be combined are compared, and as a result of the comparison, another panoramic image is captured in the panoramic image with the smaller imaging range. A final wide-angle image may be generated by combining the ranges.

  According to the first embodiment described above, an image necessary for generating a wide-angle image can be obtained easily and efficiently without exchanging lenses.

B. Second Embodiment Next, a second embodiment of the present invention will be described.
In the second embodiment, when the user moves the digital camera 1 during panoramic shooting, the change in the moving direction from the state # 1 to the state # 2 and from the state # 2 to the state # 3 is used as a camera factor. It is characterized by using as a trigger information, such as a direction sensor and an acceleration sensor, and user factors, such as a movement direction instruction operation, a key operation such as a shutter key, a user gesture, and voice.

  The configuration of the digital camera 1 is the same as that shown in FIG. Further, the main routine of the panorama shooting mode is the same as that in FIG. 5, and the composition process is the same as that in FIG.

  FIG. 9 is a flowchart for explaining the operation of the continuous shooting process in the digital camera 1 according to the second embodiment. FIG. 10 is a conceptual diagram for explaining the operation of continuous shooting processing in the digital camera 1 according to the second embodiment.

  In the second embodiment, in accordance with trigger detection, in order to determine whether the digital camera 1 is currently in the state # 1, # 2, or # 3, it is inverted every time a trigger is detected. A flag and a state #N (coefficient) that is incremented by 1 each time a trigger is detected are prepared. The flag has an initial value of “0” and is inverted every time a trigger is detected. The state #N is a coefficient indicating an initial value “1” and indicating whether the current digital camera 1 is in the state # 1, # 2, or # 3. The state #N becomes “1” indicating the state # 1 until the first trigger is detected, and becomes “2” every time the trigger is detected, indicating the state # 2, and “3” ", Indicating the state # 3.

  First, the CPU 11 aligns the previous captured image with the current captured image (step S80), and determines whether a trigger is detected (step S82). If the trigger is not detected, as shown in FIG. 10, it is determined that the state # 1 is not shifted to the state # 2, that is, the state # 1 is moving in the horizontal direction (from the left end to the right). Then, it is determined whether or not the flag is “1” (step S88). In this case, since the trigger has not been detected yet, the flag is “0”, and the state #N is “1”.

  Since the flag is “0” (NO in step S88), the captured image is stored as an image for creating a panorama in state #N (= 1) (step S90). Thereafter, the process is terminated, and the process returns to the main routine shown in FIG. Hereinafter, in step S90, the captured image is stored as a panorama creation image in state #N (= 1) until a trigger is detected.

  Next, as shown in FIG. 10, when transitioning from state # 1 to state # 2, for example, as a camera factor, a movement amount, direction sensor information, or a user factor, a key such as a movement direction or a shutter key. When it is detected by operation, user gesture, voice, etc. that the digital camera 1 has reached the end of the state # 1, these are detected as triggers (YES in step S82). When the trigger is detected, the flag is inverted (step S84), and the state #N is set to N + 1 (step S86). In this case, the flag is “1”, and the state #N is “2”.

  Next, it is determined whether or not the flag is “1” (step S86). In this case, since the flag is “1”, the captured image is not saved as a panoramic image (step S92), the process is terminated, and the process returns to the main routine shown in FIG. Hereinafter, until the next trigger is detected (until the moving direction of the digital camera 1 is in the state # 3), the processing is terminated without saving the captured image as a panorama creation image, and the process shown in FIG. The operation to return to the main routine shown is repeated. Therefore, in the state # 2, the captured image is not saved.

  Next, as shown in FIG. 10, when transitioning from state # 2 to state # 3, for example, movement factors, direction sensor information, and user factors such as movement direction and shutter key are used as camera factors. When it is detected by operation, user gesture, voice, etc. that the digital camera 1 reaches the end of the state # 2 and shifts to the state # 3, these are detected as triggers (YES in step S82). When the trigger is detected, the flag is inverted (step S84), and the state #N is set to N + 1 (step S86). In this case, the flag is “0”, and the state #N is “3”.

  Next, it is determined whether or not the flag is “1” (step S86). In this case, since the flag is “0”, the captured image is stored as an image for creating a panorama in state #N (= 3) (step S90). Thereafter, the process is terminated, and the process returns to the main routine shown in FIG. Hereinafter, until the next trigger is detected, in step S90, the captured image is stored as an image for creating a panorama in state #N (= 3).

  As a result of the above-described operation, the user is moving the digital camera 1 from the left end to the right and a plurality of images taken in the state # 1 and the digital camera 1 is moving from the lower right to the left # 3 are obtained.

  Next, using the plurality of captured images, a plurality of images in state # 1 are synthesized by a method similar to that in the first embodiment described above (see FIG. 7) to obtain panorama # 1 and state # 3. A plurality of images are combined to create panorama # 2, and a predetermined area on the lower side of panorama # 1 and a predetermined area on the upper side of panorama # 2 are partially overlapped (for example, using α blending) And combine them to create one wide-angle image.

  According to the second embodiment described above, an image necessary for generating a wide-angle image can be obtained easily and efficiently without exchanging lenses.

C. Third Embodiment Next, a third embodiment of the present invention will be described.
In the first and second embodiments described above, the user takes a desired scene as shown by the arrows in FIG. 3 in order to capture all the images required to obtain a wide-angle composite image. On the other hand, the digital camera 1 is held in the vertical direction so that the longitudinal direction of the angle of view becomes the vertical direction, the shutter SW is pressed down at the upper left (half press → full press), and first, the shutter SW is pressed down in the horizontal direction ( It moves from the left end to the right (state # 1), moves vertically at a predetermined position (state # 2), and further moves horizontally (left) (state # 3). However, it is difficult for the user to know how to move the digital camera 1 or whether the required image has been reliably obtained.

  Therefore, in the third embodiment, when the user presses the shutter switch in the panoramic imaging mode, the shooting frame indicating the area to be imaged by the digital camera 1 and the movement indicating which direction should be moved. By displaying the direction on the image display unit 15, the user is guided. Further, during the panoramic imaging mode, an image formed on the CCD 5 of the digital camera 1 is currently displayed as a preview image (low resolution) on the image display unit 15 and a synthesized image synthesized using the preview image is displayed as an image. The display unit 15 displays the image with translucency (transmittance 50%).

  In this way, during the panoramic imaging mode, an imaging frame indicating an imaging area to be imaged next, a moving direction indicating which direction should be moved, and a reduced image obtained by combining captured images are displayed as images. Since the display is made on the display unit 15, the user can easily recognize in which direction the digital camera 1 should be moved.

  The configuration of the digital camera 1 is the same as that shown in FIG. Further, the main routine of the panorama shooting mode is the same as that in FIG. 5, and the composition process is the same as that in FIG.

  FIG. 11 is a flowchart for explaining the operation of the continuous shooting process in the digital camera 1 according to the third embodiment. FIG. 12 is a conceptual diagram for explaining the operation of continuous shooting processing in the digital camera 1 according to the third embodiment.

  First, the CPU 11 aligns the previous captured image with the current captured image (step S100), and determines whether the current state is any of states # 1, # 2, and # 3 (step S100). S102). When the user moves the digital camera 1 from the left end to the right direction (state # 1 in step S102), the shooting frame FR1 and the movement direction M1 are set as shown in FIG. It is displayed (step S104). Then, the captured image is stored as an image for creating a panorama in state # 1 (step S106), and a simple composite image IMG is generated from the reduced image of the stored captured image and displayed in a semi-transparent (transmittance of 50%). (Step S108).

  Next, it is determined whether or not the digital camera 1 has reached a predetermined position P1 (in this case, the final position in the state # 1: see FIG. 12) (step S110). If the predetermined position P1 has not been reached (NO in step S110), the process ends without changing the current state # 1, and the process returns to the main routine.

  Thereafter, until the digital camera 1 reaches the predetermined position P1, in step S106, the captured image is saved as an image for creating a panorama in the state # 1, and in step S108, the composite image at that time is translucent ( (Transmittance 50%). Then, when the digital camera 1 reaches the predetermined position P1 (YES in step S110), since the state # 1 is shifted to the state # 2, the current state is changed to the state # 2 (step S112).

  Next, when the user shifts to the state # 2 in which the digital camera 1 is moved downward from the terminal position of the state # 1 (state # 2 in step S102), as shown in FIG. 12, the shooting frame FR2, the moving direction M2 is displayed (step S114). Then, without saving the captured image as an image for creating a panorama (step S116), whether or not the digital camera 1 has reached the predetermined position P2 (in this case, the final position in the state # 2: see FIG. 12). Is determined (step S118). If the predetermined position P2 has not been reached (NO in step S118), the process ends without changing the current state # 2, and the process returns to the main routine shown in FIG.

  Hereinafter, until the digital camera 1 reaches the predetermined position P2, the operation is continued without being saved as a panorama creation image. When the digital camera 1 reaches the predetermined position P2 (YES in step S118), the state # Since the state shifts from 2 to state # 3, the current state is changed to state # 3 (step S120).

  Next, when the user shifts the digital camera 1 from the state # 2 to the state # 3 in which the digital camera 1 is moved to the left (state # 3 in step S102), as shown in FIG. 12, the shooting frame FR3 and the moving direction M3 are displayed. (Step S122). Then, the captured image is stored as an image for creating a panorama in the state # 3 (step S124), a simple composite image is generated from the reduced image of the stored captured image, and is displayed semi-transparently (with a transmittance of 50%). (Step S126).

  Thereafter, until the digital camera 1 reaches a predetermined position P3 (end point), in step S124, the captured image is stored as an image for creating a panorama in state # 3, and in step S126, the composite image at that time is stored. The display is translucent (transmittance 50%). When the digital camera 1 reaches the predetermined position P3 (YES in step S128), continuous shooting is terminated (step S130).

  As a result of the above-described operation, the user is moving the digital camera 1 from the left end to the right and a plurality of images taken in the state # 1 and the digital camera 1 is moving from the lower right to the left # 3 are obtained.

  Next, using the plurality of captured images, a plurality of images in state # 1 are synthesized by a method similar to that in the first and second embodiments described above (see FIG. 7), and panorama # 1, state A plurality of images # 3 are combined to create panorama # 2, and a predetermined area below panorama # 1 and a predetermined area above panorama # 2 are partially overlapped (for example, α Combine (using blend) to create a single wide-angle image.

  In the above-described third embodiment, as shown in FIG. 12, a margin (margin) MG is provided in the photographing frames FR1 to FR3 displayed on the image display unit 15, that is, an image range actually captured. It is preferable to use a slightly larger photographing frame. By providing the margin MG, a margin is generated in the range in which the digital camera 1 is moved, and the user's stress can be reduced.

  In the third embodiment described above, the change in the movement direction from state # 1 to state # 2 and from state # 2 to state # 3 is determined based on whether or not a predetermined position has been reached. Not limited to this, as in the second embodiment described above, as camera factors, information such as movement amount, direction sensor and acceleration sensor, as user factors, movement direction instruction operations, key operations such as shutter keys, A gesture, voice, or the like may be used as a trigger.

  According to the third embodiment described above, the composite image is displayed on the image display unit 15 in real time, and the photographing frame and the moving direction to which the digital camera 1 should be moved are displayed. Since it is only necessary to move the digital camera 1 while looking at the frame and the moving direction, it is possible to easily and efficiently capture a plurality of images required to generate a wide-angle image that cannot be obtained by a single imaging. Therefore, it is possible to easily generate an image with a wide angle of view.

  In the first to third embodiments described above, the digital camera 1 is moved from left to right, from top to bottom, and from right to left at the time of panoramic photography. As shown in (a), a plurality of images may be taken while moving in one direction, and a single wide-angle image may be generated by combining them. Alternatively, as shown in FIG. 13B, the digital camera 1 is moved three times or more in the horizontal direction, such as left to right, top to bottom, right to left, top to bottom, and left to right. Alternatively, a wide-angle image may be generated by combining three or more panoramic images.

  In the third embodiment described above, the shooting frame indicating the shooting range and the moving direction are displayed. However, the present invention is not limited to this, and as shown in FIG. 14, the entire frame 20 of the final wide-angle image is displayed. Then, the portion 30 acquired as a captured image may be filled.

As mentioned above, although several embodiment of this invention was described, this invention is not limited to these, The invention described in the claim, and its equal range are included.
Below, the invention described in the claims of the present application is appended.

(Appendix 1)
The invention according to attachment 1 includes acquisition means for acquiring a plurality of panoramic images, and wide-angle image generation means for generating one wide-angle image by combining the plurality of panoramic images acquired by the acquisition means. An image processing apparatus characterized by the above.

(Appendix 2)
The invention described in appendix 2 is a panoramic image obtained by combining an imaging unit, an imaging control unit that sequentially captures images by the imaging unit, and a plurality of images that are sequentially captured by the imaging unit by imaging control by the imaging control unit. The image processing apparatus according to appendix 1, further comprising: a panoramic image generating unit that generates, wherein the acquiring unit acquires a plurality of panoramic images generated by the panoramic image generating unit.

(Appendix 3)
The invention according to attachment 3 is characterized in that the panoramic image generation unit generates the plurality of panoramic images from a plurality of images sequentially captured while moving an imaging region of the imaging unit in a predetermined direction. The image processing apparatus according to Supplementary Note 2.

(Appendix 4)
According to the fourth aspect of the present invention, the panoramic image generation unit generates the plurality of panoramic images from a plurality of images sequentially captured while moving the imaging region of the imaging unit a plurality of times in a predetermined direction. The image processing apparatus according to Supplementary Note 2, which is a feature.

(Appendix 5)
According to the fifth aspect of the present invention, the panoramic image generation means is a process in which a plurality of images sequentially captured in the process of moving the imaging region by the imaging means in one direction and a process in which the panorama image generation means moves in a direction opposite to the one direction The image processing apparatus according to appendix 4, wherein the plurality of panoramic images are generated from each of the plurality of images sequentially captured.

(Appendix 6)
According to the sixth aspect of the present invention, the panoramic image generation means captures a predetermined time, a predetermined number of images, or a predetermined range of images by the imaging means in the process of moving the imaging region by the imaging means in one direction. Then, one panoramic image is generated from these images, and in the process of moving the imaging region by the imaging means in the direction opposite to the one direction, images for a predetermined time, a predetermined number, or a predetermined range are obtained. The image processing apparatus according to appendix 5, wherein one panorama image is generated from these images when the image is picked up by the image pickup means.

(Appendix 7)
The invention according to appendix 7 further includes designation means for designating a wide-angle image imaging range in advance, and the panorama image generating means generates the panoramic image based on the imaging range designated by the designation means. An image processing apparatus according to any one of appendices 2 to 6, characterized by:

(Appendix 8)
The invention according to attachment 8 further includes a comparison unit that compares the imaging ranges of the panoramic images to be combined, and the wide-angle image generation unit converts the panorama image with the smaller imaging range as a result of the comparison by the comparison unit. The image processing apparatus according to any one of appendices 1 to 7, wherein the wide-angle image is generated by combining the imaging ranges of other panoramic images.

(Appendix 9)
The invention according to attachment 9 is characterized in that the wide-angle image generation unit generates one wide-angle image by combining upper predetermined regions and lower predetermined regions of the plurality of panoramic images acquired by the acquisition unit. 9. The image processing apparatus according to any one of appendices 1 to 8, which is a feature.

(Appendix 10)
The invention according to attachment 10 includes an acquisition step of acquiring a plurality of panoramic images and a wide-angle image generation step of generating a single wide-angle image by combining the plurality of panoramic images acquired in the acquisition step. An image processing method characterized by the above.

(Appendix 11)
The invention described in appendix 11 includes a computer of an image processing apparatus that performs image processing, an acquisition function that acquires a plurality of panoramic images, and a combination of the plurality of panoramic images acquired by the acquisition function to form one wide-angle image. A program for executing a wide-angle image generation function to be generated.

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Imaging lens 3 Lens drive part 4 Shutter combined shutter 5 CCD
6 TG
7 Unit Circuit 8 Image Processing Unit 11 CPU
12 DRAM
13 Memory 14 Flash memory 15 Image display section 16 Key input section 17 Card I / F
18 Memory card

Claims (15)

Acquisition means for acquiring a plurality of panoramic images that are each generated by combining a plurality of images and that are long in one direction;
Comparison means for comparing the sizes of the imaging ranges of a plurality of panoramic images acquired by the acquisition means;
As a result of the comparison by the comparison unit, the plurality of panoramic images are synthesized in a direction orthogonal to the one direction so as to have a size that matches the smaller panoramic image of the imaging range. An image processing apparatus comprising: a wide-angle image generation unit configured to generate a wide-angle image. Imaging means;
Panorama image generating means for generating a panoramic image by combining a plurality of images sequentially captured by the imaging means;
The acquisition means includes
The image processing apparatus according to claim 1, wherein a plurality of panoramic images generated by the panoramic image generating unit are acquired. Detecting means for detecting a predetermined trigger;
When the predetermined trigger is detected by the detection unit, the panoramic image generation unit generates a panoramic image by combining a plurality of images sequentially captured by the imaging unit until the trigger is detected;
The image processing apparatus according to claim 2, wherein the wide-angle image generation unit generates a wide-angle image by combining a plurality of panoramic images generated by the panorama image generation unit. The panoramic image generating means
The panorama image is generated from a plurality of images sequentially captured by the imaging unit until a predetermined trigger is detected by the detection unit. Image processing device. Storage means for sequentially storing the sequentially captured images;
A storage control unit for storing the captured image in the storage unit when a predetermined trigger is detected by the detection unit;
After the image has been stored by the Symbol 憶 control means, when a predetermined trigger is detected by said detecting means, from a plurality of images stored in said storage means by said storage control means between the predetermined trigger, The image processing apparatus according to claim 4, wherein the panorama image is generated. The detection means includes
A movement amount or direction sensor information, a change in a movement direction, a predetermined instruction operation by a user, an instruction by a user's voice, or an instruction by a user's operation is detected as the predetermined trigger. The image processing apparatus according to claim 3. The storage control means
When a trigger is detected by the detection unit, the storage of the image captured by the imaging unit after that is prohibited in the storage unit,
6. When the trigger is detected by the detection unit while the storage is prohibited, the subsequent storage of the image captured by the imaging unit is resumed in the storage unit. 6 . The image processing apparatus described. The panoramic image generating means
The image processing apparatus according to claim 2, wherein the plurality of panoramic images are generated from a plurality of images sequentially captured while an imaging region of the imaging unit is moved in a predetermined direction. The panoramic image generating means
The image processing apparatus according to claim 2 , wherein the plurality of panoramic images are generated from a plurality of images sequentially captured while the imaging region of the imaging unit is moved a plurality of times in a predetermined direction. The panoramic image generating means
From each of a plurality of images sequentially captured in the process of moving the imaging region of the imaging means in one direction and a plurality of images sequentially captured in the process of moving in the direction opposite to the one direction, The image processing apparatus according to claim 9, wherein a plurality of panoramic images are generated. The panoramic image generating means
In the process of moving the imaging region by the imaging means in one direction, when images for a predetermined time, a predetermined number, or a predetermined range are imaged by the imaging means, one panoramic image is generated from those images,
When the imaging area of the imaging means is moved in a direction opposite to the one direction, images for a predetermined time, a predetermined number, or a predetermined range are captured by the imaging means, and one panorama is obtained from these images. The image processing apparatus according to claim 10, wherein an image is generated. Further comprising a designation means for designating a wide-angle image capturing range in advance;
The panoramic image generation hand stage,
The image processing apparatus according to claim 2, wherein the panoramic image is generated based on an imaging range designated by the designation unit. The wide-angle image generation means 12 the preceding claims, characterized in that to generate the upper predetermined region and a lower one wide-angle image by combining a predetermined region between the plurality of panorama images acquired by the acquisition unit An image processing apparatus according to any one of the above. An acquisition step of acquiring a plurality of panoramic images that are each generated by combining a plurality of images and that are long in one direction;
A comparison step of comparing the sizes of the imaging ranges of a plurality of panoramic images acquired by the acquisition step ;
As a result of the comparison in the comparison step, the plurality of panoramic images acquired in the acquisition step are orthogonal to the one direction so as to have a size that matches the smaller panoramic image of the imaging range. And a wide-angle image generation step of generating one wide-angle image by combining with the image processing method. In the computer of the image processing apparatus that performs image processing,
An acquisition function for acquiring a plurality of panoramic images each generated by combining a plurality of images and being long in one direction;
Comparing function of comparing the size of the imaging range of the plurality of panorama images between acquired by the acquisition function,
The comparison function by the comparison result, so that the size matching the size of the smaller of the panoramic image in the imaging range and orthogonal direction of the plurality of panorama images between acquired by the acquisition function to said one direction A program for executing a wide-angle image generation function for generating a wide-angle image by combining the two.

Images