JP2013146110A - Imaging device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a user precisely recognize at which position a current subject image is located in a panoramic image that is an imaging target during panoramic imaging.SOLUTION: An image processing section 17 applies image processing to a photographic image of a subject inputted from an imaging section 16, so as to sequentially create image data of plural frames. An image synthesizing section 52 acquires the image data of one frame created by the image processing section 17 for every inputted image data acquisition command in photographing a panoramic image, and synthesizes the image data of one currently acquired frame with the image data of one or more frames acquired in the previous acquisition command, so as to create a panoramic intermediate image indicating a region photographed until now among panoramic images scheduled to be created. A display control section 54 displays the image of each frame created by the image processing section 17 and the panoramic intermediate image created by the image synthesizing section 52 together on a display section 19.

Description

  The present invention relates to an imaging apparatus, method, and program, and more particularly, to an imaging apparatus, method, and program capable of panoramic imaging.

Conventionally, there is a panoramic imaging function as one function of an imaging apparatus.
The function is to move the digital camera in the horizontal direction while maintaining the state in which the user presses the shutter switch, and the imaging device executes a plurality of imaging processes in the meantime, and each of the plurality of imaging processes is performed. This is a function for generating image data of a panoramic image by combining a plurality of image data obtained as a result in the horizontal direction (horizontal direction).

  In Patent Document 1, a digital camera displays a horizontal frame superimposed on an image currently being captured, and uses a bar extending horizontally in the horizontal frame to perform panoramic imaging. A technique for presenting a progress state (hereinafter referred to as “the technique of Patent Document 1”) is disclosed.

Japanese Patent Laid-Open No. 06-303562

However, in the case of the technique disclosed in Patent Document 1, the user can check the history of imaging so far during imaging, for example, what kind of images have been captured since the start of panoramic imaging. Can not.
For this reason, there are cases where necessary images are missing from the panorama image recorded as image data after the end of panorama imaging, or conversely, there are cases where extra images are mixed, and the user can select a desired image. In order to obtain a panoramic image, it was necessary to repeat the imaging operation many times.

  The present invention has been made in view of such a situation, and an object of the present invention is to allow a user to easily view a history of imaging so far during panoramic imaging.

  In order to achieve the above object, the invention of claim 1 is an imaging apparatus comprising imaging means for sequentially outputting each of image data sequentially obtained by imaging at predetermined time intervals as frame image data. Image synthesizing means for sequentially generating image data of the synthesized image by sequentially obtaining the image data of the frames output from the imaging means and sequentially synthesizing at least a part of the sequentially obtained image data of the plurality of frames. And display control means for executing control to display the composite image sequentially generated as image data by the image composition means while displaying the frames sequentially output as image data from the imaging means. It is an imaging device provided with.

  The invention according to claim 2 is characterized in that, in the invention according to claim 1, the display control means executes control to superimpose and display the composite image on the frame. To do.

  The invention according to claim 3 is the invention according to claim 1 or 2, further comprising first reduction means for reducing the image data sequentially output from the imaging means, and the image composition. The means sequentially generates data of the reduced composite image by using the image data reduced by the first reduction means, and the display control means includes the frame and the reduced image data. Control for displaying the composite image is executed.

  According to a fourth aspect of the invention, there is provided the second reduction means for reducing the image data sequentially generated by the image synthesizing means in the invention according to any one of the first to third aspects. Further, the display control means executes control for displaying the frame and the reduced composite image.

  According to a fifth aspect of the present invention, in the invention according to any one of the first to fourth aspects of the present invention, the image synthesizing unit further performs the recording of the synthetic image for recording when a predetermined condition is satisfied. It further comprises recording control means for executing control for generating image data and recording image data of the composite image generated for recording by the image composition means.

  In order to achieve the above object, the invention described in claim 6 includes an imaging unit that sequentially outputs each of the image data obtained sequentially by imaging at predetermined time intervals as frame image data. In the imaging method executed by the apparatus, the image data of the frames output from the imaging unit are sequentially acquired, and at least a part of the sequentially acquired image data of the plurality of frames is sequentially combined, thereby Control for simultaneously displaying an image composition step for sequentially generating image data, the frame sequentially output as image data from the imaging unit, and the composite image sequentially generated as image data by the processing of the image composition step. And a display control step to be executed.

  In order to achieve the above object, an invention according to claim 7 is an imaging apparatus comprising imaging means for sequentially outputting each of image data sequentially obtained by imaging at predetermined time intervals as frame image data. The image data of the composite image is obtained by sequentially acquiring the image data of the frames output from the imaging unit in a controlling computer, and sequentially synthesizing at least a part of the sequentially acquired image data of the plurality of frames. Display that executes control for simultaneously displaying an image composition function that is sequentially generated, the frame that is sequentially output as image data from the imaging unit, and the composite image that is sequentially generated as image data by the use of the image composition function It is a program for realizing a control function.

  According to the present invention, the user can easily view the history of imaging so far during panoramic imaging.

1 is a block diagram illustrating a hardware configuration of a digital camera as an embodiment of an imaging apparatus according to the present invention. FIG. 2 is a functional block diagram illustrating a functional configuration for the digital camera of FIG. 1 to execute an imaging process. It is a figure explaining imaging operation in case the normal imaging mode and the panorama imaging mode are each selected as an operation mode of the digital camera of FIG. It is a figure which shows an example of the panoramic image produced | generated by the panoramic imaging mode shown in FIG. FIG. 2 is a back view showing an external configuration of the digital camera of FIG. 1 on which a panoramic image is displayed. It is a figure explaining the synthetic | combination process of the image process part of FIG. It is a flowchart which shows an example of the flow of an imaging process. It is a flowchart which shows the detailed flow of a panoramic imaging process among the imaging processes of FIG.

  Embodiments according to the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a hardware configuration of a digital camera 1 as an embodiment of an imaging apparatus according to the present invention.

  The digital camera 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an optical system 15, an imaging unit 16, and an image processing unit 17. A storage unit 18, a display unit 19, an operation unit 20, a communication unit 21, an angular velocity sensor 22, and a drive 23.

The CPU 11 executes various processes according to a program stored in the ROM 12 or a program loaded from the storage unit 18 to the RAM 13.
The ROM 12 also appropriately stores data necessary for the CPU 11 to execute various processes.

For example, in the present embodiment, programs that realize the functions of the imaging control unit 51 to the display control unit 54 in FIG. 2 to be described later are stored in the ROM 12 and the storage unit 18. Therefore, when the CPU 11 executes processing according to these programs, each function of the imaging control unit 51 to the display control unit 54 in FIG. 2 described later can be realized.
Note that at least some of the functions of the imaging control unit 51 to the display control unit 54 in FIG. 2 to be described later can be transferred to the image processing unit 17.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. Also connected to the bus 14 are an optical system 15, an imaging unit 16, an image processing unit 17, a storage unit 18, a display unit 19, an operation unit 20, a communication unit 21, an angular velocity sensor 22, and a drive 23.

  The optical system 15 is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject. The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor of the imaging unit 16. The zoom lens is a lens that freely changes the focal length within a certain range. The optical system 15 is also provided with a peripheral device that adjusts the focus, exposure, and the like as necessary.

The imaging unit 16 includes a photoelectric conversion element, AFE (Analog Front End), and the like. The photoelectric conversion element is composed of, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. The photoelectric conversion element photoelectrically converts (captures) an optical signal of a subject image incident and accumulated during a certain time interval, and sequentially supplies an analog electric signal obtained as a result to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog electric signal, and outputs a digital signal obtained as a result as an output signal of the imaging unit 16.
Hereinafter, the output signal of the imaging unit 16 is referred to as “image data of a captured image”. Therefore, the image data of the captured image is output from the imaging unit 16 and is supplied to the image processing unit 17 and the like as appropriate.

The image processing unit 17 includes a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), and the like.
The image processing unit 17 cooperates with the CPU 11 to perform image processing such as noise reduction, white balance, and camera shake correction on the image data of the captured image input from the imaging unit 16.
Here, the image data of the captured image input from the imaging unit 16 at regular intervals is hereinafter referred to as “frame image data”. In this embodiment, the frame is adopted as a processing unit. That is, the image processing unit 17 outputs the image data of the frame supplied from the imaging unit 16 after performing various image processing.

  The storage unit 18 is configured by a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores image data of a frame output from the image processing unit 17, image data of a panoramic intermediate image to be described later, and the like. The storage unit 18 also stores various data necessary for various image processing.

  The display unit 19 is configured as a flat display panel including, for example, an LCD (Liquid Crystal Device) and an LCD driving unit. The display unit 19 displays an image expressed by image data supplied from the storage unit 18 or the like, for example, a through image described later in units of frames.

  In addition to the shutter switch 41, the operation unit 20 has a plurality of switches such as a power switch, an imaging mode switch, and a reproduction switch (not shown). When a predetermined switch among these plural switches is pressed, the operation unit 20 supplies a command assigned to the predetermined switch to the CPU 11.

  The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  The angular velocity sensor 22 is composed of a gyro or the like, detects the amount of angular displacement of the digital camera 1, and supplies a digital signal indicating the detection result (hereinafter simply referred to as “angle displacement amount”) to the CPU 11. Note that the angular velocity sensor 22 has a function of detecting the direction of geomagnetism, thereby exhibiting the function of an azimuth sensor as necessary.

  A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 23. Then, the program read from the removable medium 31 is installed in the storage unit 18 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 18 in the same manner as the storage unit 18.

  FIG. 2 shows a series of processes (hereinafter referred to as “imaging process”) in the process executed by the digital camera 1 in FIG. 1 from capturing an image of a subject and recording image data of the captured image obtained as a result on the removable medium 31. It is a functional block diagram showing a functional configuration for executing

The CPU 11 is provided with an imaging control unit 51 that controls execution of imaging processing.
For example, the imaging control unit 51 can selectively switch between the normal imaging mode and the panoramic imaging mode as the operation mode of the digital camera 1, and can execute processing according to the operation mode after switching.

FIG. 3 is a diagram illustrating an imaging operation when the normal imaging mode and the panoramic imaging mode are selected as the operation modes of the digital camera 1 in FIG. 1.
Specifically, FIG. 3A is a diagram illustrating an imaging operation in the normal imaging mode. FIG. 3B is a diagram illustrating the imaging operation in the panoramic imaging mode.
In each of FIG. 3A and FIG. 3B, the picture in the back of the digital camera 1 shows the real world including the subject of the digital camera 1. Also, the vertical dotted lines shown in FIG. 3B indicate the positions a, b, and c in the moving direction of the digital camera 1. The moving direction of the digital camera 1 refers to the direction in which the optical axis of the digital camera 1 moves when the user changes the imaging direction (angle) of the digital camera 1 about its own body.

The normal imaging mode refers to an operation mode when imaging an image having a size (resolution) corresponding to the angle of view of the digital camera 1.
In the normal imaging mode, as shown in FIG. 3A, the user presses the shutter switch 41 of the operation unit 20 to the lower limit while the digital camera 1 is fixed. The operation of pressing the shutter switch 41 to the lower limit in this way is hereinafter referred to as “full press operation” or simply “full press”.
The imaging control unit 51 controls execution of a series of processing until the image data of the frame output from the image processing unit 17 immediately after the full-press operation is performed is recorded on the removable medium 31 as a recording target.
Hereinafter, a series of processes executed under the control of the imaging control unit 51 in the normal imaging mode is referred to as “normal imaging process”.

On the other hand, the panorama image capturing mode refers to an operation mode for capturing a panoramic image.
In the panoramic imaging mode, as shown in FIG. 3B, the user moves the digital camera 1 in the direction of the black arrow in the figure while maintaining the full pressing operation of the shutter switch 41.
The imaging control unit 51 controls the image composition unit 52 while the full-press operation is maintained, and immediately after the angular displacement amount from the angular velocity sensor 22 reaches a certain value, the image processing unit 17 outputs it immediately after that. Repeatedly storing the image data of the frame that has been temporarily stored in the storage unit 18.
Thereafter, the user instructs the end of panorama imaging by performing an operation of releasing the full-press operation, that is, an operation of releasing a finger or the like from the shutter switch 41 (hereinafter, such an operation is referred to as a “release operation”). .
When the end of panoramic imaging is instructed, the imaging control unit 51 combines the image data of a plurality of frames stored so far in the storage unit 18 in the horizontal direction in the stored order, thereby generating a panoramic image. Generate image data.
Then, the imaging control unit 51 records the image data of the panoramic image on the removable medium 31 as a recording target.
In this manner, the imaging control unit 51 controls a series of processes until generating image data of a panoramic image and recording it on the removable medium 31 as a recording target in the panoramic imaging mode.
Hereinafter, a series of processes executed under the control of the imaging control unit 51 in the panoramic imaging mode is referred to as “panoramic imaging processing”.

FIG. 4 shows image data of a panoramic image obtained as a result of the panoramic imaging process.
That is, in the panorama imaging mode, when an imaging operation as shown in FIG. 3B is performed, the imaging control unit 51 controls execution of the panorama imaging process. As a result, image data of a panoramic image P3 as shown in FIG. 4 is generated and recorded on the removable medium 31.

Further, in the present embodiment, the imaging control unit 51 of FIG. 2 executes through imaging processing and through display processing regardless of the normal imaging mode and the panoramic imaging mode.
That is, the user presses the shutter switch 41 of the operation unit 20 halfway (predetermined position not reaching the lower limit) in order to cause the digital camera 1 to execute AF (Auto Focus) processing or the like before performing the full pressing operation. The operation can be performed. The operation of pressing down to the middle of the shutter switch 41 (a predetermined position that does not reach the lower limit) is hereinafter referred to as “half-press operation” or simply “half-press”.
When the halfway pressing operation is performed, the imaging control unit 51 controls the imaging unit 16 and the image processing unit 17 to continue the imaging operation by the imaging unit 16. The imaging control unit 51 stores image data of frames sequentially output from the image processing unit 17 via the imaging unit 16 while the imaging operation by the imaging unit 16 is continued. Unit 18) for temporary storage. Such a series of control processing by the imaging control unit 51 is referred to as “through imaging processing”.
Further, the imaging control unit 51 controls the display control unit 54 to sequentially read out each image data temporarily recorded in the memory (the storage unit 18 in the present embodiment) at the time of through imaging, and to correspond to each frame. Images are sequentially displayed on the display unit 19. Such a series of control processing by the imaging control unit 51 is referred to as “through display processing”. The frame image displayed on the display unit 19 by the through display process is hereinafter referred to as “through image”.

  In the present embodiment, when a full-press operation is performed in a state where the panorama imaging mode is selected, an image showing an area captured up to now among the panorama images to be generated (hereinafter referred to as “panorama halfway image”). Is displayed on the display unit 19 superimposed on the through image.

FIG. 5 is a back view showing an external configuration of the digital camera 1 in FIG. 1 on which a panoramic image is displayed.
FIG. 5A shows the external configuration of the digital camera 1 when it is moved to the position a in FIG. FIG. 5B shows an external configuration of the digital camera 1 when it is moved to the position b in FIG. FIG. 5C shows the external configuration of the digital camera 1 when it is moved to the position c in FIG.
That is, the transition of the display state of the digital camera 1 when the user sequentially moves the digital camera 1 to the positions a to c in the panoramic imaging mode is shown in FIGS. 5 (A) to 5 (C). Has been.
Specifically, when the digital camera 1 sequentially moves to a position a, a position b, and a position c, the through images PA1, PB1, and PC1 correspond to the positions a, b, and c on the display unit 19, respectively. Displayed sequentially. In this embodiment, each of the panoramic images PA2, PB2, and PC2 is displayed so as to be superimposed on each of the through images PA1, PB1, and PC1.
Since the user can perform panoramic imaging operation while visually recognizing such panoramic images PA2, PB2, and PC2, it is possible to easily and accurately visually recognize the imaging history from the start of panoramic imaging to the present. Can do. As a result, the user can easily and easily obtain a desired panoramic image without repeating the imaging operation many times.

As shown in FIG. 2, the CPU 11 of the digital camera 1 further includes an image composition unit 52, an image reduction unit 53, A display control unit 54 is provided.
Note that, as described above, the functions of the image composition unit 52, the image reduction unit 53, and the display control unit 54 are not particularly required to be mounted on the CPU 11 as in the present embodiment. It is also possible to transfer at least a part to the image processing unit 17.

The image composition unit 52 executes a process of sequentially synthesizing the frame image data sequentially supplied from the image processing unit 17 in the horizontal direction in the supplied order (hereinafter referred to as “image composition process”). . As a result of the image synthesis process, image data of the panoramic intermediate image is obtained and supplied to the image reduction unit 53.
Specifically, for example, the imaging control unit 51 monitors the detection result of the angular velocity sensor 22 and issues an acquisition command to the image composition unit 52 every time the cumulative value of the angular displacement amount reaches a certain value. The image compositing unit 52 acquires the image data of the frame supplied from the image processing unit 17 immediately after receiving the acquisition command as a compositing target. That is, the image compositing unit 52 sequentially acquires the image data of the compositing target frame each time an acquisition command is issued from the imaging control unit 51 after the full pressing operation of the shutter switch 41 is started.
Then, the image composition unit 52 sequentially synthesizes the image data of a plurality of frames acquired as a composition target from the start of the full pressing operation of the shutter switch 41 to the present in the horizontal direction in the order of acquisition. .
As a result of such image synthesis processing, image data of the panoramic intermediate image is obtained by the image synthesis unit 52 and supplied to the image reduction unit 53.

Here, the image data to be subjected to the image composition processing is not particularly limited. For example, the image data of each of a plurality of frames acquired from the start of the full pressing operation of the shutter switch 41 to the present. There may be. However, in the present embodiment, the image data of the panoramic intermediate image generated during the previous combining process and the image data of the frame acquired as the combining target immediately before are used as the targets of the combining process.
That is, in the present embodiment, the size (resolution) of the panoramic intermediate image generated by the image composition unit 52 is set to the same fixed size as the maximum size of the panoramic image that can be finally generated.
For this reason, during panoramic imaging, that is, while the digital camera 1 is moving, even if several frames of image data acquired so far are combined in the horizontal direction, the horizontal size is the maximum size. In other words, an area that cannot be filled with the effective image of the frame appears. In the present embodiment, such a region is a region in which a predetermined color (for example, black in the example of FIG. 5) is uniformly applied. Such an area, that is, an area in which the pixel values are all constant (for example, constant at 0 indicating black) is hereinafter referred to as a “blank area”.
Therefore, in the present embodiment, the image compositing unit 52 uses the image data of the previous panoramic intermediate image and the image data of the frame acquired immediately before as the compositing target, to the blank area of the previous panoramic intermediate image. The image composition process for overwriting the frame is executed.

FIG. 6 is a diagram for explaining the composition processing of the image composition unit 52 in FIG.
In the present embodiment, it is assumed that a storage area 101 as shown in FIG. 6 is provided in the storage unit 18, for example.
The storage area 101 is divided into N frame storage areas 101-1 to 101 -N. Here, N is an arbitrary integer value of 2 or more. However, the size of the frame storage areas 101-1 to 101-N in the horizontal direction (the x-axis direction in the figure) is the horizontal size (resolution) of the panoramic intermediate image, that is, the horizontal level of the panoramic image that can be finally generated. It is the same as the maximum size of the direction. N corresponds to the maximum number of image data of frames that can be acquired by the image composition unit 52 as a composition target in one panorama imaging process.
In other words, when the K-th acquisition command is issued after the panoramic imaging process is started after the shutter switch 41 is fully pressed, at least one of the frame image data acquired as a synthesis target by the image synthesis unit 52. Are stored in the frame storage area 101-K.
Here, the reason why it is described as at least part of the image data of the frame is that the size of the image data of the frame may be larger than the size of the frame storage area 101-K. In such a case, only image data corresponding to a predetermined part of the frame is stored in the frame storage area 101-K.

As described above, in this embodiment, the entire image data stored in the storage area 101 is image data of a panoramic intermediate image. In the storage area 101, the same pixel value (for example, 0 indicating black) is uniformly stored as an initial state. That is, in the initial state, all of the N frame storage areas 101-1 to 101-N are blank areas.
Therefore, at the timing acquired by the image composition unit 52 for the K-th synthesis target frame, out of the N frame storage areas 101-1 to 101 -N, the frame storage areas 101-1 to 101-(K−1). ) Includes image data of an effective image. Here, the effective image refers to an image in which (at least a part) of each of the first to (K-1) th frames to be combined is combined in that order in the horizontal direction (x-axis direction in FIG. 6). . On the other hand, the frame storage areas 101-K to 101-N are blank areas.
Among the frame storage areas 101-K to 101-N, which are blank areas, at least a part of the image data of the frame acquired as the synthesis target for the Kth time is stored in the frame storage area 101-K. As a result, the entire data stored in the storage area 101 is generated as image data of a panoramic intermediate image.
2 reads out the entire data stored in the storage area 101 as image data of a panoramic intermediate image and supplies it to the image reduction unit 53.

Each time the panoramic image data is supplied from the image composition unit 52, the image reduction unit 53 performs a reduction process for reducing the size (resolution) of the image data. Here, the reduction ratio can be arbitrarily set within a range in which the panoramic halfway image after reduction is smaller than the through image (frame).
The image data of the panoramic intermediate image reduced in size (resolution) in this way is supplied from the image reduction unit 53 to the display control unit 54.

The display control unit 54 performs control for causing the display unit 19 to display a frame represented by the image data temporarily stored in the storage unit 18 as a through image.
Further, during the panorama imaging process, the display control unit 54 superimposes and displays the image represented by the reduced image data supplied from the image reduction unit 53, that is, the reduced panoramic image on the through image. Control to be displayed on the unit 19 is executed.

Thereafter, when the user's finger or the like is released from the shutter switch 41, that is, when a release operation is performed, the imaging control unit 51 executes control for recording the image data to be recorded on the removable medium 31.
Specifically, in the normal imaging mode, the imaging control unit 51 executes control for recording the image data of the frame output from the image processing unit 17 on the removable medium 31 as image data to be recorded.
On the other hand, in the case of the panoramic imaging mode, the imaging control unit 51 performs control for recording the image data of the panoramic image on the removable media 31 as the image data to be recorded via the image synthesis unit 52.
For example, the image composition unit 52 reads out image data of a portion of the effective image other than the blank area from the data stored in the storage area 101 (FIG. 6), and appropriately performs image processing such as reduction processing and enlargement processing. Apply. As a result, image data of a panoramic image to be recorded is generated. The image data of the panoramic image generated in this way is supplied to the drive 23 and recorded on the removable medium 31 as image data to be recorded.

  Next, with reference to FIG. 7, an imaging process executed by the digital camera 1 having such a functional configuration will be described.

FIG. 7 is a flowchart illustrating an example of the flow of imaging processing.
In the present embodiment, the imaging process starts when a power supply (not shown) of the digital camera 1 is turned on.

In step S1, the imaging control unit 51 in FIG. 2 executes an operation detection process and an initial setting process.
The operation detection process is a process for detecting the state of each switch of the operation unit 20. The imaging control unit 51 can detect whether the normal imaging mode or the panoramic imaging mode is set as the operation mode by executing the operation detection process.
In addition, as one of the initial setting processes of the present embodiment, a process of setting a constant value of the angular displacement amount and a threshold value (for example, 360 degrees) that is the maximum limit of the angular displacement amount is employed.
Specifically, a constant value of the angular displacement amount and a threshold value (for example, 360 degrees) that is the maximum limit of the angular displacement amount are stored in advance in the ROM 12 of FIG. 1 and read from the ROM 12 and written in the RAM 13. Is set. The constant value of the angular displacement amount is used in the determination process in step S35 of FIG. On the other hand, a threshold (for example, 360 degrees) that is the maximum limit of the amount of angular displacement is used in the determination process in step S45 in FIG.
In this embodiment, as shown in steps S34 and S40 in FIG. 8 to be described later, the angular displacement amount detected by the angular velocity sensor 22 is cumulatively added, and the cumulative angular displacement amount as the cumulative added value or the total amount is added. The amount of angular displacement (the difference between the two will be described later) is stored in the RAM 13. Therefore, a process of resetting the accumulated angular displacement amount and the total angular displacement amount to 0 is adopted as one of the initial setting processes of the present embodiment. Note that the accumulated angular displacement amount is compared with the above-described constant value in the determination process in step S35 of FIG. On the other hand, the total angular displacement amount is compared with the above-described constant value in the determination process in step S45 of FIG.
Furthermore, a process for resetting the error flag to 0 is employed as one of the initial setting processes of the present embodiment. The error flag is a flag that is set to 1 when an error occurs during the panoramic imaging process (see step S44 in FIG. 8 described later).

In step S2, the imaging control unit 51 determines whether or not the shutter switch 41 is half-pressed.
If the shutter switch 41 is not half-pressed, it is determined as NO in Step S2, and the process proceeds to Step S12.

In step S12, the imaging control unit 51 determines whether an instruction to end the process has been given.
The processing end instruction is not particularly limited, but in the present embodiment, it is assumed that a notification that a power source (not shown) of the digital camera 1 has been turned off is employed.
Therefore, in the present embodiment, when the power is turned off and the imaging control unit 51 is notified to that effect, it is determined as YES in Step S12, and the entire imaging process ends.
On the other hand, when the power source is in the on state, no notification that the power source is in the off state is not made, so that it is determined as NO in step S12, and the process returns to step S2, and thereafter The process is repeated. That is, in this embodiment, as long as the power is kept on, the loop processing of Step S2: NO and Step S12: NO is repeatedly executed until the shutter switch 41 is half-pressed, and the imaging processing is performed. It will be in a standby state.

  If the shutter switch 41 is half-pressed during this standby state, it is determined as YES in Step S2, and the process proceeds to Step S3.

In step S3, the imaging control unit 51 starts the above-described through imaging process and through display process.
Thereby, the through image is displayed on the display unit 19. In the present embodiment, it is assumed that the through image is continuously displayed on the display unit 19 until the normal imaging process in step S8 or the panorama imaging process in step S9, which will be described later, is completed.

  In step S4, the imaging unit 16 is controlled to perform so-called AF processing.

In step S5, the imaging control unit 51 determines whether or not the shutter switch 41 is fully pressed.
If the shutter switch 41 is not fully pressed, it is determined NO in step S5. In this case, the process returns to step S4, and the subsequent processes are repeated. That is, in this embodiment, until the shutter switch 41 is fully pressed, the loop process of step S4 and step S5: NO is repeatedly executed, and the AF process is executed each time.

Thereafter, when the shutter switch 41 is fully pressed, it is determined as YES in Step S5, and the process proceeds to Step S6.
In step S6, the imaging control unit 51 determines whether or not the currently set imaging mode is the panorama imaging mode.
If it is not the panoramic imaging mode, that is, if the normal imaging mode is currently set, it is determined NO in step S6, and the process proceeds to step S7.
In step S7, the imaging control unit 51 executes the normal imaging process described above.
That is, one frame of image data output from the image processing unit 17 immediately after the full pressing operation is performed is recorded on the removable medium 31 as a recording target. Thereby, the normal imaging process of step S8 is completed, and the process proceeds to step S12. In addition, since the process after step S12 was mentioned above, the description is abbreviate | omitted here.

On the other hand, when the panoramic imaging mode is currently set, it is determined as YES in Step S6, and the process proceeds to Step S8.
In step S8, the imaging control unit 51 executes the panorama imaging process described above.
Although details of the panorama imaging process will be described later with reference to FIG. 8, in principle, image data of a panorama image is generated and recorded on the removable medium 31 as a recording target. Thereby, the panorama imaging process of step S8 is completed, and the process proceeds to step S9.

In step S9, the imaging control unit 51 determines whether or not the error flag is 1.
Although details will be described later with reference to FIG. 8, when the panorama image data is recorded on the removable medium 31 as a recording target and the panorama imaging process in step S8 is properly completed, the error flag is set to 0. Yes. In such a case, it is determined as NO in Step S10, and the process proceeds to Step S12. In addition, since the process after step S12 was mentioned above, the description is abbreviate | omitted here.
On the other hand, if any error occurs during the panorama image capturing process in step S8, the panorama image capturing process ends inappropriately. In such a case, since the error flag is 1, it is determined as YES in Step S9, and the process proceeds to Step S10.

In step S <b> 10, the imaging control unit 51 displays the error content on the display unit 19. Specific examples of the displayed error contents will be described later.
In step S11, the imaging control unit 51 cancels the panoramic imaging mode and resets the error flag to 0.
Thereafter, the process returns to step S1, and the subsequent processes are repeated. That is, the imaging control unit 51 prepares for the next new imaging operation by the user.

The flow of the imaging process has been described above with reference to FIG.
Next, with reference to FIG. 8, the detailed flow of the panorama imaging process of step S9 among the imaging processes of FIG. 7 will be described.

FIG. 8 is a flowchart for explaining the detailed flow of the panoramic imaging process.
As described above, when the shutter switch 41 is fully pressed in the panoramic imaging mode, if YES is determined in step S6 in FIG. 7, the process proceeds to step S8, and the following process is performed as the panoramic imaging process. Is executed.

  That is, in step S <b> 31 of FIG. 8, the imaging control unit 51 acquires the angular displacement amount from the angular velocity sensor 22.

In step S32, the imaging control unit 51 determines whether or not the angular displacement amount acquired in the process of step S31 is greater than zero.
In a state where the user has not moved the digital camera 1, the amount of angular displacement is 0. Therefore, NO is determined in step S32, and the process proceeds to step S33.

In step S33, the imaging control unit 51 determines whether the continuation of the angular displacement amount 0 has elapsed for a predetermined time. As the predetermined time, for example, an appropriate time longer than the time required from when the user fully presses the shutter switch 41 until the digital camera 1 starts moving can be used.
If the predetermined time has not elapsed, it is determined as NO in Step S33, the process returns to Step S31, and the subsequent processes are repeated. That is, when the duration of the state in which the user does not move the digital camera 1 is shorter than the predetermined time, the imaging control unit 51 performs panorama imaging by repeatedly executing the loop processing of step S31 to step S33: NO. Put the process in a standby state.

  If the user moves the digital camera 1 during this standby state, the amount of angular displacement from the angular velocity sensor 22 becomes a value greater than zero. In such a case, it is determined as YES in Step S32, and the process proceeds to Step S34.

  In step S34, the imaging control unit 51 updates the cumulative angular displacement amount by adding the angular displacement amount acquired in the process of step S32 to the cumulative angular displacement amount so far (cumulative angular displacement amount = Cumulative angular displacement + angular displacement). That is, the value stored as the accumulated angular displacement amount in the RAM 13 is updated.

The cumulative angular displacement amount is a value obtained by cumulatively adding the angular displacement amounts as described above, and indicates the movement amount of the digital camera 1.
Here, in this embodiment, every time the user moves the digital camera 1 by a certain amount, one frame of image data (compositing target) for generating image data of the panoramic intermediate image is transferred from the image processing unit 17 to the image combining unit 52. Shall be supplied.
In order to realize this, the cumulative angular displacement amount corresponding to the “constant amount” as the movement amount of the digital camera 1 is previously given as the “constant value” by the initial setting process in step S1 of FIG.
That is, in this embodiment, every time the cumulative angular displacement amount reaches a certain value, one frame of image data (composition target) is supplied from the image processing unit 17 to the image composition unit 52, and the cumulative angular displacement amount is 0. Reset to.
Such a series of processing is executed as processing after the next step S35.

That is, in step S35, the imaging control unit 51 determines whether or not the cumulative angular displacement amount has reached a certain value.
If the cumulative angular displacement amount has not reached a certain value, it is determined as NO in Step S35, the process returns to Step S31, and the subsequent processes are repeated. That is, as long as the user moves the digital camera 1 by a certain amount and the cumulative angular displacement does not reach a certain value, the imaging control unit 51 repeatedly executes the loop processing of steps S31 to S35.
After that, when the user moves the digital camera 1 by a certain amount and the accumulated angular displacement reaches a certain value, it is determined as YES in Step S35, and the process proceeds to Step S36.

In step S <b> 36, the image composition unit 52 acquires one frame image data from the image processing unit 17 under the control of the imaging control unit 51.
That is, the imaging control unit 51 issues an acquisition command to the image composition unit 52 when the process proceeds to step S36 assuming that the accumulated angular displacement amount has reached a certain value.
Receiving this acquisition instruction, the image composition unit 52 acquires one frame of image data from the image processing unit 17 as the process of step S36.

In step S <b> 37, the image composition unit 52 generates image data of the panoramic intermediate image by overwriting the acquired one frame of image data (at least a part thereof) on the blank area of the storage area 101 (FIG. 6). To do.
When the image data of the panoramic image is supplied from the image composition unit 52 to the image reduction unit 53, the process proceeds to step S38.

In step S38, the image reducing unit 53 reduces the size (resolution) of the image data of the supplied panoramic intermediate image.
When the image data of the reduced panoramic image is supplied from the image reduction unit 53 to the display control unit 54, the process proceeds to step S39.

  In step S <b> 39, the display control unit 54 causes the display unit 19 to display an image represented by the image data supplied from the image reduction unit 53, that is, a panoramic intermediate image after reduction, superimposed on the through image.

  In step S40, the imaging control unit 51 updates the total angular displacement amount by adding the current angular displacement amount (= substantially constant value) to the previous total angular displacement amount (total angular displacement amount). = Total angular displacement + cumulative angular displacement). That is, the value stored as the total angular displacement amount in the RAM 13 is updated.

  In step S41, the imaging control unit 51 resets the accumulated angular displacement amount to zero. That is, the value stored as the accumulated angular displacement amount in the RAM 13 is updated to 0.

As described above, the accumulated angular displacement amount is used to control the timing at which one frame of image data (compositing target) is supplied from the image processing unit 17 to the image synthesizing unit 52, that is, the acquisition command issuance timing. Therefore, the cumulative angular displacement amount is reset to 0 every time it reaches a constant value and an acquisition command is issued.
Therefore, the imaging control unit 51 cannot recognize how far the digital camera 1 has moved from the start of the panorama imaging process to the present even if the accumulated angular displacement amount is used.
Therefore, in order to enable such recognition, in the present embodiment, the total angular displacement amount is employed separately from the cumulative angular displacement amount.
In other words, the total angular displacement amount is a value obtained by cumulatively adding the angular displacement amount, but is not reset to 0 even when reaching a certain amount, and until the panoramic imaging process is completed (details will be described later). It is a value that continues to be cumulatively added (until the process of step S47 is executed).

  In this way, when the total angular displacement is updated in the process of step S40, and when the cumulative angular displacement is reset to 0 in the process of step S41, the process proceeds to step S42.

In step S42, the imaging control unit 51 determines whether a release operation has been performed.
If the release operation is not performed, that is, if the shutter switch 41 is fully pressed by the user, it is determined as NO in Step S42, and the process proceeds to Step S43.

In step S43, the imaging control unit 51 determines whether an image acquisition error has occurred.
The image acquisition error is not particularly limited. For example, in the present embodiment, the digital camera 1 is moved as a certain amount in an oblique direction, an up-down direction, or a reverse direction.

If no error has occurred in image acquisition, it is determined as NO in step S43, and the process proceeds to step S44.
In step S44, the imaging control unit 51 determines whether or not the total angular displacement exceeds a threshold value.
As described above, the total angular displacement amount is a cumulative added value of the angular displacement amount from the start of the panoramic imaging process (after the full pressing operation is performed) to the time point when the process of step S42 is executed. .
Here, in this embodiment, the maximum amount of movement by which the user can move the digital camera 1 during panoramic imaging is determined in advance. The total angular displacement amount corresponding to the “maximum movement amount” as the movement amount of the digital camera 1 is given in advance as the “threshold value” by the initial setting process in step S1 of FIG.
Thus, in this embodiment, that the total angular displacement amount has reached the threshold value means that the digital camera 1 has moved by the maximum movement amount.
Therefore, if the total amount of angular displacement has not reached the threshold value, that is, if the amount of movement of the digital camera 1 has not reached the maximum amount of movement, the user can continue to move the digital camera 1, and therefore NO in step S44. Is determined, the process is returned to step S31, and the subsequent processes are repeated.
In other words, if one of the errors includes the fact that the continuation of the angular displacement amount 0 has elapsed for a predetermined time (the digital camera 1 does not move for a predetermined time), as long as the full-press operation is continued in a state where no error occurs. The loop processing of steps S31 to S44 is repeatedly executed.
During this time, as described above with reference to FIG. 5, the display of the through image is updated and the display of the panoramic image is also updated in accordance with the movement of the digital camera 1.
Since the user can operate panoramic imaging while visually recognizing the continuously updated panoramic image in this way, it is possible to easily and accurately visually recognize the imaging history from the start of panoramic imaging to the present. . As a result, the user can easily and easily obtain a desired panoramic image without repeating the imaging operation many times.

Thereafter, in a state where no error occurs, whether the release operation is performed (YES in the process of step S42) or the digital camera 1 has moved to the maximum movement amount (YES in the process of step S45). If it is determined, the process proceeds to step S46.
In step S <b> 46, the imaging control unit 51 generates image data of a panoramic image via the image composition unit 52 and records it on the removable medium 31 as image data to be recorded.

In step S47, the imaging control unit 51 resets the total angular displacement amount to zero.
As a result, the panorama imaging process ends properly. That is, the process of step S8 in FIG. 7 ends properly, and it is determined as NO in the next process of step S9. In addition, since the process after it determines with it being NO by the process of step S9 was mentioned above, the description is abbreviate | omitted here.

If any error occurs during the series of processes described above, that is, if it is determined as YES in the process of step S33, or if it is determined as YES in the process of step S43, the process is as follows. Proceed to step S44.
In step S44, the imaging control unit 51 sets the error flag to 1.
In this case, the process of step S46 is not executed, that is, the image data of the panorama image is not recorded, and the panorama imaging process ends improperly.
That is, the process of step S8 in FIG. 7 ends improperly, it is determined that the next step S9 is YES, and the error content is displayed in the process of step S10.
The display of the error content in this case is not particularly limited as described above. For example, a message display such as “Image acquisition failed” or “Time is over” can be employed.

As described above, the digital camera 1 of the present embodiment includes the imaging unit 16 that sequentially outputs each of the image data sequentially obtained by imaging at predetermined time intervals as frame image data.
The digital camera 1 of the present embodiment also includes an image composition unit 52 and a display control unit 54.
Each time the digital camera 1 moves by a predetermined amount (every time the cumulative angular displacement amount reaches a certain value), the image composition unit 52 converts the image data of the frame output from the imaging unit 16 to the image processing unit 17. And stored in the storage unit 18. Then, the image synthesis unit 52 sequentially generates image data of the panoramic intermediate image (synthesized image) by synthesizing at least a part of the image data of the plurality of frames accumulated in the storage unit 18 so far. To do.
The display control unit 54 controls to simultaneously display frames (that is, through images) that are sequentially output as image data from the imaging unit 16 and panoramic intermediate images (synthesized images) that are sequentially generated as image data by the image combining unit 52. Execute.
Thereby, as described above with reference to FIG. 5, the display of the through image is updated according to the movement of the digital camera 1, and the display of the panoramic intermediate image (composite image) is also updated.
Since the user can operate panorama imaging while visually recognizing the continuously updated panoramic image (composite image) in this way, the imaging history from the start of panorama imaging to the present can be easily and accurately viewed. can do. As a result, the user can easily and easily obtain a desired panoramic image without repeating the imaging operation many times.

In particular, in the digital camera 1 according to the present embodiment, the display control unit 54 performs control for superimposing a panoramic intermediate image (composite image) on the through image and causing the display unit 19 to display it.
In this case, since the user can view the through image and the panoramic image at the same time, the user can see what the subject is currently captured from the through image, and the subject is included in which part of the panoramic image. It becomes possible to visually recognize whether or not.

The digital camera 1 of the present embodiment further includes an image reduction unit 53 that reduces the image data of the panoramic halfway image.
As a result, the display control unit 54 can superimpose the reduced panoramic intermediate image on the through image and cause the display unit 19 to display it.
In this case, since the through image is not largely hidden by the panoramic halfway image, the user can surely visually recognize the subject to be imaged next from the through image.

In the digital camera 1 of the present embodiment, the image composition unit 52 performs panorama as image data of a composite image for recording when a release operation is performed or a predetermined condition such as the total angular displacement reaches a threshold value is satisfied. Generate image data of an image.
In this case, the imaging control unit 51 executes control for recording the image data of the panoramic image generated for recording on the removable medium 31.
In this way, the user can easily and easily record image data of a desired panoramic image on the removable medium 31 without repeating the imaging operation many times.

  Note that the present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within a scope that can achieve the object of the present invention are included in the present invention.

  For example, in the above-described embodiment, the reduced panoramic halfway image is superimposed on the through image and displayed on the display unit 19. However, the display form of the through image and the panoramic intermediate image is displayed at the same time. However, the present invention is not particularly limited to this.

For example, the screen of the display unit 19 may be divided into two parts, and a through image may be displayed on one of the upper and lower screens, and an intermediate panoramic image may be displayed on the other screen.
In this way, by clearly distinguishing and displaying through images and panoramic images,
The user can view the panorama image representing the panorama image formation process more clearly. As a result, the user can more easily and accurately view the history of imaging from the start of panoramic imaging to the present, and can easily and easily obtain a desired panoramic image with fewer imaging operations. It becomes possible.

Further, for example, a panoramic intermediate image can be displayed without being reduced as necessary.
Further, in the present embodiment, the image composition unit 52 uses the image data of the previous panorama intermediate image and the image data of the frame acquired as the object of synthesis immediately before, in the blank area of the previous panorama intermediate image. However, it is also possible to display only a panoramic image without displaying a blank area.

  In the above-described embodiment, the image reduction unit 53 is provided in the subsequent stage of the image composition unit 52, but may be provided in the previous stage of the image composition unit 52. In this case, the image reduction unit 53 sequentially reduces the image data of the frames sequentially supplied from the image processing unit 17. Therefore, the image composition unit 52 generates image data of the panoramic intermediate image by compositing the image data of each reduced frame.

In the above-described embodiment, the angular velocity sensor 22 detects the angular displacement of the digital camera 1. However, the method of detecting the angular displacement amount is not particularly limited to this.
For example, a method of detecting the angular displacement of the digital camera 1 by image processing for analyzing a through image may be employed.

  In the above-described embodiment, the panorama halfway image and the panorama image have a horizontally long configuration. However, the present invention is not particularly limited thereto, and is configured to be long in the direction in accordance with the moving direction of the digital camera 1, for example, a vertically long image. It is good also as a structure.

For example, in the above-described embodiment, the imaging apparatus to which the present invention is applied has been described as an example configured as the digital camera 1.
However, the present invention is not particularly limited to this, and can be applied to general electronic devices having an imaging function that enables panoramic image imaging. For example, the present invention can be applied to a portable personal computer, a portable navigation device, It can be widely applied to portable game machines and the like.

  The series of processes described above can be executed by hardware or can be executed by software.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a recording medium into an imaging device or a computer that controls the imaging device. Here, the computer may be a computer incorporated in dedicated hardware. Alternatively, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  A recording medium including such a program is provided not only to the removable medium 31 distributed separately from the apparatus main body in order to provide the program to the user, but also to the user in a state of being incorporated in the apparatus main body in advance. Recording medium. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, and the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, a ROM 12 in which a program is recorded, a hard disk included in the storage unit 18, and the like.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.

  DESCRIPTION OF SYMBOLS 1 ... Digital camera, 11 ... CPU, 12 ... ROM, 13 ... RAM, 15 ... Optical system, 16 ... Imaging part, 17 ... Image processing part, 18 ... Memory | storage part, 19 ... Display part, 20 ... Operation part, 21 ... Communication unit, 22 ... Angular velocity sensor, 23 ... Drive, 31 ... Removable media, 41 ... Shutter switch, 51 ... Imaging control unit, 52 ... Image composition unit, 53 ... Image reduction unit, 54 ... Display control unit

Claims (7)

In an imaging apparatus including an imaging unit that sequentially outputs image data obtained sequentially by imaging at predetermined time intervals as frame image data.
Image composition for sequentially generating image data of a composite image by sequentially acquiring image data of the frames output from the imaging unit and sequentially combining at least a part of the sequentially acquired image data of the plurality of frames. Means,
Display control means for executing control for displaying the composite image sequentially generated as image data by the image composition means while displaying the frames sequentially output as image data from the imaging means;
An imaging apparatus comprising: The display control means executes control to display the composite image superimposed on the frame;
The imaging apparatus according to claim 1. A first reduction means for reducing the image data sequentially output from the imaging means;
The image synthesizing unit sequentially generates the reduced composite image data by using the image data reduced by the first reduction unit,
The display control means executes control to display the frame and the reduced composite image;
The imaging apparatus according to claim 1 or 2, wherein A second reduction means for reducing the image data sequentially generated by the image composition means;
The display control means executes control to display the frame and the reduced composite image;
The imaging apparatus according to claim 1 or 2, wherein The image synthesizing unit further generates image data of the synthesized image for recording when a predetermined condition is satisfied,
A recording control means for executing control for recording the image data of the composite image generated for recording by the image composition means;
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. In an imaging method executed by an imaging apparatus including an imaging unit that sequentially outputs image data obtained sequentially by imaging at predetermined time intervals as frame image data,
Image composition for sequentially generating image data of a composite image by sequentially acquiring image data of the frames output from the imaging unit and sequentially combining at least a part of the sequentially acquired image data of the plurality of frames. Steps,
A display control step for executing control for simultaneously displaying the frames sequentially output as image data from the imaging unit and the composite image sequentially generated as image data by the processing of the image composition step;
An imaging method comprising: A computer that controls an imaging apparatus including an imaging unit that sequentially outputs each of image data obtained by imaging at predetermined time intervals as frame image data.
Image composition for sequentially generating image data of a composite image by sequentially acquiring image data of the frames output from the imaging unit and sequentially combining at least a part of the sequentially acquired image data of the plurality of frames. Function and
A display control function for executing a control for simultaneously displaying the frames sequentially output as image data from the imaging unit and the composite image sequentially generated as image data by performing the image composition function;
A program characterized by realizing.

Images