JP5948779B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

Compared with a 2: 3 aspect ratio image captured by a 35 mm silver salt film or a 3: 4 aspect ratio image captured by a digital camera, There is a so-called panoramic image.
A technique for generating such panoramic image data is described in Patent Document 1. In the technique described in Patent Document 1, for example, the user moves the digital camera so as to rotate in the horizontal direction while being substantially fixed in the vertical direction while maintaining the state of pressing the shutter switch. Then, the digital camera uses a plurality of images captured during this time as processing targets, and executes a process of combining so as to join some areas of the plurality of images, thereby obtaining one panoramic image. Generate data for

JP-A-6-303562

  However, when a panoramic image in which the subject is captured at an appropriate position is demanded while placing importance on the surrounding landscape, if the subject moves during shooting, a panoramic image having a composition desired by the user cannot be obtained as a result. There was a fear.

  The present invention has been made in view of such a situation, and an object thereof is to easily and appropriately obtain a panoramic image having a composition desired by a user even when a subject within a shooting range has a movement. To do.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention is provided in common from an acquisition unit that acquires a plurality of images captured continuously and a plurality of images acquired by the acquisition unit. A first detection unit that detects a subject image to be detected , a change in the subject image detected from the plurality of images by the first detection unit, and a first subject image in which the change is detected ; A second detection unit that detects a second subject image in which a change different from the first subject image is detected; a plurality of images acquired by the acquisition unit; and a change by the second detection unit. Based on the detected first subject image and the second subject image , a first panorama image including the first subject image and a second panorama including the second subject image. generating means for generating an image, the generating Characterized by comprising display control means for displaying the first panorama image and the second panoramic image generated by the step on the display means, the

  According to the present invention, it is possible to easily and appropriately obtain a panoramic image having a composition desired by a user even when a subject within a shooting range has a movement.

1 is a block diagram illustrating a hardware configuration of an image processing apparatus according to an embodiment of the present invention. It is a functional block diagram which shows the functional structure for performing a panorama image generation process among the functional structures of the image processing apparatus of FIG. It is a schematic diagram which shows an example about the production | generation of the data of a panoramic image. 3 is a flowchart for explaining a flow of panoramic image generation processing executed by the image processing apparatus of FIG. 1 having the functional configuration of FIG. 2. 5 is a flowchart for explaining a flow of a panoramic image generation process with a subject executed in step S6 of FIG. It is a schematic diagram which shows the other example about the production | generation of the data of a panoramic image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an image processing apparatus 1 according to an embodiment of the present invention.
The image processing apparatus 1 is configured as a digital camera, for example.

  The image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an image processing unit 14, a bus 15, an input / output interface 16, and an imaging. A unit 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22 are provided.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The image processing unit 14 is configured by a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), and the like, and performs various image processing on image data in cooperation with the CPU 11.

  The CPU 11, ROM 12 and RAM 13 are connected to each other via a bus 15. An input / output interface 16 is also connected to the bus 15. An imaging unit 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22 are connected to the input / output interface 16.

  Although not shown, the imaging unit 17 includes an optical lens unit and an image sensor.

The optical lens unit is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and output as an output signal of the imaging unit 17.
Such an output signal of the imaging unit 17 is hereinafter referred to as “captured image data”. The captured image data is appropriately supplied to the CPU 11, the image processing unit 14, and the like.

The input unit 18 includes various buttons such as a shutter switch, and inputs various information and commands in accordance with user instruction operations.
The output unit 19 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 20 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 31 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20.

FIG. 2 is a functional block diagram showing a functional configuration for executing a panoramic image generation process among the functional configurations of such an image processing apparatus 1.
“Panorama image generation processing” refers to processing for generating data of a plurality of panoramic images using data of a plurality of captured images acquired successively.
Here, the “panoramic image” is a landscape image or a 3: 3 aspect ratio image captured by a 35 mm silver salt film or a 3: 4 aspect ratio image captured by a digital camera. This is a vertically wide-angle image. In the present embodiment, a panoramic image simply refers to a panoramic image of a still image.
That is, a moving image composed of a plurality of panoramic images that are still images is particularly referred to as a “panoramic moving image”.
The panorama image generation process stores panorama image data designated by the user from among the generated plurality of panorama image data, or generates panorama moving image data based on the generated multiple panorama image data. And the process of memorizing it.

When the image processing apparatus 1 executes the panoramic image generation process, the imaging control unit 51 and the storage control unit 52 function in the CPU 11.
In this case, the storage unit 20 is provided with an image storage unit 53 as an area for storing various image data such as captured image data and panoramic image data.

The imaging control unit 51 controls various imaging operations by the imaging unit 17.
Specifically, when the user presses the shutter switch of the input unit 18 while holding the image processing apparatus 1 as a digital camera, the image processing apparatus 1 starts panoramic image generation processing. When the panoramic image generation process starts, the imaging control unit 51 starts the continuous imaging operation of the imaging unit 17. During this time, the user moves the digital camera so as to rotate in the horizontal direction (hereinafter referred to as a panoramic shooting direction) while maintaining the state of pressing the shutter switch while being substantially fixed in the vertical direction. Thereafter, when a predetermined condition such as the elapse of a predetermined time or the movement of the image processing apparatus 1 by a predetermined amount is satisfied, the imaging control unit 51 ends the continuous imaging operation in the imaging unit 17.

  The storage control unit 52 executes control for storing various image data such as captured image data, panoramic image data, and panoramic moving image data in the image storage unit 53.

  When the image processing apparatus 1 executes panoramic image generation processing, the image processing unit 14 includes an image acquisition unit 54, a slit setting unit 55, a subject detection unit 56, a moving object detection unit 57, and a panorama. The image generation unit 58 and the panoramic video generation unit 59 function.

  The image acquisition unit 54 sequentially acquires data of a plurality of captured images output by the continuous imaging operation of the imaging unit 17, that is, data of a plurality of images captured continuously.

The slit setting unit 55 sets each of the plurality of captured image data acquired by the image acquisition unit 54 as a processing target, and sets a plurality of regions divided from each of the plurality of captured images as a slit region.
As described above, the “slit region” refers to a region divided from the captured image, but is used as a unit for detecting a subject image to be described later or a subject image that is a moving object. Therefore, in the present embodiment, a strip-shaped area that is divided in the captured image in the vertical direction or the horizontal direction by a predetermined length (a predetermined number of pixels) is employed as the slit area.

The subject detection unit 56 detects a subject image for each slit region set by the slit setting unit 55 using each of a plurality of captured image data acquired by the image acquisition unit 54 as a processing target.
Specifically, the subject detection unit 56 detects a subject image in consideration of an edge in the slit region.

The moving object detection unit 57 detects a change in the subject image detected by the subject detection unit 56, and detects the changed subject image as a moving object.
For example, the moving object detection unit 57 sets a slit area where a subject image is detected as a processing target, and sets a slit area of another captured image in the same spatial position as the processing target slit area as a comparison target, and sets the processing target and the comparison processing target as When a change in each pixel value is detected and a change of a predetermined amount or more is detected, a subject image included in the processing target is detected as a moving object.

  The panorama image generation unit 58 generates a plurality of panoramic image data based on the data of the plurality of images acquired by the image acquisition unit 54 and the subject image detected by the moving object detection unit 57 as being changed. . A specific example of panoramic image data generated by the panoramic image generation unit 58 will be described later with reference to FIGS. 3 and 6.

  The panoramic video generation unit 59 generates panoramic video data based on a plurality of panoramic image data generated by the panoramic image generation unit 58. That is, the panoramic video generation unit 59 can generate data of the panoramic moving image by using a moving image composed of a plurality of panoramic images as a panoramic moving image.

Further, hereinafter, the imaging control unit 51 and the storage control unit 52 that function in the CPU 11, and the functions of the image acquisition unit 54 to the panoramic video generation unit 59 that function in the image processing unit 14, that is, the execution function of the panoramic image generation process, This will be specifically described with reference to FIG.
FIG. 3 is a schematic diagram illustrating an example of generation of panoramic image data. Specifically, FIG. 3A is a schematic diagram illustrating an acquired captured image, and FIG. 3B is a schematic diagram illustrating a panoramic image generated for each movement of the subject image based on the acquired captured image. It is.

  As shown in FIG. 3A, the image acquisition unit 54 acquires each data of a plurality of captured images (first captured image P1 to sixth captured image P6) captured in the panoramic shooting direction by the imaging unit 17. To do.

  For example, the slit setting unit 55 sets a plurality of slit regions for each processing target, with each piece of data of the plurality of captured images (first captured image P1 to sixth captured image P6) as a processing target.

  The subject detection unit 56 detects the subject image reflected in the slit region for each slit region set by the slit setting unit 55. As a result, in the example shown in FIG. 3A, temple buildings, trees, people, and the like are detected as subject images.

  In the example illustrated in FIG. 3A, the moving body detection unit 57 includes the slit region SL1 of the first captured image P1, the slit region SL2 of the second captured image P2, and the slit region SL3 of the third captured image P3. A person is detected as a moving object.

  As shown in FIG. 3B, the panorama image generation unit 58 generates a panorama image for each slit region including a person (subject) that is a moving object. That is, the panorama image generation unit 58 generates panorama image PRP1 data including a person in the slit area SL1, panorama image PRP2 data including a person in the slit area SL2, and a panorama image PRP3 including a person in the slit area SL3. Generate each of the data.

  The panoramic video generation unit 59 generates panoramic video data using the panoramic images PRP1, PRP2, and PRP3 shown in FIG. In other words, the panoramic video generation unit 59 generates panoramic video data composed of a plurality of panoramic images.

FIG. 4 is a flowchart for explaining the flow of panoramic image generation processing executed by the image processing apparatus 1 of FIG. 1 having the functional configuration of FIG.
The panorama image generation process is started when the user starts an operation for starting the panorama image generation process on the input unit 18, that is, when the shutter button is pressed.

  In step S1, the imaging control unit 51 controls the imaging unit 17 to perform continuous imaging.

  In step S2, the image acquisition unit 54 acquires data of a plurality of captured images. Specifically, for example, in the example of FIG. 3A, the image acquisition unit 54 acquires data of the first captured image P1 to the sixth captured image P6.

  In step S <b> 3, the slit setting unit 55 sets a slit area for each of a plurality of captured image data acquired in step S <b> 2.

  In step S4, the subject detection unit 56 detects a subject image for each slit region set in step S3.

  In step S5, the moving object detection unit 57 detects a subject image that is a moving object from the subject images detected in the process of step S4. For example, in the example of FIG. 3A, a person is detected as a moving object from each of the slit area SL1 of the first captured image P1, the slit area SL2 of the second captured image P2, and the slit area SL3 of the third captured image P3. The

In step S6, the panorama image generation unit 58 executes a panorama image generation process with a subject.
“Subject panorama image generation processing” refers to a series of processing until data of a panoramic image including a subject image that is a moving object is generated. The subject panorama image generation process will be described later with reference to FIG.
Thereby, each data of the several panoramic images PRP1-PRP3 as shown, for example in FIG.3 (b) is produced | generated.

In step S7, the storage control unit 52 determines whether or not to save a still image.
If the still image is to be stored, YES is determined in step S7, and the process proceeds to step S8.

In step S8, the storage control unit 52 determines whether to select a panoramic image to be saved. For example, the image processing apparatus 1 causes the user to perform a selection operation using the input unit 18 by sequentially displaying a plurality of generated panoramic images on the output unit 19.
Until such a selection operation is performed, it is determined as NO in Step S8, and the process returns to Step S8. That is, until such a selection operation is performed, the determination process in step S8 is repeatedly executed, and the panorama image generation process enters a standby state.
Thereafter, when such a selection operation is performed, the storage control unit 52 selects the panoramic image to be saved by analyzing the contents of such a selection operation, and advances the processing to step S9.

  In step S <b> 9, the storage control unit 52 stores the selected panoramic image data in the image storage unit 53. Thereby, the panoramic image generation process is completed.

In the foregoing, the process for storing a still image has been described.
Next, processing when a still image is not saved will be described. In this case, it is determined as NO in Step S7, and the process proceeds to Step S10.
In step S10, the panoramic video generator 59 generates panoramic video data.

  In step S <b> 11, the storage control unit 52 stores panoramic video data in the image storage unit 53. Thereby, the panoramic image generation process is completed.

  FIG. 5 is a flowchart for explaining the flow of panoramic image processing with a subject executed in step S6 of FIG.

  In step S21, the panoramic image generation unit 58 extracts each data of the slit area where the subject image that is a moving object exists. Specifically, as illustrated in FIG. 3A, the panoramic image generation unit 58 includes the slit region SL1 of the first captured image P1 in which the subject image is detected as a moving object in the data of each captured image, the first Each data of the slit region SL2 of the second captured image P2 and the slit region SL3 of the third captured image P3 is extracted.

  In step S <b> 22, the panoramic image generation unit 58 generates a plurality of panoramic image data obtained by combining the data of each captured image in units of slit areas so as to include the extracted slit area data. Specifically, the panoramic image generation unit 58 extracts the extracted slit areas, that is, the slit areas where the subject image is a moving object (the slit areas SL1 of the first captured image P1, the slit areas SL2 and 3 of the second captured image P2). The data of the slit area SL3) of the captured image P3 and the other slide areas are combined to generate panoramic images PRP1, PRP2, and PRP3 corresponding to the movement of the moving object as shown in FIG. 3B. To do. Thereafter, the panoramic image processing with the subject ends, and the process returns to step S6.

As described above, the image processing apparatus 1 generates a plurality of panoramic image data of various compositions even when the subject image within the shooting range has a motion, so that the panorama of the composition desired by the user is generated. Image data can be selected easily and appropriately.
Further, the image processing apparatus 1 can generate panoramic moving image data that incorporates elements of a moving image even though it is a panoramic image, by utilizing the movement of a subject image that exists within the imaging range. .

  Next, another example of generating a panoramic image will be described with reference to FIG. FIG. 6 is a schematic diagram illustrating another example of generating panoramic image data. Specifically, FIG. 6A is a schematic diagram showing the acquired captured image, and FIG. 6B shows a panoramic image generated by removing the subject image that becomes a moving object based on the acquired captured image. It is a schematic diagram shown. FIG. 6A illustrates an example in which the moving amount of the subject image that is a moving object is large in the seventh captured image P7 to the tenth captured image P10, and the subject image that is a moving object does not exist within the same imaging range. In FIG. 6A, in the eleventh captured image P11 and the twelfth captured image P12, the moving amount of the subject image serving as a moving object is small, and the subject image serving as a moving object exists within the same imaging range of all the captured images. An example is shown.

In the above-described embodiment, as shown in the first captured image P1 to the third captured image P3 in FIG. 3A, the moving amount of the subject image that is a moving object is small, and the moving object is present in all captured images having the same imaging range. The panoramic image data that can be generated when the subject image is reflected has been described.
On the other hand, in this example, as shown in the seventh captured image P7 to the tenth captured image P10 in FIG. 6A, the captured image has a large moving amount of the subject image that is a moving object and has the same imaging range. The panoramic image data that can be generated when there is a captured image in which a subject image that is a moving object is not reflected will be described.

  In the case of this example, as in the above-described embodiment, not only can panoramic image data be generated in accordance with the movement of the subject image that is a moving object, but also the moving object is in the same imaging range in each captured image. By utilizing the fact that the subject image is not reflected, it is possible to generate panoramic image data from which the subject image that is a moving object is removed.

  Specifically, the panorama image generation unit 58 uses a portion of the seventh captured image P7 to the tenth captured image P10 where a moving subject image does not appear in the same imaging range of each captured image. Data of a panoramic image PRP4 as shown in 6 (b) is generated. In FIG. 6B, the portion indicated by the vertical stripe pattern is a portion corresponding to the eleventh captured image P11 and the twelfth captured image P12, and the moving amount of the subject image serving as a moving object is small, and the same captured image is obtained in each captured image. Since there is a subject image that becomes a moving object within the range, there is no portion that can be used, and the subject image that becomes a moving object cannot be removed. Therefore, the panoramic image generation unit 58 generates panoramic image data other than the portion corresponding to the eleventh captured image P11 and the twelfth captured image P12 from which the subject image that is a moving object cannot be removed.

In the case of this example, panoramic video data can be generated by a generation method different from that of the above-described embodiment.
Specifically, when the captured images (the first captured image P1 to the sixth captured image P6) as in the example of the above-described embodiment are used, the panoramic video generation unit 59 uses the first captured image P1 to the first captured image P1. The panoramic video data is generated using the panoramic images PRP1, PRP2, and PRP3 generated from the six captured images P6.
On the other hand, when the captured images (seventh captured image P7 to twelfth captured image P12) as in this example are used, the panoramic video generation unit 59 is a moving object between captured images having the same imaging range. Data of one common background image used for generating a panoramic moving image is generated using an image of a portion where a subject image is not reflected. Then, the panoramic video generation unit 59 generates a plurality of panoramic image data by combining the background image and the subject image to be a moving object for each movement of the subject image to be a moving object. The panoramic video generation unit 59 generates panoramic moving image data using the generated panoramic image data for each motion of the subject image that is the moving object. In other words, the panoramic video generation unit 59 performs panorama processing based on a part other than the subject image with the change detected by the common moving object detection unit 57 and a plurality of subject images with the change detected by the moving object detection unit 57. Generate moving image data.

The image processing apparatus 1 configured as described above includes an image acquisition unit 54, a subject detection unit 56, a moving object detection unit 57, and a panoramic image generation unit 58.
The image acquisition unit 54 acquires data of a plurality of images captured continuously.
The subject detection unit 56 detects subject images from the plurality of images, with each of the data of the plurality of images acquired by the image acquisition unit 54 as a processing target.
The moving object detection unit 57 detects a change in the subject image detected from the plurality of images of the subject detection unit 56.
The panorama image generation unit 58 generates a plurality of panoramic image data based on the data of the plurality of images acquired by the image acquisition unit 54 and the subject image detected by the moving object detection unit 57 as being changed. .
Therefore, in the image processing apparatus 1, a plurality of panoramic image data is obtained based on the data of the plurality of images acquired by the image acquisition unit 54 and the subject image detected by the moving object detection unit 57. Generate. For this reason, the image processing apparatus 1 can easily and appropriately obtain a panoramic image having a composition desired by the user even when the subject image within the shooting range has a motion.

In addition, the image processing apparatus 1 uses the image acquisition unit 54 as a processing target for each of a plurality of image data, and sets a plurality of regions divided from each of the plurality of images as slit regions, respectively. Is further provided.
The subject detection unit 56 detects a subject image for each slit region set by the slit setting unit 55.
Therefore, in the image processing apparatus 1, since the subject image is detected in units of slit areas, it is only necessary to replace the slit areas as they are when generating panoramic images, thereby reducing the processing burden relating to the generation of panoramic images. can do.

The storage control unit 52 further selects one panoramic image from among the plurality of panoramic images generated by the panoramic image generation unit 58.
Therefore, the image processing apparatus 1 can generate a panoramic image having a desired composition among the plurality of generated panoramic images.

The image processing apparatus 1 further includes a panoramic video generation unit 59 that forms a plurality of generated panoramic images as one moving image and generates panoramic moving image data.
Therefore, the image processing apparatus 1 can generate a panoramic moving image from a plurality of generated panoramic images.

The image processing apparatus 1 is configured to generate a panoramic moving image based on a portion other than the subject image having the change detected by the common moving object detection unit 57 and a plurality of subject images having the change detected by the moving object detection unit 57. A panoramic video generation unit 59 that performs data is further provided.
Therefore, the image processing apparatus 1 can generate a panoramic moving image from a plurality of generated panoramic images.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, the image acquisition unit 54 is described as acquiring data of the captured image supplied by the imaging unit 17 in the example of FIG. 2, but is not particularly limited to the example of FIG. The supply source is not limited as long as the image data can be generated. For example, the image acquisition unit 54 can also acquire image data supplied from the removable media 31 or the Internet via the communication unit 21 as image data.

  In the above-described embodiment, the slit setting unit 55 sets the slit area and determines the detection of the subject image for each slit area in consideration of reducing the processing burden in generating the panoramic image. Instead, processing may be performed by detecting a region for each detected subject image. In this case, since the processing is performed in units of pixels, the processing load increases, but the detection range can be narrowed, so that a panoramic image with less sense of incongruity can be generated.

In the above-described embodiment, the image processing apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having a panoramic image generation processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the image processing apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG. .
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 20 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall device configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalent scope thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
An acquisition means for acquiring a plurality of images captured continuously;
First detection means for detecting a common subject image from a plurality of images acquired by the acquisition means;
Second detection means for detecting changes in subject images detected from the plurality of images by the first detection means;
Generation that generates a plurality of panoramic images each including a subject image in which the change is detected based on the plurality of images acquired by the acquisition unit and the subject image in which the change is detected by the second detection unit. Means,
An image processing apparatus comprising:
[Appendix 2]
A slit area setting means for setting a plurality of areas divided from each of the plurality of images acquired by the acquisition means as slit areas, respectively,
The image processing apparatus according to appendix 1, wherein the first detection unit detects the subject image for each slit region set by the slit region setting unit.
[Appendix 3]
Selecting means for selecting one panoramic image from among the panoramic images generated by the generating means;
Recording means for controlling the panorama image selected by the selection means to be recorded on a recording medium;
The image processing apparatus according to appendix 1 or 2, further comprising:
[Appendix 4]
The image processing apparatus according to appendix 1 or 2, further comprising moving image generation means for generating a panoramic moving image composed of the plurality of panoramic images.
[Appendix 5]
The generation unit includes a plurality of images acquired by the acquisition unit, other than the subject image, the change of which is detected by the second detection unit, and a plurality of changes of which are detected by the second detection unit. The image processing apparatus according to any one of appendices 1 to 4, wherein a plurality of panoramic images each including the subject image in which the change is detected is generated based on the subject image.
[Appendix 6]
In the image processing method executed by the image processing apparatus,
An acquisition step of acquiring a plurality of images captured continuously;
A first detection step of detecting a common subject image from a plurality of images acquired by the processing of the acquisition step;
A second detection step of detecting a change in the subject image detected from the plurality of images by the processing of the first detection step;
A plurality of panoramic images each including the subject image in which the change is detected based on the plurality of images acquired by the processing in the acquisition step and the subject image in which the change is detected in the processing in the second detection step A generation step for generating
An image processing method comprising:
[Appendix 7]
Computer
Acquisition means for acquiring a plurality of images captured continuously;
First detecting means for detecting a common subject image from a plurality of images acquired by performing the function of the acquiring means;
Second detection means for detecting changes in subject images detected from the plurality of images by demonstrating the function of the first detection means;
Based on the plurality of images acquired by the function of the acquisition unit and the subject image detected by the function of the second detection unit as a change, the subject image from which the change is detected is obtained. Generating means for generating a plurality of panoramic images each including
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Image processing part, 15 ... Bus, 16 ... Input / output interface, 17. ..Imaging section, 18 ... input section, 19 ... output section, 20 ... storage section, 21 ... communication section, 22 ... drive, 31 ... removable media, 51 ... Imaging control unit, 52 ... storage control unit, 53 ... image storage unit, 54 ... image acquisition unit, 55 ... slit setting unit, 56 ... subject detection unit, 57 ... moving object detection , 58... Panoramic image generation unit, 59... Panoramic video generation unit

Claims (9)

An acquisition means for acquiring a plurality of images captured continuously;
First detection means for detecting a common subject image from a plurality of images acquired by the acquisition means;
A change in the subject image detected from the plurality of images is detected by the first detection means, and the first subject image in which the change is detected and a change in which the first subject image is different are detected. Second detection means for detecting a second subject image ;
The first subject image is included based on the plurality of images obtained by the obtaining unit and the first subject image and the second subject image whose change is detected by the second detecting unit. Generating means for generating a first panoramic image and a second panoramic image including the second subject image ;
Display control means for causing the display means to display the first panorama image and the second panorama image generated by the generation means;
An image processing apparatus comprising: A slit area setting means for setting a plurality of areas divided from each of the plurality of images acquired by the acquisition means as slit areas, respectively,
The image processing apparatus according to claim 1, wherein the first detection unit detects the subject image for each slit region set by the slit region setting unit. A selection unit that selects a specific panoramic image from a plurality of different panoramic images generated by the generation unit;
Recording control means for controlling the recording means to record the specific panoramic image selected by the selection means;
The image processing apparatus according to claim 1, further comprising:   The image processing apparatus according to claim 1, further comprising a moving image generation unit configured to generate a panoramic moving image composed of the plurality of types of panoramic images.   The generation unit includes a plurality of images acquired by the acquisition unit, other than the subject image, the change of which is detected by the second detection unit, and a plurality of changes of which are detected by the second detection unit. 5. The image processing apparatus according to claim 1, wherein a panorama image corresponding to each of the subject images in which the change is detected is generated based on the subject image of the subject.   The said selection means makes a user select at least 1 specific panoramic image from the said several different panoramic images displayed on the said display means, The any one of Claim 3 thru | or 5 characterized by the above-mentioned. Image processing device. The selection means further comprises a detection means for detecting a user input operation to the input means,
The image processing according to claim 3, wherein the selection unit selects a specific panoramic image from the plurality of different panoramic images based on a detection result by the detection unit. apparatus. In the image processing method executed by the image processing apparatus,
An acquisition step of acquiring a plurality of images captured continuously;
A first detection step of detecting a common subject image from a plurality of images acquired by the processing of the acquisition step;
A change in the subject image detected from the plurality of images is detected by the processing of the first detection step, and a change different from the first subject image in which the change is detected and the first subject image is detected. A second detection step for detecting a second subject image ;
Based on the plurality of images acquired by the process of the acquisition step and the subject image for which the change is detected by the process of the second detection step, the first subject image in which the change is detected, and the first image on the basis of the second subject image, a generation step of generating a first panoramic image included the first object image and a second panorama image included the second subject image,
A display control step for causing the display means to display the first panorama image and the second panorama image generated by the generation step;
An image processing method comprising: Computer
Acquisition means for acquiring a plurality of images captured continuously;
First detecting means for detecting a common subject image from a plurality of images acquired by performing the function of the acquiring means;
A change in the subject image detected from the plurality of images is detected by the function of the first detection means, and the first subject image from which the change is detected differs from the first subject image. Second detection means for detecting the detected second subject image ;
The first in which the change is detected based on the plurality of images acquired by the function of the acquisition unit and the subject image detected as having changed by the function of the second detection unit . Generating a first panoramic image including the first subject image and a second panoramic image including the second subject image based on the subject image and the second subject image means,
Display control means for causing the display means to display the first panorama image and the second panorama image generated by the generation means;
A program characterized by functioning as

Images