JP2002084412A - Image processing unit and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing unit and an image processing method by which positioning of images picked up by an image pickup means can be simplified so as to remarkably reduce the processing time. SOLUTION: The image processing unit is provided with a means 58 that detects a point light source in a still state included in picked-up images obtained from the same object and with a means 53 that positions the images on the basis of the detected point light source. Since deviation among the images caused by camera-shake or the like can be grasped from the deviation of the point light source and the point light source is used for a reference, the arithmetic operation for positioning the images can be simplified and the image processing time can be decreased.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus and an image processing method for aligning a plurality of images obtained by an image pickup apparatus such as a digital camera having a multiple image photographing function.

[0002]

2. Description of the Related Art In order to perform multiple image processing for obtaining one composite image from a plurality of captured images, there is known a digital camera capable of capturing a plurality of images in a multiple image capturing mode. The multiple image processing includes image super-resolution processing, omnifocal image creation processing, and blur control processing.

Incidentally, in order to perform the above-described multiplex image processing, it is necessary that the positions of a plurality of acquired images match. If the angle of view and other conditions at the time of shooting change, image data at the same position on a plurality of images does not always indicate the same part of the same subject.
For example, even if the same subject is shot in a short time using the continuous shooting function and multiple images are obtained, the position of the corresponding pixel between the images will change greatly due to camera shake or movement of the subject. I will.

For this reason, before performing the synthesizing process, a process of aligning each image (so-called registration process) is performed in advance. As a method of the alignment processing, for example, a “corresponding point extraction method based on template matching” disclosed in Japanese Patent Application Laid-Open No. 8-161503 is known.

[0005]

However, in the conventional alignment processing, the amount of image data to be processed is relatively large,
The calculation for alignment is also complicated, and there is a problem that a long time is spent for image processing.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problem, and an image processing apparatus and an image processing method capable of simplifying alignment of a plurality of images picked up by an image pickup means and greatly reducing a processing time. Another object of the present invention is to provide an imaging device.

[0007]

The object of the present invention is to provide a means for detecting a point light source in a stationary state included in a plurality of captured images of the same subject, and a method for aligning a plurality of images based on the detected point light source. And means for performing the following.

According to this image processing apparatus, a stationary point light source included in a plurality of captured images is detected, and the plurality of images are aligned based on the point light source.

Further, since the point light source is in a stationary state, it is easy to understand a slight shift between images caused by camera movement due to camera shake due to a shift of the point light source between a plurality of images. As a result, the calculation for aligning a plurality of images is simplified, and the processing time is shortened.

In the image processing apparatus, the stationary point light source is a concept that includes not only a completely stationary point light source but also a point light source that can be regarded as stationary compared to the shooting time.

[0011] The object is to provide a means for detecting a focused area of a plurality of captured images of the same subject, and a means for aligning a plurality of images using image data of only the detected focused area. The present invention is also solved by an image processing apparatus characterized by comprising:

In this image processing apparatus, for example, in portrait photography, if a specific area near the main subject is detected as a focused area, an image in which the position of the main subject is properly synthesized can be synthesized. Since it is small when viewed from the whole image, the calculation area is also small, and the processing time is reduced.

[0013] Further, the above object is to provide a means for setting one of a plurality of captured images of the same subject as a reference image, a means for performing positioning of a plurality of remaining images based on the reference image, The present invention is also solved by an image processing apparatus characterized by comprising:

In this image processing apparatus, for example, if one image having a large depth of field is used as a reference, even if the image for creating a multiplex image is out of focus, the image may be in any of the distances from the reference image. In contrast, the positions of a plurality of images can be easily adjusted, and furthermore, since only the comparison with the reference image is performed, the calculation for the alignment is simplified and the processing time is shortened.

The object is to provide an image processing method for aligning a plurality of captured images of the same subject, wherein the plurality of images are aligned with respect to a stationary point light source included in the plurality of captured images. The image processing method is also characterized by performing the following.

According to this processing method, the positioning accuracy is improved, the calculation for the positioning is simplified, and the processing time is shortened. Further, the photographer can arbitrarily designate a point light source that can be regarded as a stationary state in a plurality of photographed images.

[0017] Further, the object is to provide an image processing method for aligning a plurality of captured images of the same subject, wherein the plurality of images are aligned using only the in-focus area of the plurality of captured images. The problem is also solved by an image processing method characterized by performing.

In this processing method, if a specific area is detected as a focused area, an image positioned at the specific area can be synthesized, and the processing area can be reduced, so that the processing time is shortened.

The above object is also achieved by an image processing method for aligning a plurality of captured images of the same subject, based on one reference image of three or more captured images. The image processing method is characterized in that the image is aligned.

In this processing method, a plurality of images need only be compared with the reference image, so that the calculation for alignment is simplified and the processing time is shortened.

[0021]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an external perspective view of a digital camera as an image pickup apparatus to which the image processing apparatus according to the first embodiment of the present invention is applied, and FIG. 2 is a rear view of the digital camera. is there.

1 and 2, the digital camera 1 is provided with a photographic lens 11 on the front surface of a camera body 1A and a finder window 14 and a distance measuring window 19 on the upper front surface. I have. Inside the camera body 1A, a CCD 12 as an example of an image pickup device that receives an optical image from the photographing lens 11 and performs photoelectric conversion is provided.

Further, a release (shutter) button 13 and a photographing mode setting key 1 are provided on the upper surface of the camera body 1A.
7 and a liquid crystal panel 18 are provided, and an insertion port 16 for holding the recording medium 15 in a detachable manner is provided on a side surface of the camera body 1A. In the digital camera 1, image data obtained by the CCD 12 can be recorded on a recording medium 15.

With the shooting mode setting key, the liquid crystal panel 18
The user can set exposure conditions such as aperture priority and shutter speed priority, switch macro shooting, and further set zoom while looking at the display contents of.

On the back of the camera body 1A, a liquid crystal monitor 111 for live view display, an image processing mode setting key 110 and the like are provided. The digital camera 1 has a multiple image shooting mode in addition to the normal shooting mode. With the image processing mode setting key 110, a normal shooting mode and a multiple image shooting mode can be selected and set.

In the multiple image photographing mode, a plurality of images of the same subject are captured in one photographing operation, and a composite process is performed using the obtained plurality of image data. , All-focus images, high-resolution images,
It has a function of creating images such as a blur control image and a noise reduction image.

FIG. 3 is a block diagram showing the electrical configuration of the digital camera 1, wherein thin arrows indicate the flow of control data and thick arrows indicate the flow of image data.

In FIG. 3, the CCD 12 is set to be slightly displaceable in the optical axis direction in the multiple image shooting mode, particularly in the super-resolution shooting mode for generating a high-resolution image. It is driven to be displaced by the CCD position control unit 54 during photographing. CCD position controller 5
Numeral 4 is composed of, for example, a piezoelectric element, and drives the CCD 12 by deformation thereof.

The A / D converter 51 converts an analog image signal from the CCD 12 into digital image data, and the RAM 52 temporarily stores the image data.

The stationary point light source detection unit 58 is included in a plurality of captured images in image processing for increasing the resolution.
A stationary point light source for image registration reference (e.g.,
A star in a night view image).

The positioning unit 53 includes a stationary point light source detection unit 5
Based on the point light source detected in step 8, the amount of displacement between the second and third image data is calculated. The high-resolution image generation unit 57 is OFF (no processing) in the normal shooting mode
In the super-resolution shooting mode, the image is turned on (with processing), and based on the calculation result obtained by the positioning unit 53, the resolution of the second and third images is increased. And synthesizes it into one image.

The recording medium 15 stores the image data read from the RAM 52 in the normal photographing mode, and stores the image data generated by the high-resolution image generating section 57 in the super-resolution photographing mode.

The CPU 50 controls the whole of the digital camera 1 by controlling each unit. For example, shooting conditions when the release button 13 is pressed, the setting state of the image processing mode setting key 110, and the like are stored, and the shooting lens driving unit 56 is controlled based on the distance measurement result in the distance measurement window 19. And controls the display on the liquid crystal panel 17 and the liquid crystal monitor 111.

In this embodiment, the shift photographing method is used as the super-resolution photographing method. However, this is an example, and another method may be used.

Next, a photographing sequence by the digital camera 1 having the above configuration will be described with reference to FIGS.

Prior to photographing, the photographer uses the photographing mode setting key 17 and the liquid crystal panel 18 to set exposure conditions such as aperture priority and shutter speed priority and to switch the white balance mode.

An image processing mode setting key 110 and a liquid crystal monitor 111 are used to select a normal shooting mode or a super-resolution shooting mode. FIG. 4 shows the settings for the image processing. When the image processing mode setting key 110 is turned off, normal shooting is performed. When turned on, super-resolution shooting is performed. In other words, multiple images with different shooting positions are taken,
Combine to generate a high-resolution image.

In S201, the CPU 50 determines whether or not the release button 13 has been pressed. Release button 1
When 3 is pressed (YES in S201), S202
Then, the CPU 50 reads the photographing conditions and image processing mode settings at that time, and in step S203, fetches distance measurement data. If the release button 13 is not pressed (S2
(The determination of 01 is NO), and it waits until it is pushed.

Next, in S204, the CPU 50 determines whether or not the super-resolution photographing mode is set as the image processing mode, and if the normal photographing mode is set (S40).
In step S205, the CPU 50 turns off the stationary point light source detection unit 58, the positioning unit 53, and the high-resolution image generation unit 57.

Subsequently, in step S206, the photographing lens driving unit 5
After the photographing lens 11 is driven to focus on an appropriate object through the CCD 6, the CCD 12 is integrated in S207, and
At 208, the image signal is read as data. The read image data is converted into digital data by the A / D converter 51, and is temporarily stored in the RAM 52 in S209. Then, in S210, the CPU 50 reads out the image data from the RAM 52 and stores it in the medium 15.

On the other hand, when the super-resolution shooting mode is set as the image processing mode (the determination in S204 is YE
S), and proceeds to S221 of FIG. In S221, the CPU 5
0 sets the optimum photographing conditions for detecting the stationary point light source, and drives the photographing lens 11 via the photographing lens driving unit 56 so as to focus on an appropriate subject in S222.
Thereafter, in step S223, the CCD 12 is integrated, and the first image for detecting a stationary point light source is taken. In S224, the image data is converted into digital data by the A / D converter 51,
In S225, the data is temporarily stored in the RAM 52.

Subsequently, in step S226, the CPU 50 reads the photographing conditions at that time, and in step S227, drives the photographing lens 11 via the photographing lens driving unit 56 so as to focus on an appropriate subject. Thereafter, in S228, the CCD 12
Is integrated, and a second image for detecting a stationary point light source is taken. S
In 229, the image signal is read out and converted into digital data by the A / D converter 51.
It is temporarily stored in M52.

Further, in S231, the CPU 50 sets the CC
The position of the CCD 12 is displaced via the D position control unit 54, and it is determined whether the amount of displacement has reached a predetermined amount. If it is displaced by a predetermined amount (YE in S231)
S), and proceeds to S232 of FIG. If it has not been displaced by the predetermined amount (NO in S231), it is driven until it is displaced.

In S232 and subsequent steps, the third image is similarly photographed. That is, in step S232, the CPU 50 drives the photographing lens 11 via the photographing lens driving unit 56 so as to focus on an appropriate subject. Then, in step S233, the CCD 12 is integrated, and in step S234, the image signal is converted to the A / D converter 51. Is converted to digital data, and the image data is converted to R in S235.
It is stored in the AM 52.

Thereafter, in S 236, the CPU 50 reads these three pieces of image data from the RAM 52 and sends the first piece of image data to the stationary point light source detection unit 58. S23
In 7, the stationary point light source detection unit 58 determines whether or not a reference stationary point light source has been detected in the image data.
When a stationary point image is detected (the determination in S237 is YE
S) The second and third image data are stored in the positioning unit 5
Sent to 3.

In S238, the positioning unit 53
Based on a stationary point light source serving as a reference in the first image, the amount of displacement between the image data of the second and third images is calculated. In S239, the high-resolution image generation unit 57
Based on the calculation result obtained by the positioning unit 53,
A synthesis process for high resolution is performed to generate one image. In step S211, the CPU 50 stores the
5 is stored.

If an appropriate reference point is not detected (NO in S237), a warning is output in S240, and in S241, the synthesizing process is stopped and the process ends.

After a warning is output, the same positioning as in the related art may be performed.

As described above, a stationary point light source in a plurality of images photographed in the multiple image photographing mode is detected, and the plurality of images are aligned based on the point light source.
Even if there is a shift between images due to camera movement due to camera shake, etc., it can be accurately grasped from the shift of the point light source between multiple images, and since the reference is a point light source,
Calculations for image alignment are simplified, and processing time for high resolution can be reduced.

Next, the above-described positioning based on a stationary point light source will be described with reference to a specific image shown in FIG.

In FIG. 8, it is assumed that two night views have been photographed in order to obtain a high-resolution image. FIG. 8A shows the first image, and FIG. 8B shows the second image. The image in FIG. 8B is shifted to the upper left from the image in FIG. When performing the superimposition process to combine these two images, a point light source that is stationary within the angle of view, for example, the star P1 is used as a reference. Specifically, FIG.
The star P1 shown in FIG. 8A and the star P2 shown in FIG. 8B are positioned so as to overlap each other as shown in FIG. 8C.

As described above, when a plurality of photographed images are superimposed, if there is a stationary point light source in the image, the positioning between the plurality of images can be easily performed by using the point light source as a reference. . In particular, when multiple images of a distant view are taken, high overlay accuracy can be obtained.

In the examples shown in the flowcharts of FIGS. 5 to 7, one image is taken for detecting a stationary point light source, but the present invention is not limited to this. For example, a stationary point light source may be detected using image data (second and third shot image data in the above example) obtained by shooting for multiple image shooting, May be specified.

Further, the present invention is not limited to a point light source in a completely stationary state within the angle of view of a plurality of captured images. Good.

FIG. 9 is a front view showing a processing apparatus for performing the above-described process of synthesizing a photographed image.

In FIG. 9, reference numeral 81 denotes a computer (hereinafter, also referred to as a PC) which stores image processing software for obtaining a high-resolution image on a hard disk (not shown) in the PC main body 82, and Sometimes PC81
The above software is executed. The software is not limited to the hard disk, but the PC
May be stored in the portable medium 80 driven by the disk drive 83 in the above.

FIG. 10 is a flowchart showing the execution of the software.

Here, one image A calculated by the positioning unit 53 is, for example, an image shown in FIG. 8A, and another image B is, for example, an image shown in FIG. It is assumed that it is stored in the hard disk of the PC 81.

First, in S90, the images A and B are read from the hard disk, and then in S91, the images A and B are read.
It is determined whether or not a stationary point light source serving as a reference has been detected from the above. If a stationary point light source is detected (the determination in S91 is YE
In S) and S92, arithmetic processing for alignment is performed.

Next, in step S93, an image synthesizing process for obtaining a high-resolution image is performed. In step S94, the processed image data is stored in the hard disk, and the processing ends.

When a stationary point light source is not detected (S
(No at 91), a warning is output at S95, and S96
Then, the image synthesizing process for obtaining the high-resolution image is stopped, and the process ends.

Instead of outputting a warning and stopping the image synthesizing process, ordinary positioning may be performed.

FIG. 11 is a block diagram showing the electrical configuration of a digital camera according to a second embodiment of the present invention. The same or corresponding parts as in FIG. Omitted.

In the digital camera shown in FIG. 11, a focusing area detecting section 68 is provided.
8 detects in-focus areas of a plurality of images from the RAM 52. The positioning unit 53 calculates the amount of displacement between the two images only in the detected in-focus region.

Next, a shooting sequence by the digital camera 1 having the configuration shown in FIG. 11 will be described with reference to FIGS.

Before photographing, the photographer uses the photographing mode setting key 17 and the liquid crystal panel 18 to set exposure conditions such as aperture priority and shutter speed priority, and to switch the white balance mode.

The user selects between the normal shooting mode and the super-resolution shooting mode using the image processing mode setting key 110 and the liquid crystal monitor 111. The settings for image processing are shown in FIG.
Is the same as

When the release button 13 is pressed in S301, in S302, the CPU 50 reads the photographing conditions and image processing mode settings at that time. In S303, the distance measurement data is fetched.

In S304, the CPU 50 determines whether or not the image mode is set to the super-resolution photographing mode.
When the normal photographing mode is set (NO in S304), in S305, the CPU 50 causes the in-focus area detecting unit 68, the positioning unit 53, and the high-resolution image generating unit 5 to operate.
7 is turned OFF.

Then, normal photographing is performed. That is, S
At 305, the photographing lens 1 is transmitted via the photographing lens driving unit 56.
Is driven to focus on an appropriate subject, and then S3
At 07, the CCD 12 is integrated, and at S308, the image signal is read out as data. The read image data is A / D
The data is converted into digital data by the converter 51, and S309
Then, it is temporarily stored in the RAM 52. Subsequently, S310
Then, the CPU 50 reads out the image data from the RAM 52 and stores it in the medium 15 in S311.

On the other hand, when the super-resolution photographing mode is set as the image processing mode (the determination in S304 is Y
ES), and proceeds to S321 in FIG. In S321, the CP
U50 turns on the combination area detection unit 68, the positioning unit 53, and the high-resolution image generation unit 57, and sets shooting conditions.

Further, in step S322, the CPU 50 drives the photographing lens 55 through the photographing lens driving unit 56 so as to focus on an appropriate subject.
And the first image is taken. Then, in S324,
The image data is converted into digital data by the A / D converter 51, and the image data is temporarily stored in the RAM 52 in S325.

Subsequently, in step S326, the CPU 50 fetches the photographing conditions. In step S327, the CPU 50 shifts the position of the CCD 12 by a predetermined amount via the CCD position control unit 54, and similarly performs photographing of the second image. That is, S329
In step S330, the image signal is read out and A / D converted. In step S331, the image data is stored in the RAM 5.
2 is temporarily stored.

Then, in S332, the CPU 50 reads the two pieces of image data from the RAM 52 and sends them to the in-focus area detecting section 68 and the positioning section 53. In S333, the CPU 50 determines whether or not the focus area detection unit 68 has detected the focus area of each image. When the in-focus area of each image is detected (the determination in S333 is YE
S), information of the detected in-focus area is sent to the positioning unit 53. If the in-focus area is not detected (NO in S333), the process waits until the in-focus area is detected.

Next, in step S334, the positioning unit 53 calculates the amount of positional deviation between the two pieces of image data only in the in-focus area. In step S335, the high-resolution image generation unit 57 performs a synthesis process for increasing the resolution based on the calculation result obtained by the positioning unit 53, and executes 1
The CPU 50 generates an image and stores it in the medium 15.

As described above, a specific region of a plurality of photographed images is detected as a focused region, and a plurality of images are aligned based on the focused region.
An image in which the position of the main subject matches exactly is synthesized,
Since the calculation area is specified and small, the processing time is reduced.

Next, the above-described positioning based on the in-focus area will be described with reference to a specific image shown in FIG.

As shown in FIGS. 14 (a) and 14 (b), when photographing a scene in which a house is located behind a person (back side) and a mountain is further behind the person, in order to obtain a high-resolution image, Suppose that multiple image shooting is performed. The main subject is a person, and it is the part where the person wants to perform high-resolution processing.

When combining a plurality of images (each of the images shown in FIGS. 14A and 14B) obtained by the multiple image shooting, a focused area is specified, and an area centered on the position, for example, FIG.
4 (b), the calculation of the alignment of the image data of only the area surrounded by the dotted line I is performed, and the image synthesis is performed based on the calculation result.

By doing so, for example, during portrait shooting, a high-resolution image of the main subject is preferentially obtained, and the processing time is shortened because the calculation area is smaller than the entire image area. .

The above image generation may be processed by the PC 81 shown in FIG.

FIG. 15 is a flowchart showing the execution of the software when processing is performed by the PC 81.

Here, it is assumed that the image A and another image B calculated by the positioning unit 53 are stored in the hard disk of the PC 81, respectively.

First, in steps S100, images A and B are read from the hard disk. Then, in step S101, a focused area is detected from the images A and B, and in step S102, the image data is subjected to a positioning calculation process. In step S103, an image synthesizing process for obtaining a high-resolution image is performed. In step S104, the processed image data is stored in the hard disk, and the process ends.

FIG. 16 is a block diagram showing the electrical configuration of a digital camera according to the third embodiment of the present invention. The same or corresponding parts as in FIG. .

The digital camera shown in FIG. 16 has an omnifocal photographing mode as a multiple image photographing mode, and includes an omnifocal image generator 77. The omnifocal image generation unit 77 generates an omnifocal image based on the calculation result obtained by the positioning unit 53.

At the time of photographing an all-focus image, the CPU 50 controls the diaphragm 79 via the diaphragm control unit 78 in order to photograph an image with a large depth of field for positioning separately from that for creating a multiplex image.
Is minimized, and the integration time at the minimum aperture is calculated from the data obtained by the photometry unit 710. Also, C
The PU 50 also functions as a unit that sets an image captured with a large depth of field as a reference image for positioning a plurality of other images.

The integration time control section 711 integrates the CCD 12 according to the integration time calculated by the CPU 50.

Next, a photographing sequence by the digital camera 1 according to the third embodiment will be described with reference to FIGS.

The photographer uses the photographing mode setting key 17 and the liquid crystal panel 18 to set exposure conditions such as aperture priority and shutter speed priority and switch white balance mode before photographing. An image processing mode setting key 110 and a liquid crystal monitor 111 are used to select a normal shooting mode or a super-resolution shooting mode. FIG. 17 shows the settings for the image processing mode. Image processing mode setting key 1
When 10 is turned off, normal shooting is performed. When turned on, omnifocal photography is performed. That is, a plurality of images, such as an image focused on the foreground and an image focused on the background, are captured and combined to generate an all-focus image.

First, when the release button 13 is pressed in S401, in SS402, the CPU 50 reads the photographing conditions and image processing mode settings at that time. In S403, distance measurement data is fetched.

Next, in S404, it is determined whether or not the image processing mode is set to the omnifocal image shooting mode.
If the normal shooting mode is set (NO in S404), the CPU 50 determines in S405 that the positioning unit 5
3 and the omnifocal image generation unit 77 are turned off.

Then, normal photographing is performed. That is, S
At 406, the CPU 50 drives the photographic lens 11 via the photographic lens driving unit 56 so as to focus on an appropriate subject, and then integrates the CCD 12 at S407. In S408, the image signal read from the CCD 12 is converted into digital data by the A / D converter 51, and is temporarily stored in the RAM 52 in S409. Subsequently, S410
Then, the CPU 50 reads out the image data from the RAM 52 and stores it in the medium 15 in S411.

On the other hand, when the omnifocal image shooting mode is set as the image processing mode (YES in S404), the process proceeds to S421 in FIG. In S421,
The CPU 50 turns on the positioning unit 53 and the omnifocal image generation unit 77 to capture the shooting conditions.

Further, in S422, after the CPU 50 drives the photographing lens 11 via the photographing lens driving section 56 so as to focus on an appropriate first subject, for example, a mountain shown in FIG. In step S423, the CCD 12 is integrated.
The first image is taken. Next, in S424, the image data is converted into digital data by the A / D converter 51,
In S425, it is temporarily stored in the RAM 52.

Subsequently, in S426, the CPU 50 captures the photographing conditions, and in S427, focuses the photographing lens 11 on a second object different from the first object, for example, a person via the photographing lens driving unit 56. After that, the second image is similarly photographed. That is, in S428, the CCD
In step S429, the image signal is converted into digital data by the A / D converter 51.
An image as shown in (b) is obtained. Then, in S430, the image data is stored in the RAM 52.

Further, in S431, the CPU 50 drives the photographing lens 11 via the photographing lens driving unit 56 so as to focus on the vicinity between the first subject and the second subject, and in S432, The aperture 79 is minimized via the aperture control unit 78 so as to maximize the depth of field, and in S433, the integration time for the minimum aperture is calculated from the data obtained by the photometry unit 710.

Then, in S434, the integration time control unit 7
11, the CCD 12 is integrated with the calculated integration time, and a third image is taken as a reference.
The image signal is converted into digital data by the A / D converter 51 to obtain, for example, an image as shown in FIG.
Then, the image is stored in the RAM 52 in S436.

Next, in S437, the CPU 50 reads out these three pieces of image data from the RAM 52 and sends them to the positioning unit 53. In step S438, the positioning unit 53 calculates the amount of displacement between the first and second shot image data with respect to the third shot image as the reference image.

In step S439, the omnifocal image generator 77 synthesizes the first and second captured images to generate an omnifocal image based on the calculation result obtained by the positioning unit 53. After synthesizing one image, the process proceeds to S411 in FIG.
5 is stored.

As described above, one of the three images shot in the multiple image shooting mode is used as the reference image, and the remaining images are aligned based on the reference image. For example, if one image having a large depth of field is used as a reference, even if there is a defocus in the image for creating a multiplex image, the image can be obtained regardless of the distance of the reference image.
The positions of a plurality of images can be easily adjusted.

Next, the above-described positioning based on an image having a large depth of field will be described with reference to a specific image shown in FIG.

As shown in FIG. 21 (a), when there is a mountain in the background of a person, multiple images are taken in order to obtain an image in which both the person and the mountain are in focus. FIGS. 21A and 21B show an image focused on a mountain and an image focused on a person, respectively, obtained by performing multiple image capturing. An all-in-focus image is obtained by performing a bonding process using only the in-focus portions of the two images.

However, the part of the person in FIG. 21A is largely out of focus, and the part of the person who is in focus as shown in FIG. Have difficulty.

Therefore, here, for positioning processing,
Further, another image having a large depth of field as shown in FIG. In the image, the person and the mountain are in focus to some extent, and the position of the person can be better understood as compared with FIG. In the image shown in FIG. 21C, both the position of the person and the position of the mountain are easy to understand.
The images of (a) and (b) can be easily positioned.

In this case, the aperture 79 is set to the minimum when the positioning image is taken. However, if the mountain and the person are focused to some extent as shown in FIG. You don't have to. It is desirable to adjust the aperture 79 at the time of photographing for positioning so that the photographing time is shortened and both the mountain and the person are focused to some extent.

The generation of the omnifocal image is also described in FIG.
The processing may be performed by the PC 81 shown in FIG.

FIG. 22 is a flowchart showing the execution of the software when processing is performed by the PC 81.

Here, the image A calculated by the positioning unit 53 is shown in FIG. 21A, the image B is shown in FIG.
Is as shown in FIG. 21C, and these images A, B, and C are
It is assumed that it is stored in the hard disk of the PC 81.

First, in step S110, the images A, B, and C are read from the hard disk. Then, in step S111, the reference image determination unit detects the image with the deepest depth among the images A, B, and C and determines the image as the reference image. (For example, image C is determined as a reference image).

Then, in step S112, the arithmetic processing for the positioning of the remaining images A and B is performed with reference to the image C.
In S113, an image combining process of images A and B is performed to obtain an all-focus image. In S114, the image as shown in FIG. 21D obtained by the combining process is stored in the hard disk, and the process ends.

In each of the above embodiments, a plurality of captured images are obtained by the digital camera 1, but a plurality of captured images may be obtained by an imaging device other than the digital camera 1. .

[0114]

According to the first aspect of the present invention, a shift between images caused by a camera shake or the like can be grasped from a shift of a point light source, and the reference is a point light source. Can be simplified, and the image processing can be shortened.

According to the second aspect of the present invention, a desired area in a plurality of images, for example, a specific area near the main subject is detected as an in-focus area, so that an image in which the main subject is properly aligned is synthesized. In addition, since it is sufficient to calculate the in-focus area, the processing time can be reduced.

According to the invention of claim 3, for example,
By referring to one image with a deep depth of field,
Even if the image for creating multiple images is out of focus,
Regardless of the distance between the reference images, the positions of the plurality of images can be easily adjusted.

According to the fourth aspect of the present invention, the positioning accuracy can be improved, the calculation for the positioning can be simplified, and the processing time can be greatly reduced. Further, the photographer can arbitrarily designate a point light source that can be regarded as a stationary state in a plurality of photographed images.

According to the fifth aspect of the present invention, by detecting the specific area as the in-focus area, it is possible to reliably synthesize an image in which the specific area is positioned, and to calculate the in-focus area. Processing time can be reduced.

According to the invention of claim 6, positioning can be performed by comparing a plurality of images with a reference image, so that calculation for positioning can be simplified and processing time can be shortened.

[Brief description of the drawings]

FIG. 1 is an external perspective view showing a digital camera according to a first embodiment of the present invention.

FIG. 2 is a rear view showing the same digital camera.

FIG. 3 is a block diagram showing an electrical configuration of the digital camera.

FIG. 4 is a table showing a setting example of an image processing mode of the digital camera.

FIG. 5 is a flowchart illustrating a shooting sequence according to the first embodiment.

FIG. 6 is a flowchart showing a shooting sequence following FIG. 5;

FIG. 7 is a flowchart showing a shooting sequence following FIG. 6;

FIG. 8 is an explanatory diagram when a high-resolution image is generated by the digital camera according to the first embodiment.

FIG. 9 is a front view illustrating a computer that performs a process for generating a high-resolution image.

FIG. 10 is a flowchart illustrating processing on a computer according to the first embodiment.

FIG. 11 is a block diagram illustrating an electrical configuration of a digital camera according to a second embodiment of the present invention.

FIG. 12 is a flowchart illustrating a shooting sequence according to the second embodiment.

FIG. 13 is a flowchart showing a shooting sequence following FIG. 12;

FIG. 14 is an explanatory diagram when a high-resolution image is generated by the digital camera according to the second embodiment.

FIG. 15 is a flowchart illustrating processing by a computer according to the second embodiment.

FIG. 16 is a block diagram showing an electrical configuration of a digital camera according to a third embodiment of the present invention.

FIG. 17 is a table showing a setting example of an image processing mode of the digital camera.

FIG. 18 is a flowchart illustrating an imaging sequence according to the third embodiment.

FIG. 19 is a flowchart showing a shooting sequence following FIG. 18;

FIG. 20 is a flowchart showing a shooting sequence following FIG. 19;

FIG. 21 is an explanatory diagram when an omnifocal image is generated by the digital camera according to the third embodiment.

FIG. 22 is a flowchart illustrating processing by a computer according to the third embodiment.

[Explanation of symbols]

 1 digital camera (imaging device) 12 CCD 50 (reference image setting means) 53 · Positioning means 58 ·························· Focusing area detecting means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) // H04N 101: 00 H04N 101: 00 (72) Inventor Katsuhito Shinkawa 2-chome Azuchicho, Chuo-ku, Osaka-shi, Osaka No. 3-13 Osaka International Building Minolta Co., Ltd. (72) Inventor Kazunori Sato 2-3-1 Azuchicho, Chuo-ku, Osaka-shi, Osaka F-term in Osaka International Building Minolta Co., Ltd. 5B057 BA02 CA08 CA12 CA16 CB08 CB12 CB16 CE08 5C022 AA13 AB28 AB55 AB68 AC03 AC32 AC42 AC69 CA02 5C023 AA11 AA27 AA31 AA37 AA38 BA01 BA09 BA12 BA13 CA03 CA08 DA04 DA08 EA03 5C076 AA11 AA19 AA21 BA01 5L096 CA04 CA09 DA01EA

Claims (6)

[Claims] 1. A system comprising: means for detecting a point light source in a stationary state included in a plurality of captured images of the same subject; and means for aligning a plurality of images based on the detected point light source. An image processing apparatus, comprising: 2. An image forming apparatus comprising: means for detecting a focus area of a plurality of captured images of the same subject; and means for aligning a plurality of images using image data of only the detected focus area. An image processing apparatus, comprising: And means for setting one of a plurality of captured images of the same subject as a reference image, and means for performing positioning of the remaining images based on the reference image. An image processing apparatus characterized by the above-mentioned. 4. An image processing method for aligning a plurality of captured images of the same subject, wherein the plurality of images are aligned with respect to a stationary point light source included in the plurality of captured images. Characteristic image processing method. 5. An image processing method for aligning a plurality of captured images of the same subject, wherein the plurality of images are aligned using image data of only a focused region of the plurality of captured images. Image processing method. 6. An image processing method for aligning a plurality of photographed images of the same subject, wherein a position of a plurality of remaining images is determined based on one of three or more photographed images. An image processing method comprising performing matching.