JPH11185018A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exactly overlap images by correcting the images so as to match the object positions of the respective images based on the feature position information of input images and the deviation between the respective coordinates of position information at correspondent points. SOLUTION: A feature point setting part 33 sets the plural feature points (coordinates) of the input image. Image data around the feature points are segmented as a template and inputted to a correspondent point search part 35. The correspondent point search part 35 searches a point corresponding to the template in an image different from the image inputted to the feature point setting part 33 and outputs the correspondent point coordinate position to a correspondent point position memory 36. A correction parameter calculating part 37 reads out the feature point coordinate in a memory 34 and the correspondent point coordinate in a memory 36 and calculates the position relation (parallel moving amount or rotating angle) between two images. An interpolation operating part 38 corrects the images based on the inputted position relation and outputs them to an image synthesizing part 39. The image synthesizing part 39 executes dynamic range expanding processing or object field depth expanding processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for synthesizing an image exceeding a dynamic range of an input device or a depth of field from a plurality of images obtained by photographing the same subject under different photographing conditions. The present invention relates to an image processing apparatus that performs position correction so that a subject coincides when superimposed.

[0002]

2. Description of the Related Art In general, a CCD (Charge C) is constructed by synthesizing a plurality of images obtained by photographing the same subject with different exposures.
image processing technology for creating an image having a dynamic range greater than the dynamic range inherent to an image pickup device such as an Oupled Device (for example, JP-A-63-23259).
No. 1) and an image processing technique of combining a plurality of images of the same subject photographed at different focal positions to create an image having a depth of field larger than the depth of field of the input optical system (Japanese Patent Application Laid-Open No. No. 1-309478).

[0003] The technique of creating a wide dynamic range image is realized by the configuration shown in FIG. Illustrated images A and B are images captured with different exposures. The relationship between the amount of incident light incident on the image sensor and the signal value of the image obtained by the image input device is shown in FIG. "[A] Long exposure" and "[B] Short exposure".

In the adder 11, the pixel values of the images A and B are added, and the value shown as "[C] addition signal" in FIG. The shape of the “[C] addition signal” is determined by the exposure ratio of two images. The linear converter 13 calculates the incident light amount I from the addition signal value S.
Is estimated, and the matrix circuit 15 converts the signal values of each of the R, B, and G colors into a luminance signal value Y.

[0005] Then, the luminance compression section 16 compresses the luminance value so that the estimated incident light quantity falls within a predetermined number of gradations, and outputs a compressed luminance signal value Y '. In the divider 17,
Y ′ / Y is calculated, and R,
The signal of each of G and B is multiplied by the multiplier 18,
The result is stored in the frame memory 19.

Since this image processing technique uses a plurality of image data, an image having better color reproduction and less noise can be obtained as compared with a case where a short-time exposure image is simply subjected to tone curve processing.

An image processing technique for creating an image having a depth of field greater than the depth of field of the input optical system described above is realized, for example, by the configuration shown in FIG. First, an image photographed by changing the focal position is weighted and added by the adder 21 and stored in the frame memory 22. It is known that blurring of the entire image is uniformed when weighted addition of an image whose focal position is changed is performed according to the focal position.

Therefore, in the recovery filter setting circuit 25,
A blur recovery filter for the image after the addition is determined and output to the matrix operation circuit 23. Here, the method of creating the blur recovery filter is not a part related to the essence of the present invention, and thus the description thereof is omitted. The matrix operation circuit 23 applies a restoration filter to the added image and stores the result in the frame memory 2.
4 is stored. As a result, the image stored in the frame memory 24 has a deeper depth of field than the imaging optical system.

[0009]

In the above-described image processing technique, a series of processes such as a dynamic range compression process and a restoration process are performed after a photographed image is once added or weighted and added for each pixel. for that reason,
If the images to be added are displaced from each other, they will be displaced when added, and accurate results cannot be obtained.

[0010] Therefore, in the case of the dynamic range expansion processing, since nondestructive reading is possible, a CMD (Charge Modurated Device) element which does not cause a displacement during photographing an image with a plurality of exposures in principle is used. Expensive equipment for business use was required, such as using a dedicated device, and in the process of expanding the depth of field, a robust stage was required. Further, when photographing is performed with the focal position changed, a slight change in magnification for each focal position also has an effect.

On the other hand, digital cameras, scanners, tripods, and the like used by general users are mostly simple ones that cannot be expected to have higher accuracy than commercial equipment. Therefore, the advantages of the image processing technique described above cannot be fully utilized.

For example, when taking a picture with a digital camera mounted on a tripod, the camera moves when the shutter is pressed, or when taking a picture taken with a silver halide camera with a film scanner, depending on how the film is inserted, And so on.

Accordingly, the present invention provides an image processing for detecting a positional relationship between a plurality of images having different photographing conditions such as an exposure and a focus position, and correcting the images so as to be accurately overlapped by a parameter value indicating the detected positional relationship. It is intended to provide a device.

[0014]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an image processing apparatus for performing a positioning process between a plurality of images obtained by shooting substantially the same composition (imaging area) under different imaging conditions. A search area setting unit that divides at least one image of the input image into a plurality of divided areas and provides a search area for a feature point representing a feature of an image in each of the divided areas; First position information extracting means for extracting characteristic position information of a subject in the composition in association with coordinates for each divided region of the divided image based on a feature amount of the image; A second position information for searching for a point corresponding to the characteristic position information for each of the input images, and extracting the obtained corresponding point as position information associated with coordinates for the image. Extraction means, correction parameter detection means for detecting a correction parameter based on a deviation between each coordinate of the extracted characteristic position information and position information of the corresponding point, and each image based on the detected correction parameter. And an image correcting unit for correcting an image so that the subject positions coincide with each other.

In the image processing apparatus having the above configuration, one image is selected from a plurality of input images, the image is divided into a plurality of regions, and feature points are set from the feature amounts of the divided regions. Then, the coordinate position of the feature point is stored. Based on the coordinates of the feature point, a corresponding point is searched for each of the other input images that have not been selected, and the coordinate position of the corresponding point is extracted. After calculating the positional relationship (parallel movement amount, rotation angle), correcting one or both images to correct the alignment so that the subject matches, dynamic range compression processing, depth of field expansion processing, etc. Is applied.

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows an example of the configuration of an image processing apparatus according to a first embodiment of the present invention.

In this embodiment, an image photographed by a digital camera (not shown) or an image photographed by a silver halide camera and photographed by a scanner or the like is recorded on a storage medium such as a hard disk, a portable floppy disk, or a memory card. ing.

The present embodiment is roughly divided into an image data reproducing unit 31 for reading out an image recorded on the hard disk, floppy disk, memory card or the like and performing processing such as decompression, An image correction unit 30 that performs an interpolation operation on the input image by a method described later to correct the positional relationship of the subject in the image. An image synthesizing unit 39 for executing depth of field expansion.

The image correcting section 30 includes a frame memory 32 for temporarily storing the image data subjected to the decompression processing by the image data reproducing section 31, and a plurality of features described later for one selected image. A feature point setting unit 33 for setting a point (coordinate), a feature point position memory 34 for storing the feature point coordinates, and a feature point coordinate and an image data around the feature point cut out as a template in another image. A corresponding point search unit 35 for searching for a point to be matched, a corresponding point position memory 36 for storing the obtained corresponding point coordinate position, and a positional relationship between the two images based on the feature point coordinates and the corresponding point coordinates (parallel movement amount, Correction parameter calculator 3 for calculating the rotation angle)
7 and an interpolation calculation unit 38 that corrects one or both images based on the input positional relationship and outputs the corrected image to the image synthesis unit 39.

In the image processing method of the present embodiment configured as described above, the image read out from the hard disk or the memory card or the like is subjected to processing such as decompression by the image data reproducing unit 31, and the image correcting unit 30 are stored in the frame memory 32. Here, the data output from the image data reproducing unit 31 includes not only the pixel value of the image but also additional data such as the image size, the number of colors, and a color map.

Next, one of the input images is selected (here, image A) and input to the feature point setting section 33.
The feature point setting unit 33 sets a plurality of feature points (coordinates) by a method described later, and stores the feature point coordinates in the feature point position memory 3.
4 is input. The set feature point coordinates are also input to the corresponding point search unit 35, and at the same time, image data around the feature points is cut out as a template and input to the corresponding point search unit 35.

The corresponding point search unit 35 includes a feature point setting unit 33
A point corresponding to the input template is searched for in an image (here, image B) different from the image input to. A method for searching for a corresponding point will be described later.

As a result, the corresponding point coordinate position is output to the corresponding point position memory 36. The correction parameter calculation unit 37 reads the feature point coordinates in the feature point position memory 34 and the corresponding point coordinates in the corresponding point position memory 36, and calculates the positional relationship (parallel movement amount, rotation angle) between the two images. Then, the interpolation calculation unit 38 corrects one or both images based on the input positional relationship and outputs the corrected image to the image synthesis unit 39.

The image synthesizing section 39 executes the above-described dynamic range expansion processing and depth of field expansion processing. However, since the target image is an image whose position has been adjusted by the image correction section 30 in the preceding stage. Of course, a correctly superimposed image is obtained as a result image.

FIG. 2 shows the configuration of the feature point setting section 33 and will be described. The feature point setting unit 33 determines a plurality of feature point positions. However, when many feature points are concentrated on a part of the image, a positioning error may increase in other parts. Therefore, the region dividing unit 41 divides the image into a plurality of regions and sets a feature point search region inside the region so that the feature points are appropriately dispersed in the image. Here, FIG. 3A shows an example in which the whole is equally divided.

A small area which is a candidate for a template is moved in this area, and the position where the feature amount becomes the maximum is set to one of the feature points.
Output as one. However, since it takes a very long processing time to search the entirety, it is desirable to set a feature point search area smaller than the divided area as shown in FIG. The size of the feature point search area is predetermined based on the image size, for example, 5% of the image size. Alternatively, the size of the divided small area may be used as a reference.

Next, the template position determining section 42 determines a template candidate position in the feature point search area, and the template extracting section 43 cuts out image data in the template candidate based on the position. At this time, the template position determination unit 42 sequentially moves the position of the template candidate in the feature search area.

Since the change in the feature value of a template candidate that is close in position may not be so large, in such a case, the position of the template candidate can be shifted not every pixel but every few pixels. As a result, the amount of calculation can be reduced.

The feature amount calculation unit 44 calculates the feature amount of the template candidate, and the feature amount determination unit 45 determines the template position where the feature amount is maximum. In the template extraction unit 46, the data around the position where the feature amount becomes maximum is
Extract as a template. The template extracting unit 46 may have the role of the template extracting unit 43. As the feature amount, for example, a pixel value variance as shown in Expression (1) can be used.

[0030]

(Equation 1) Here, ai is the pixel value, A is the average of the pixel values in the template candidate, and the sum is taken for the entire template candidate. Although no specific formula is given, a secondary pixel value variance and an edge detection result are also preferable values as feature amounts.
At this time, the value T is given by the following equation:

[0031]

(Equation 2) If the value is smaller than a certain specified value Tth, the reliability is low and is not adopted. For example, in a portion where the change in the pixel value is small, such as an empty portion in FIG. 3B, the accuracy of matching, which will be described later, generally decreases, but the method according to the present invention has a feature having a value T as in equation (2). No feature point is set at the point candidate position, and it is possible to prevent a decrease in the accuracy of the positional relationship parameter.

FIG. 4 is a diagram for explaining the configuration of the corresponding point search unit 35. The search area setting unit 51 sets the position and size of the area for searching for the corresponding point based on the feature point coordinates read from the feature point setting unit 33. Correlation calculator 52
Then, based on the information in the search area set by the search area setting unit 51, the image data around the corresponding point candidate position is stored in the frame memory 32b so as to have the same size as the template input from the feature point setting unit 33. The correlation value with the template data input from the feature point setting unit 33 is calculated by cutting out the image data inside.

The correlation value judging section 53 judges a position having the highest correlation with the template extracted by the template extracting section 46, and uses the result as a corresponding point position to store the corresponding point position memory 3.
6 is output.

At this time, in general, the corresponding point candidate is moved for each pixel to obtain a correlation value. However, in order to detect a corresponding point position with an accuracy equal to or less than a pixel, the corresponding point candidate is interpolated with reference to surrounding correlation values. The point position may be determined. As the correlation value, the sum of absolute values of the difference shown in the following equation (3), the cross-correlation shown in the equation (4), the normalized cross-correlation shown in the equation (5), and the like can be used.

[0035]

(Equation 3) Here, bi and B are pixel values in a small area moving in the search area of the image B and their average, σ _A and σ _B are standard deviations of the template data, and σ _A ² and σ _B ² Is the variance.

When calculating the above equations (2) to (4), high-speed processing can be performed by calculating only a certain representative color. If the corresponding points are searched by adding the correlation values of the R, G, and B colors, the corresponding points can be searched with higher accuracy.

Then, the correction parameter calculation unit 37 calculates the image A, the image A from the feature point coordinate data and the corresponding point coordinate data.
The positional relationship between B is determined. The required parameter is (6)
Cos θ, sin θ, sx, sy, and M in the equation.

[0038]

(Equation 4) Here, (x, y) indicates a feature point position, (X, Y) indicates a corresponding point position, and M corresponds to a relative magnification between images. The feature point position memory 34 and the corresponding point position memory 36 include:
Since a plurality of data are stored, if the parameters are obtained in a least-square manner, the following equation (7) is obtained.

[0039]

(Equation 5) Here, N is the number of feature points. From equation (8), M, cosθ, s
inθ is determined as in equation (9).

[0040]

(Equation 6)

Also, originally, cos θ and sin θ are cos ² θ +
Since there is a relationship of sin ² θ = 1, if it can be assumed that there is no change in magnification in the target image, M = 1.0 in the result of Expression (9).
And when the relationship is as shown in equation (10), (11)
It is better to correct cosθ and sinθ as in the formula. s
x and sy are calculated from the corrected cos θ and sin θ.

[0042]

(Equation 7)

In the above equations (6) to (9), the arbitrary origin is set as the rotation center of the image.
The center of gravity of each corresponding point position may be considered to coincide. The center of gravity of the feature point is (/ x, / y) (where /
Is used as a symbol indicating the average. As shown in the image information of each equation, the average is represented as an overbar in each equation), and the position of the center of gravity of the corresponding point is given by (/ X, / Y), and cosθ, sinθ, sx,
sy is given by equations (12) and (13), respectively.

[0044]

(Equation 8)

As described above, according to the present embodiment, a plurality of feature points are extracted from an image and are automatically aligned, so that a dynamic range can be expanded even by a general user without using expensive equipment for business use. The effect of the processing and the depth of field expansion processing can be used to the maximum. However, when distortion occurs in the image to be processed, it is necessary to correct the distortion in advance by an arbitrary method.

Next, a second embodiment according to the present invention will be described. This embodiment is different from the first embodiment in the internal configuration of the feature point setting unit 33.

FIG. 5A is a diagram showing a conceptual configuration of the feature point setting unit 33. This configuration is different from the configuration shown in FIG. 1 in that an area specifying unit 81 is added. FIG. 5B is a diagram illustrating a configuration example of the area specifying unit 81.
In the area designating section 81, a plurality of areas of interest are designated by the area designating section 83 with respect to the input image data. The feature points are set in the area specified here by the above-described method.

The display control section 84 overlays the designated area (data from the area position memory 85 described later) on the input image data as shown in FIGS. 6A and 6B, for example. The data is displayed in the section 82.
The display unit 82 may be a display attached to the PC or a dedicated unit may be separately provided. Further, the area instruction section 83 may be instructed by a mouse attached to the PC, or may be separately provided. The display control unit 84 includes:
Although described as a part of the configuration of the area designation unit 81, an operating system (O
A configuration using the function of S) may be used.

Further, the region designating section 83 may designate a region of interest by coordinates in the image. Then, the area position memory unit 85 stores the position and size of the area specified by the area specifying unit 83 and outputs it to the area dividing unit 41. The area dividing section 41 sets a feature point in the designated area by the above-described method based on the area position information from the area position memory section 85.

According to the configuration of the present embodiment, it is possible to improve the positioning accuracy of a portion that the user wants to pay attention to, and it is not necessary to search for a feature point in a portion that is not necessary so that the calculation time can be reduced.

When a moving subject such as a person or a vehicle is included in a captured image, an error occurs in the alignment of the entire image when an image of a moved portion is used for calculation. However, in the configuration of this embodiment, It is possible not to use such data.

Further, by specifying a specific area among the plurality of areas specified by the area specifying unit 83 and setting more feature points only inside the specific area than the other areas, the positioning accuracy of the specific area is further improved. It is possible.

In the case of the dynamic range enlargement technique, the images are arranged in the order of the exposure value, and in the case of the depth of field enlargement technique, the images are arranged in the order of the focal position, and the above operation is performed between adjacent images. Accuracy can be improved as compared with a case where an image is selected and executed at random.

FIG. 7 is a diagram showing a third embodiment of the present invention. The configuration of the image correction unit 30 illustrated in FIG. 7 includes a display unit 82, a display control unit 84, and a feature point instruction unit 90 in addition to the configuration of the image correction unit 30 illustrated in FIG. A plurality of feature point positions are set by the above-described method and stored in the feature point memory 34.

The feature point coordinates stored in the feature point memory 34 are sent to the feature point designating section 90, and the position is overlaid on the target image on the display section 82 through the display control section 84 (for example, FIG. b)) At this time, if the feature point is set on a moving subject (moving object) such as a person or a vehicle, it causes an error in the correction coefficient calculated by the equations (6) to (10). Therefore, when a moving object is included, the feature point instruction unit 90 detects the object, excludes the feature points set on the moving object from data used for calculation, and stores the data from the feature point memory 34. delete.

In this operation, the user may judge the displayed data and indicate the data to be excluded from the feature point designating section 90. After the corresponding point search, the data in the feature point memory 34 and the corresponding point memory 36 are simultaneously displayed on the display unit 82.
And if the user determines that no correspondence is taken, the feature point designating unit 90 may be used to give an instruction to exclude the data.

Next, a fourth embodiment according to the present invention will be described with reference to FIG. In the fourth embodiment, the area specifying unit 81 in FIG. 5 is changed to a configuration as shown in FIG.

The area designating section 81 shown in FIG. 8 comprises an area designating section 83, an area position memory 85, and a state determining section 101. In the present embodiment, the range specified in the feature point setting area is determined based on the statistical and physical properties of the image.

First, the entire image is divided into a predetermined size as shown in FIG. At this time, the size to be divided is set to a range larger than the template cut out by the feature point setting unit.

Each of the divided parts is sent to the state determination unit 101, and it is determined whether or not to specify a feature point setting area.
The state determination unit 101 scans the input image data,
If the number of saturated pixels exceeds a certain ratio, it is determined that the pixel is not suitable as a feature point setting area.

When the average value of the input data exceeds a certain threshold value, the state determination unit 101 may determine that the input data is not suitable as a feature point setting area. Area indicator 8
3 sequentially outputs the image data of the divided areas to the state determination unit 101 as shown in FIG. 9B, and includes the range when it is determined that the image data is suitable as the feature point setting area. It is recorded in the area position memory 85.

In the above description, the determination is made for each divided area. However, the entire area recorded in the area position memory 85 may be used as the image data to be output to the state determination unit. Further, as shown in FIG.
In step 3, the user may specify a certain divided area and sequentially include the divided areas in the feature point specification area from the surrounding area.

Further, as shown in FIG. 10, the user designates one point in the area designating section 83, assumes a rectangle extended around the point, and uses the image data as the state determining section 101.
May be determined.

Although the saturated pixel value is used as a criterion, the state determination unit 101 determines the power of the high-frequency component when an orthogonal transform such as discrete Fourier transform or discrete cosine transform is applied to the input image, The determination may be made based on the components of the wavelet transform.

Further, the state determination unit 101 may use the contrast of the input data or the difference between the maximum and minimum pixel values for the determination. Next, a fifth embodiment according to the present invention will be described with reference to FIG.

This embodiment is an example in which three or more images are to be processed. Here, in the drawings, the same components as those in FIG. 1 described above are denoted by the same reference numerals, and description thereof will be omitted.

In this embodiment, the image data reproducing unit 3
The image data reproduced in 1 is input to one of the frame memories 32a and 32b by the image switching unit 111 which performs a switching operation according to the instruction of the image selection instruction unit 114. The image selection instructing unit 114 is configured to input an image to the two frame memories alternately in consideration of the processing time of the apparatus so that a plurality of input images of the same subject can be sequentially processed. It controls the switching unit 111.

The feature points / corresponding points are detected from the images stored in the frame memories 32a and 32b by the above-described method, and the correction parameter calculator 37 determines the positional relationship between the images (parallel movement amount / rotation angle / relative magnification). Is calculated. The parameter calculated by the correction parameter calculation unit 37 is the image I
D and the reference image ID are stored in the correction parameter storage unit 113.

The correction parameter conversion unit 112 reads the data stored in the correction parameter storage unit 113, and converts each data into a translation amount and a rotation angle as viewed from a pre-specified correction reference image. Subsequently, in the interpolation calculation unit 38, the correction parameter conversion unit 112
The image is corrected according to the parameters viewed from the correction reference image.

Here, the data conversion in the correction parameter conversion unit 112 and the correction in the interpolation calculation unit 38 are performed by the image switching unit 1.
The operation may be performed for each of the 11 operations, or the image may be read again into the frame memory 32 after calculating the relative positional relationship for all the target images.

In this embodiment, the frame memory 32
In the above description, the input image is switched by the signal switching unit 111. However, the number of frame memories that can store all target images may be prepared.

Although the above embodiments have been described, the present invention includes the following inventions. (1) In a plurality of images having different photographing conditions, the image is divided into a plurality of divided areas, and a feature point search area that is the same as or smaller than each of the divided areas is provided in the divided areas. The characteristic position information of the subject is extracted based on the statistic of, and a corresponding point is searched in an image different from the image from which the characteristic position information of the subject is extracted among the plurality of images. Correction parameters (rotation, movement, magnification change, etc. between images) are detected from the dynamic position information and the position information of the corresponding points, and the image is corrected based on the rotation, movement, magnification change between the images as the correction parameters. Image processing device.

The present invention corresponds to the first embodiment. Thereby, when a plurality of images are superimposed, more accurate superimposition of images can be performed. (2) In the image processing device according to the above (1),
When the image is corrected based on the rotation / movement / magnification change, the parameter is converted so as to represent the rotation / movement / magnification change viewed from a specific reference image, and then the correction is performed.

The present invention corresponds to the fifth embodiment. Thereby, when a plurality of images are superimposed, an appropriate arbitrary correction parameter can be set for each image, and more accurate superimposition of images according to the photographer's intention can be performed. . (3) The image processing apparatus according to (1), wherein, for the plurality of divided regions, a region to be divided is limited to a part of an image.

The present invention corresponds to the second embodiment. Thereby, when a plurality of images are superimposed, the speed of image processing is improved, and an appropriate region can be set for each image, and more accurate image superposition according to the photographer's intention is performed. Will be able to do it. (4) In the image processing device according to the above (1) or (3), among the characteristic position information of the subject based on the statistics of the image, the subject whose position moves among a plurality of images. The position information set above is not used for detecting rotation, movement, and change in magnification between images.

The present invention corresponds to the third embodiment. Accordingly, when overlapping a plurality of images, the region where the moving object exists is excluded, so that more accurate image overlay can be performed. (5) In the image processing device according to the above (3),
The restricted area for limiting the area to be divided to a part of the image is any one of a ratio of high-frequency components when orthogonally transformed, a ratio of saturated pixels, an average value of pixel values, a difference between maximum and minimum pixel values, and contrast. Is limited based on the value of.

The present invention corresponds to the fourth embodiment. Thereby, when a plurality of images are superimposed, the speed of the image processing is improved, and an appropriate region can be automatically set for each image, and according to the photographer's intention,
More accurate superimposition of images can be performed. (6) In the image processing device according to the above (5),
It is characterized in that the restricted area is expanded around a specific point.

The present invention corresponds to the fourth embodiment. Accordingly, when a plurality of images are superimposed, an appropriate area is set for each image centering on a specific point, so that more accurate superimposition of images can be performed.

[0079]

As described above in detail, according to the present invention, a positional relationship between a plurality of images having different photographing conditions such as an exposure and a focus position is detected, and a parameter value indicating the detected positional relationship is accurately detected. An image processing device that corrects images so that they overlap can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to a first embodiment.

FIG. 2 is a diagram illustrating a configuration example of a feature point setting unit illustrated in FIG. 1;

FIG. 3A is a diagram illustrating divided regions obtained by dividing an image into a plurality of regions, and a feature point search region set in each divided region;
FIG. 3B is a diagram for explaining a feature point.

FIG. 4 is a diagram illustrating a configuration example of a corresponding point search unit illustrated in FIG. 1;

FIG. 5A is a diagram illustrating a configuration example of an image processing apparatus according to a second embodiment, and FIG. 5B is a diagram illustrating a configuration example of an area specifying unit;

FIG. 6A is a diagram illustrating an example of a designated area displayed on an image, and FIG. 6B is a diagram illustrating a target position displayed on the image.

FIG. 7 is a diagram illustrating a configuration example of an image processing apparatus according to a third embodiment.

FIG. 8 is a diagram illustrating a configuration example of an area specifying unit used in an image processing apparatus according to a fourth embodiment.

9A is a diagram illustrating an input image divided into a plurality of divided regions, FIG. 9B is a diagram for describing designation of a feature point setting region by scanning, and FIG. c)
FIG. 9 is a diagram for describing designation of a feature point setting area by extension.

FIG. 10 is a diagram for describing designation of a rectangle extended from a designated point as a feature point setting area.

FIG. 11 is a diagram illustrating a configuration example of an image processing apparatus according to a fifth embodiment.

FIG. 12 is a diagram showing a configuration example for creating a conventional wide dynamic range image.

FIG. 13 is a diagram illustrating a configuration example for creating an image having a conventional deep depth of field.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 30 ... Image correction part 31 ... Image data reproduction part 32 ... Frame memory 33 ... Feature point setting part 34 ... Feature point position memory 35 ... Corresponding point search part 36 ... Corresponding point position memory 37 ... Correction parameter calculation part 38 ... Interpolation calculation part 39 ... Image synthesis unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁶ Identification code FIG06F 15/70 350B

Claims (3)

[Claims] 1. An image processing apparatus for performing a registration process between a plurality of images obtained by photographing substantially the same composition under different photographing conditions, wherein at least one of the input images is divided into a plurality of divided regions. Search area setting means for providing a search area for a feature point representing a feature of an image in the divided area; characteristic position information of a subject in the composition based on the feature amount of the image from the search area of each of the feature points First position information extracting means for extracting the position information in association with the coordinates of each divided region of the divided image, and for each of the input images other than the divided image, the characteristic information corresponding to the characteristic position information. Second position information extracting means for searching for a point and extracting the obtained corresponding point as position information associated with the coordinates of the image; and the extracted characteristic position information and the corresponding position information. Correction parameter detection means for detecting a correction parameter based on a deviation between coordinates of point position information, and image correction means for correcting an image based on the detected correction parameter so that a subject position of each image matches. An image processing apparatus comprising: 2. The position information extracted by the first and second position information extracting means is position information excluding position information relating to coordinates of a divided area where an object moving in the image is detected. The image processing apparatus according to claim 1, wherein: 3. The position information extracted by the first and second position information extracting means includes a ratio of a high-frequency component and a ratio of a saturated pixel when each of the divided regions in the image is orthogonally transformed. Extracting coordinate-related position information for a desired divided region from the input image based on one of a mean value of pixel values, a difference between maximum and minimum pixel values, and a contrast. 3. The image processing device according to 2.