JP2008199658A - Image processor and image processing method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: that if there is a positional deviation of an object among a plurality of images where the same object is photographed, deviation or the like occur during addition and the advantages of dynamic range compression processing or depth-of-field expansion processing can not be effectively utilized. <P>SOLUTION: An image processor and image processing method thereof are provided in which, in order to perform registration processing between images resulting from photographing approximately identical composition on different photographic conditions, at least one image of a plurality of input images is divided into a plurality of divided regions, a search region of a feature point representing features of the image is set within each divided region and based on a result of comparison between a calculation value obtained by calculation using a pixel value of a pixel included in the search region and a certain specific value, the search region for actually searching for the feature point is selected from among a plurality of provided search regions. On the basis of a feature quantity of that image, characteristic position information of the object within the composition is extracted from the selected search region while relating the characteristic position information of the object within the composition to coordinates of divided images for the divided regions. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to composition processing of images exceeding the dynamic range and depth of field of an input device from a plurality of images obtained by shooting the same subject under different shooting conditions, and in particular, the subject matches when the images are superimposed. The present invention relates to an image processing apparatus that performs alignment correction and an image processing method thereof.

  In general, an image processing technique for creating an image having a dynamic range that is greater than or equal to a dynamic range unique to an image sensor such as a CCD (Charge Coupled Device) by combining a plurality of images obtained by photographing the same subject with different exposures (For example, Patent Document 1) or an image processing technique for creating an image having a depth of field deeper than the depth of field of the input optical system by combining a plurality of images obtained by photographing the same subject at different focal positions. (For example, Patent Document 2).

  This technique for creating a wide dynamic range image is realized by the configuration shown in FIG.

Images A and B shown in the figure are images taken with different exposures, and the relationship between the amount of incident light incident on the image sensor and the signal value of the image obtained by the image input device is shown in FIG. “[A] Long time exposure”, “[B] Short time exposure”.
In the adder 11, the pixel values of the images A and B are added, and a value indicated as “[C] addition signal” in FIG. 12B is stored in the frame memory 12.

  The shape of this “[C] addition signal” is determined by the exposure ratio of the two images. The linear conversion unit 13 estimates the incident light amount I from the added signal value S, and the matrix circuit 15 converts the signal values of R, B, and G colors into luminance signal values Y.

Then, the luminance compression unit 16 compresses the luminance value so that the estimated incident light quantity falls within the predetermined number of gradations, and outputs the compressed luminance signal value Y ′. In the divider 17, Y ′ / Y is calculated, multiplied by the signals of R, G, and B, which are the outputs of the linear converter 16, by the multiplier 18, and the result is stored in the frame memory 19.
Since this image processing technique uses a plurality of image data, an image with good color reproduction and less noise can be obtained as compared with a case where a short-time exposure image is simply subjected to tone curve processing.

Further, the image processing technique for creating an image having a depth of field deeper than the depth of field of the input optical system described above is realized, for example, with the configuration shown in FIG.
First, images taken with different focal positions are weighted and added by the adder 21 and stored in the frame memory 22. It is known that blurring of the entire image is made uniform by weighted addition of an image with the focal position changed according to the focal position.

Therefore, the recovery filter setting circuit 25 determines a blur recovery filter in the image after the addition, and outputs it to the matrix operation circuit 23. Here, the method for creating the blur recovery filter is not a part related to the essence of the present invention, and therefore will be omitted. The matrix operation circuit 23 applies a recovery filter to the added image and stores the result in the frame memory 24.
As a result, the image stored in the frame memory 24 is an image having a deeper depth of field than the imaging optical system.
Japanese Unexamined Patent Publication No. 63-232591 JP-A-1-309478

  In the image processing technique described above, a captured image is temporarily added or weighted for each pixel, and then a series of processing such as dynamic range compression processing and recovery processing is performed. For this reason, if the images to be added are misaligned with each other, they are shifted when added, and an accurate result cannot be obtained.

  For this reason, non-destructive readout is possible in the case of dynamic range expansion processing. Therefore, in principle, a device using a CMD (Charge Modulated Device) element that does not cause a positional shift during image capture with a plurality of exposures. In the process of using and increasing the depth of field, an expensive apparatus for business use is required such as a solid stage is required. Further, when shooting with the focal position changed, a subtle change in magnification for each focal position also affects.

On the other hand, digital cameras, scanners, tripods, etc. used by general users are mostly simple ones that cannot be expected to be accurate compared to commercial equipment. Therefore, the above-described advantages of the image processing technique cannot be fully utilized.
For example, when shooting with a digital camera on a tripod, the camera moves when you press the shutter, or when you take an image taken with a silver salt camera with a film scanner, they will deviate from each other depending on how you insert the film Etc.

  Therefore, the present invention detects a positional relationship between a plurality of images having substantially the same composition and different shooting conditions such as exposure and focal position, and corrects the images so that they are accurately overlapped by a parameter value indicating the detected positional relationship. An object is to provide an image processing apparatus and an image processing method thereof.

In order to achieve the above-mentioned object, the present invention is an image processing apparatus that first performs alignment processing between a plurality of images obtained by photographing substantially the same composition under different photographing conditions, and includes at least one input image. A search area setting unit that divides an image into a plurality of divided areas and provides a search area for a feature point representing a feature of the image in each divided area, and pixels of pixels included in the search area provided in the divided area Search area selection for selecting a search area for actually searching for a feature point from a plurality of the provided search areas based on a comparison result between a calculated value calculated using a value and a specified value And position information extracting means for extracting characteristic position information of the subject in the composition from the selected search area in association with coordinates for each divided area based on the features of the image. And To provide an image processing apparatus to obtain.
Second, an image processing method for performing alignment processing between a plurality of images obtained by capturing substantially the same composition under different imaging conditions, wherein at least one of the input images is divided into a plurality of divided regions, There are a search area setting step for providing a search area for feature points representing image features in each divided area, and a calculated value calculated using pixel values of pixels included in the search area provided in the divided area. Based on a comparison result with a prescribed value, a search region selection step for selecting a search region for actually searching for a feature point from a plurality of the search regions provided, and based on the feature amount of the image A position information extracting step of extracting characteristic position information of a subject in the composition from the selected search area in association with coordinates for each divided area of the divided image.

  According to the present invention, a positional relationship between a plurality of images having different shooting conditions such as exposure and focal position with almost the same composition is detected, and correction is performed so that the images are accurately overlapped by a parameter value indicating the detected positional relationship. An image processing apparatus and an image processing method therefor can be provided.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows a configuration example of an image processing apparatus according to the first embodiment of the present invention.

  In this embodiment, an image taken with a digital camera (not shown) or an image taken with a silver halide camera and taken with a scanner or the like is recorded on a storage medium such as a hard disk, a portable floppy disk, or a memory card. Has been.

  The present embodiment can be broadly divided into an image data reproduction unit 31 that reads an image recorded on the hard disk, floppy (registered trademark) disk, memory card, and the like, and performs processing such as decompression. An image correction unit 30 that performs an interpolation operation on the input image by a method to be described later to correct the positional relationship of the subject of the image, and for the image whose positional relationship is corrected by the image correction unit 30, And an image composition unit 39 that executes depth of field expansion.

  The image correction unit 30 also includes a frame memory 32 that temporarily stores the image data that has been decompressed by the image data reproduction unit 31, and a plurality of feature points (coordinates) that will be described later for one selected image. ), A feature point position memory 34 for storing the feature point coordinates, and a corresponding point in another image from the feature point coordinates and image data around the feature point cut out as a template. Corresponding point search unit 35 for searching, corresponding point position memory 36 for storing the obtained corresponding point coordinate position, and positional relationship (parallel movement amount, rotation angle) between two images from the feature point coordinates and the corresponding point coordinates A correction parameter calculation unit 37 for calculating the image and an interpolation calculation unit 38 that corrects one or both images from the input positional relationship and outputs the corrected image to the image composition unit 39.

  In the image processing method of the present embodiment configured as described above, an image read from the hard disk, memory card, or the like is subjected to processing such as decompression by the image data reproduction unit 31, and the frame of the image correction unit 30 Stored in the memory 32. Here, the data output from the image data reproducing unit 31 includes not only the pixel value of the image but also additional data such as the image size, the number of colors, and a color map.

Next, one of the input images is selected (here, image A) and input to the feature point setting unit 33.
The feature point setting unit 33 sets a plurality of feature points (coordinates) by a method described later, and the feature point coordinates are input to the feature point position memory 34. The set feature point coordinates are also input to the corresponding point search unit 35, and simultaneously, image data around the feature point is cut out as a template and input to the corresponding point search unit 35.

The corresponding point search unit 35 searches for a point corresponding to the input template in an image different from the image input to the feature point setting unit 33 (here, the image B). A method for searching for corresponding points will be described later.
As a result, the corresponding point coordinate position is output to the corresponding point position memory 36. The correction parameter calculation unit 37 reads the feature point coordinates in the feature point position memory 34 and the corresponding point coordinates in the corresponding point position memory 36, and calculates the positional relationship (parallel movement amount, rotation angle) between the two images. The interpolation calculation unit 38 corrects one or both images from the input positional relationship and outputs the corrected image to the image composition unit 39.

  The image composition unit 39 performs the above-described dynamic range expansion processing and depth-of-field expansion processing. However, since the target image is an image that has been aligned by the previous image correction unit 30, it is naturally accurate. An image superimposed on is obtained as a result image.

FIG. 2 shows and describes the configuration of the feature point setting unit 33.
The feature point setting unit 33 determines a plurality of feature point positions. However, when many feature points are concentrated on a part of the image, an alignment error may increase in other parts. In view of this, the region dividing unit 41 divides the image into a plurality of regions and sets a feature point search region therein so that the feature points are appropriately scattered in the image. Here, FIG. 3A shows an example in which the whole is divided equally.

  A small region that is a template candidate is moved in this region, and the position where the feature amount is maximized is output as one of the feature points. However, since it takes a very long processing time to search the whole, it is desirable to set a feature point search area smaller than the divided area as shown in FIG. The size of the feature point search area is determined in advance based on the image size, for example, 5% of the image size. Alternatively, the size of the divided small area may be used as a reference.

  Next, the template position determination unit 42 determines a template candidate position in the feature point search region, and the template extraction unit 43 cuts out image data in the template candidate based on the position. At this time, the template position determination unit 42 sequentially moves the position of the template candidate within the feature search area.

  Since the change in the feature amount of the template candidate that is close in position may not be so large, in such a case, the position of the template candidate can be shifted every several pixels instead of every pixel. The amount of calculation can be reduced.

The feature amount calculation unit 44 calculates the feature amount of the template candidate, and the feature amount determination unit 45 determines the template position where the feature amount is maximized. The template extraction unit 46 extracts data around the position where the feature amount is maximized as a template. The template extracting unit 46 may serve as the template extracting unit 43.
As this feature quantity, for example, pixel value dispersion as shown in equation (1) can be used.

Here, ai is the pixel value, A is the average pixel value in the template candidates, and is summed over the entire template candidates. Although no specific expression is given, secondary pixel value dispersion and edge detection results are also preferable values as feature amounts. At this time, the value T is expressed by the following equation:

As described above, when the value is smaller than a predetermined value Tth, it is not adopted because the reliability is low. For example, the accuracy of matching, which will be described later, generally decreases at a portion where the change in the pixel value is small, such as an empty portion in FIG. 3B, but the method of the present invention has a feature having a value T as shown in equation (2). No feature points are set at the point candidate positions, and it is possible to prevent a decrease in accuracy of the positional relationship parameters.

FIG. 4 is a diagram illustrating the configuration of the corresponding point search unit 35.
The search area setting unit 51 sets the position and size of the area for searching for the corresponding points based on the feature point coordinates read from the feature point setting unit 33. In the correlation calculation unit 52, based on the information in the search area set by the search area setting unit 51, image data around the corresponding point candidate position so as to have the same size as the template input from the feature point setting unit 33. Is extracted from the image data in the frame memory 32b, and a correlation value with the template data input from the feature point setting unit 33 is calculated.

The correlation value determination unit 53 determines a position having the highest correlation with the template extracted by the template extraction unit 46 and outputs the result to the corresponding point position memory 36 as a corresponding point position.
At this time, generally, the corresponding point candidate moves for each pixel to obtain the correlation value. In order to detect the corresponding point position with an accuracy of less than the pixel, the corresponding point position is determined by interpolation with reference to the surrounding correlation value. You may decide. As the correlation value, the sum of absolute values of differences shown in the following equation (3), cross-correlation shown in equation (4), normalized cross-correlation shown in equation (5), and the like can be used.

Here, bi and B are the pixel values in the small area moving within the search area of the image B and their averages, σA and σB are the standard deviations of the template data, and σA 2 and σB 2 are their variance values. It becomes.
When calculating the equations (2) to (4), if only a representative color is calculated, the processing can be performed at high speed.

In addition, if the corresponding points are searched by adding the correlation values of the R, G, and B colors, the corresponding points can be searched with higher accuracy.
Then, the correction parameter calculation unit 37 determines the positional relationship between the images A and B from the feature point coordinate data and the corresponding point coordinate data. The parameters to be obtained are cos θ, sin θ, sx, sy, and M in equation (6).

Here, (x, y) represents a feature point position, (X, Y) represents a corresponding point position, and M corresponds to a relative magnification between images.
Since a plurality of data is stored in the feature point position memory 34 and the corresponding point position memory 36, when the parameter is obtained in the least squares, the equation (7) is obtained.

Here, N is the number of feature points. From equation (8), M, cos θ, and sin θ are determined as in equation (9).

Further, since cosθ and sinθ originally have a relationship of cos2θ + sin2θ = 1, when it can be assumed that there is no magnification change in the target image, M = 1.0 is obtained as a result of the equation (9), and (10) If the relationship is as in the equation, it is better to correct cos θ and sin θ as in equation (11). sx and sy are calculated from corrected cos θ and sin θ.

In the above-described equations (6) to (9), an arbitrary origin is set as the rotation center of the image. However, it may be considered that the centroids of the set / searched feature points and corresponding point positions coincide with each other. The centroid position of the feature point is (/ x, / y) (where / is used as a symbol meaning the average. In addition, as shown in the image information of each equation, this average is The barycentric position of the corresponding point is given by (/ X, / Y), and cos θ, sin θ, sx, sy are given by equations (12) and (13), respectively.

  As described above, the present embodiment extracts a plurality of feature points from an image and automatically aligns them. Therefore, even a general user can perform dynamic range expansion processing and processing without using expensive business equipment. The effect of the depth of field expansion process can be utilized to the maximum extent. However, when distortion is generated in the processing target image, it is necessary to correct it in advance by an arbitrary method.

Next, a second embodiment according to the present invention will be described.
This embodiment differs from the first embodiment described above in the internal configuration of the feature point setting unit 33.

FIG. 5A is a diagram illustrating a conceptual configuration of the feature point setting unit 33. This configuration is different from the configuration shown in FIG. 1 in that an area specifying unit 81 is added.
FIG. 5B is a diagram illustrating a configuration example of the area specifying unit 81. In the area designating unit 81, a plurality of target areas are instructed in the area instruction unit 83 for the input image data. The feature points are set in the area specified here by the method described above.

  The display control unit 84 overlays the designated area (data from the area position memory 85 described later) on the input image data as shown in FIGS. , Display the data. The display unit 82 may be a display attached to the PC, or a dedicated display may be provided. The area instruction unit 83 may be instructed by a mouse attached to the PC or may be provided separately. The display control unit 84 has been described as a part of the configuration of the area specifying unit 81, but may be configured to use an operating system (OS) function of a personal computer or the like.

  Further, the area instruction unit 83 may designate a region of interest with coordinates in the image. Then, the area position memory unit 85 stores the position / size of the area designated by the area instruction unit 83 and outputs it to the area dividing unit 41. Based on the region position information from the region position memory unit 85, the region dividing unit 41 sets feature points in the designated region by the method described above.

With the configuration of the present embodiment, it is possible to increase the alignment accuracy of the portion that the user wants to pay attention to, and it is possible to reduce the calculation time because the feature point search is not required in the portion that is not so necessary.
In addition, when a moving subject such as a person or a vehicle is included in the photographed image, an error occurs in the alignment of the entire image when the image of the moved part is used for the calculation. You can avoid using partial data.

  Furthermore, it is possible to further improve the alignment accuracy of the specific area by instructing a specific area among a plurality of areas instructed by the area instructing unit 83 and setting more feature points only in the inside than other areas. It is.

  In the case of the dynamic range expansion technique, images are arranged in the order of exposure values, and in the case of the depth of field expansion technique, the images are arranged in the order of the focal positions, and the above operation is performed between adjacent images, so that the images are randomly selected. The accuracy can be improved as compared with the case of selecting and executing.

FIG. 7 is a diagram showing a third embodiment of the present invention.
The configuration of the image correction unit 30 illustrated in FIG. 7 includes a display unit 82, a display control unit 84, and a feature point instruction unit 90 in addition to the configuration of the image correction unit 30 illustrated in FIG. A plurality of feature point positions are set by the method described above and stored in the feature point memory 34.

  The feature point coordinates stored in the feature point memory 34 are sent to the feature point instruction unit 90, and the position is overlaid on the target image on the display unit 82 through the display control unit 84 (for example, FIG. 3B). .

  At this time, if the feature point is set on a moving subject (moving object) such as a person or a vehicle, it causes an error of the correction coefficient calculated by the equations (6) to (10). Therefore, the feature point instruction unit 90 detects a moving object if it is included, excludes the feature point set on the moving object from the data used for calculation, and stores the data from the feature point memory 34. delete.

  In this operation, the user may determine the displayed data and indicate data to be excluded from the feature point instruction unit 90. Further, after the corresponding point search, the data of the feature point memory 34 and the corresponding point memory 36 are simultaneously displayed on the display unit 82, and when the user determines that the correspondence is not achieved, the feature point indicating unit 90 stores the data. It may be used for instructing exclusion.

Next, a fourth embodiment according to the present invention will be described with reference to FIG. In the fourth embodiment, the area designating unit 81 shown in FIG. 5 is changed to the configuration shown in FIG.
The area designation unit 81 shown in FIG. 8 includes an area instruction unit 83, an area position memory 85, and a state determination unit 101.
In the present embodiment, the range to be designated as the feature point setting area is determined based on the statistical and physical properties of the image.

First, the entire image is divided into a predetermined size as shown in FIG. The size to be divided at this time is larger than the template cut out by the feature point setting unit.
Each of the divided parts is sent to the state determination unit 101, and it is determined whether or not to specify the feature point setting area. The state determination unit 101 scans the input image data, and determines that it is not suitable as a feature point setting region if the saturated pixels exceed a certain ratio.

Further, the state determination unit 101 may determine that it is not suitable as a feature point setting area when the average value of input data exceeds a certain threshold.
As shown in FIG. 9B, the region instruction unit 83 sequentially outputs the image data of the divided regions to the state determination unit 101, and when it is determined that the region is appropriate as the feature point setting region, the range is displayed. It records in the area position memory 85 so as to include it.

In the above description, the determination is made for each divided region, but the entire range recorded in the region position memory 85 may be used as the image data output to the state determination unit. Further, as shown in FIG. 9C, the user may designate a certain divided area by the area instructing unit 83 and include it in the feature point designation area in order from the surrounding area.
Further, as shown in FIG. 10, the user may designate a certain point in the area instruction unit 83 and assume a rectangle expanded around the point, and the state determination unit 101 may determine the image data. .

In addition, although the saturated pixel value is used as a determination criterion, the state determination unit 101 uses the power of a high-frequency component when the image is subjected to orthogonal transform such as discrete Fourier transform or discrete cosine transform of the input image, or wavelet transform. You may judge by a component.
Further, the state determination unit 101 may use the contrast of input data or the difference between the maximum and minimum pixel values for determination.

Next, a fifth embodiment of the present invention will be described with reference to FIG.
This embodiment is an example in which three or more images are processed. Here, in the figure, the same components as those in FIG. 1 described above are denoted by the same reference numerals, and the description thereof is omitted.

  In the present embodiment, the image data reproduced by the image data reproducing unit 31 is input to one of the frame memories 32 a and 32 b by the image switching unit 111 that performs a switching operation according to an instruction from the image selection instruction unit 114. The image selection instruction unit 114 estimates the processing time of the apparatus so that a plurality of images taken of the same subject that have been input can be sequentially processed, and the images are input alternately to the two frame memories. The switching unit 111 is controlled.

  The feature points / corresponding points are detected from the images stored in the frame memories 32a and 32b by the above-described method, and the positional relationship (parallel movement amount / rotation angle / relative magnification) between the images is calculated by the correction parameter calculation unit 37. . The parameters calculated by the correction parameter calculation unit 37 are stored in the correction parameter storage unit 113 together with the image ID and the reference image ID.

  The correction parameter conversion unit 112 reads data stored in the correction parameter storage unit 113, and converts each data into a parallel movement amount / rotation angle viewed from an image serving as a reference for correction specified in advance. Subsequently, in the interpolation calculation unit 38, the correction parameter conversion unit 112 corrects the image according to the parameters viewed from the correction reference image.

Here, the data conversion in the correction parameter conversion unit 112 and the correction in the interpolation calculation unit 38 may be performed for each operation of the image switching unit 111. After calculating the relative positional relationship for all the target images, the data conversion is performed again. The image may be read into the frame memory 32.
In the present embodiment, two frame memories 32 are used and the input image is switched by the signal switching unit 111. However, a number of frame memories that can store all target images may be prepared.

Although the above embodiments have been described, the present invention includes the following inventions.
(1) In multiple images with different shooting conditions,
An image is divided into a plurality of divided areas, and feature point search areas that are the same as or smaller than each of the divided areas are provided in the divided areas, and the characteristic position of the subject is determined from each feature point search area based on image statistics. Information is extracted, and a corresponding point in the image different from the image from which the characteristic position information of the subject is extracted is searched from among the plurality of images, and the characteristic position information and the position information of the corresponding point are searched. An image processing apparatus that detects a correction parameter (rotation / movement / magnification change between images) and corrects an image based on rotation / movement / magnification change between images as the correction parameter.

The present invention corresponds to the first embodiment.
As a result, more accurate images can be superimposed when a plurality of images are superimposed.

(2) In the image processing apparatus according to (1),
When the image is corrected based on the rotation / movement / magnification change, the correction is performed after converting the parameters so as to represent the rotation / movement / magnification change seen from a specific reference image.

The present invention corresponds to the fifth embodiment.
As a result, when a plurality of images are overlaid, any appropriate correction parameter can be set for each image, and more accurate image superposition according to the photographer's intention can be performed. .

(3) For the plurality of divided regions,
2. The image processing apparatus according to 1, wherein a region to be divided is limited to a part of an image.
The present invention corresponds to the second embodiment.
As a result, the speed of image processing is improved when a plurality of images are superimposed, and an appropriate area can be set for each image, and more accurate images can be superimposed according to the intention of the photographer. Will be able to.

(4) In the image processing apparatus according to (1) or (3),
Of the characteristic position information of the subject based on the statistics of the image, the position information set on the subject whose position is moving between a plurality of images is used to detect rotation, movement, and magnification change between images. It is characterized by not.
The present invention corresponds to the third embodiment.
Thereby, when overlapping a plurality of images, the area where the moving object exists is excluded, so that more accurate image overlapping can be performed.

(5) In the image processing apparatus according to (3),
The restricted area that limits the area to be divided to a part of the image is one of the ratio of high frequency components, the ratio of saturated pixels, the average value of pixel values, the difference between maximum and minimum pixel values, and contrast It is limited on the basis of the value of.

The present invention corresponds to the fourth embodiment.
As a result, when a plurality of images are overlaid, the speed of image processing is improved, and an appropriate area can be automatically set for each image, which is more accurate according to the photographer's intention. Images can be superimposed.

(6) In the image processing device according to (5),
It is characterized by expanding the restricted area around a specific point.
The present invention corresponds to the fourth embodiment.
As a result, when a plurality of images are overlapped, an appropriate region is set for each image with a specific point as the center, so that more accurate image overlap can be performed.

It is a figure which shows the structural example of the image processing apparatus as 1st Embodiment. It is a figure which shows the structural example of the feature point setting part shown in FIG. FIG. 3A is a diagram showing a plurality of divided regions obtained by dividing an image and feature point search regions set in each divided region, and FIG. 3B is a diagram for explaining feature points. It is a figure which shows the structural example of the corresponding point search part shown in FIG. FIG. 5A is a diagram illustrating a configuration example of an image processing apparatus according to the second embodiment, and FIG. 5B is a diagram illustrating a configuration example of an area specifying unit. FIG. 6A is a diagram illustrating an example of the designated area displayed on the image, and FIG. 6B is a diagram illustrating the position of interest displayed on the image. It is a figure which shows the structural example of the image processing apparatus as 3rd Embodiment. It is a figure which shows the structural example of the area | region designation | designated part used for the image processing apparatus as 4th Embodiment. FIG. 9A is a diagram showing an input image divided into a plurality of divided regions, FIG. 9B is a diagram for explaining designation of a feature point setting region by scanning, and FIG. It is a figure for demonstrating designation | designated of the feature point setting area | region by expansion. It is a figure for demonstrating designating the rectangle extended from the designated point as a feature point setting area | region. It is a figure which shows the structural example of the image processing apparatus as 5th Embodiment. It is a figure which shows the structural example for producing the conventional wide dynamic range image. It is a figure which shows the structural example for producing the image which has the conventional deep depth of field.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 30 ... Image correction part, 31 ... Image data reproduction part, 32 ... Frame memory, 33 ... Feature point setting part, 34 ... Feature point position memory, 35 ... Corresponding point search part, 36 ... Corresponding point position memory, 37 ... Correction parameter Calculation unit, 38 ... interpolation calculation unit, 39 ... image composition unit.

Claims (3)

An image processing apparatus that performs an alignment process between a plurality of images obtained by shooting substantially the same composition under different shooting conditions,
Search area setting means for dividing at least one of the input images into a plurality of divided areas and providing a search area for feature points representing the features of the image in each of the divided areas;
Based on a comparison result between a calculated value calculated using a pixel value of a pixel included in the search area provided in the divided area and a specified value, the search area is actually selected from the plurality of search areas provided. Search area selection means for selecting a search area for searching for feature points;
Position information extraction means for extracting characteristic position information of a subject in the composition from the selected search area in association with coordinates for each divided area based on the feature amount of the image;
An image processing apparatus comprising:   The image according to claim 1, wherein the search area selection unit selects a search area corresponding to the calculated value that is equal to or greater than a predetermined value as a search area for actually searching for a feature point. Processing equipment. An image processing method for performing an alignment process between a plurality of images obtained by shooting substantially the same composition under different shooting conditions,
A search area setting step of dividing at least one of the input images into a plurality of divided areas and providing a search area for feature points representing the features of the image in each of the divided areas;
Based on a comparison result between a calculated value calculated using a pixel value of a pixel included in the search area provided in the divided area and a specified value, the search area is actually selected from the plurality of search areas provided. A search area selection step of selecting a search area for searching for feature points;
A position information extraction step of extracting characteristic position information of a subject in the composition from the selected search area in association with coordinates for each divided area based on the feature amount of the image;
An image processing method comprising:

Images