JP5212046B2 - Digital camera, image processing apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing technique for creating a high-resolution image.

“Super-resolution processing” that creates one high-resolution image from a plurality of images taken by a camera is known. In such image processing, it is necessary to estimate a pixel value in a high-resolution space at the time of synthesis, and a method for speeding up the estimation processing, a method for estimating an alias component, and the like have been studied (for example, see Patent Document 1). .
JP 2005-072935 A

  Estimating pixel values in a high-resolution space is a very important technique that leads to higher image quality if accurate estimation is possible. However, complicated estimation calculation processing must be performed to accurately estimate the pixel value, and an expensive calculation processing device is required to perform high-speed processing. In particular, there is a problem that a processing load is heavy and takes a lot of time to be processed by a CPU (arithmetic processing unit) such as a digital camera.

  An object of the present invention is to provide a digital camera, an image processing apparatus, and an image processing program that can reduce the processing load of pixel value estimation in a high resolution space.

An image processing apparatus according to the present invention includes: an image input unit that inputs a plurality of frames including the same subject continuously in time series; a reference image is selected from the plurality of frames; and the reference image is excluded A standard image selection unit that uses the images of the plurality of frames as reference images, a region division unit that divides the standard image into a plurality of divided regions, and an image structure that determines the image structure by extracting feature amounts for each of the divided regions A discriminating unit; a misregistration detecting unit for obtaining a misregistration amount of the reference image with respect to the standard image; and developing the standard image in a high resolution space, and converting the reference image into the high resolution space according to the misregistration amount. In a high resolution space created by the resolution space development unit to be arranged and the resolution space development unit, high resolution is performed using an interpolation method according to the image structure determined by the image structure determination unit for each of the divided regions. Possess an image synthesizing section for synthesizing the spatial image, the image synthesizing unit, at least one of the size and weight of the filter region is characterized by combining a high resolution spatial image by using a plurality of interpolation method different .

  More preferably, the image structure determining unit determines an image structure for each of the divided regions using at least one of edge strength and pixel value dispersion.

  More preferably, the image composition unit divides the filter area into a plurality of sub-areas to obtain a variance, and sets an average value in the sub-area having the lowest variance as a target pixel value in the filter area. It is characterized by.

  More preferably, an object extracting unit for extracting a subject portion is further provided, and the divided area is a subject portion extracted by the object extracting portion.

  The digital camera according to the present invention is provided with a lens optical system and an imaging unit in the image processing apparatus, and the image input unit is identically photographed continuously in time series by the imaging unit via the lens optical system. A plurality of frames of images including a subject are input.

The image processing program according to the present invention is a computer that performs processing for generating an image having a higher resolution than an image input by the image input unit by inputting a plurality of frames including the same subject in time series from the image input unit. A standard image selection procedure for selecting a reference image from the plurality of frames of images and using the plurality of frames of images excluding the reference image as reference images; and a plurality of the reference images An area dividing procedure for dividing the reference image with respect to the reference image, and an area determining procedure for determining an image structure by extracting a feature amount for each divided area; A resolution space expanding procedure for expanding the reference image in a high resolution space and disposing the reference image in the high resolution space according to the amount of displacement. In the high-resolution space time space deployment procedures created, have a an image combining procedure for combining a high resolution spatial image using the interpolation method corresponding to the image structures determined the area determination procedure for each of the divided areas, wherein The image synthesis procedure is characterized in that a high-resolution spatial image is synthesized by using a plurality of interpolation methods in which at least one of the filter area size and the weighting amount is different .

  Since the digital camera, the image processing apparatus, and the image processing program according to the present invention can reduce the processing load of pixel value estimation in a high resolution space, an expensive arithmetic processing apparatus is unnecessary and can be processed at high speed.

  Embodiments relating to a digital camera, an image processing apparatus, and an image processing program according to the present invention will be described below.

  FIG. 1 is a block diagram illustrating a configuration of a digital camera 101 according to the present embodiment. The digital camera 101 in FIG. 1 includes the functions of the image processing apparatus according to the present invention and an image processing program to be processed inside the digital camera 101. In addition, the digital camera 101 according to the present embodiment has a continuous shooting mode in which a plurality of still images are taken continuously in time. Furthermore, it has a super-resolution mode for converting to a high-resolution image using a plurality of still images during or after shooting.

  In FIG. 1, a digital camera 101 includes a photographing optical system 102, a mechanical shutter 103, an image sensor 104, an AFE (analog front end) 105, an A / D converter 106, an image buffer 107, and a controller 108. A memory 109, a display unit 110, a memory card I / F 111, and an operation unit 112.

  In FIG. 1, the subject light incident on the photographing optical system 102 is incident on the light receiving surface of the image sensor 104 via the mechanical shutter 103. Here, the photographing optical system 102 includes a plurality of lenses such as a zoom lens and a focus lens, and has a diaphragm 102a.

  A photoelectric conversion unit is two-dimensionally arranged on the light receiving surface of the image sensor 104, and is converted into an electrical signal according to the amount of light incident on each photoelectric conversion unit and output to the AFE 105.

  The AFE 105 performs noise removal and amplification of the electrical signal output from the image sensor 104 and outputs the result to the A / D converter 106.

  The A / D converter 106 converts the electrical signal output from the AFE 105 into a digital signal, and temporarily stores the digital signal for one screen in the image buffer 107 as photographed image data. The image buffer 107 is also used as an image buffer for performing image processing and a display image buffer for displaying an image on the display unit 110. A plurality of still images during continuous shooting are also temporarily stored in the image buffer. 107.

  The control unit 108 performs image processing such as white balance processing, color correction processing, gamma conversion processing, and JPEG compression processing on the image data temporarily stored in the image buffer 107. The JPEG-format image data after JPEG compression is stored in the memory card 111a via the memory card I / F 111. In the present embodiment, the control unit 108 also performs processing for converting a plurality of still images into high-resolution images. Detailed processing of the control unit 108 will be described in detail later.

  The memory 109 stores settings of the shooting mode and playback mode of the digital camera 101, parameters and setting values such as exposure information and focus information, and the like. The image data stored in the memory card 111a may be stored in the memory 109.

  The display unit 110 displays a captured image, a menu screen, and the like in accordance with an instruction from the control unit 108.

  The operation unit 112 includes operation buttons such as a power button, a release button, and a cursor key. The user operates the digital camera 101 by operating these operation buttons. The operation information of these operation buttons is output to the control unit 108, and the control unit 108 controls the entire operation of the digital camera 101 according to the operation information input from the operation unit 112.

  Next, the control unit 108 in FIG. 1 will be described in detail. The control unit 108 operates according to a program stored therein in advance, and controls each unit of the digital camera 101. The control unit 108 includes a photographing processing unit 201, a reference image selection unit 202, a region division unit 203, a feature amount extraction unit 204, an image structure determination unit 205, a misregistration detection unit 206, and a high resolution space development unit. 207 and an image composition unit 208. For ease of explanation in the present embodiment, the reference image selection unit 202, the region division unit 203, the feature amount extraction unit 204, the image structure determination unit 205, the misregistration detection unit 206, the high resolution space development unit 207, the image The combining unit 208 and the like are illustrated as being included in the control unit 108, but may be configured by a dedicated hardware circuit separately from the control unit 108.

  The imaging processing unit 201 performs focus control, exposure control, and the like of an image to be captured. For example, in the focus control, the focus position is obtained from the image captured in the image buffer 107 and the position of the focus lens of the photographing optical system 102 is moved. These processes are performed using an image captured in the image buffer 107 via the image sensor 104 when the preview image is captured. Focus control and exposure control are performed using a dedicated AF sensor or AE sensor. It doesn't matter. When the preview image is captured, the mechanical shutter 103 is always opened and the electronic shutter that controls the shutter speed according to the exposure time of the image sensor 104 is used. Here, the preview image is an image for the photographer to determine a shooting composition or the like by displaying on the display unit 110 a moving image that is continuously captured in time by the image sensor 104. Then, when the photographer presses the release button of the operation unit 112, the actually captured image data is temporarily stored in the image buffer 107. When the shooting mode is set to the continuous shooting mode or the high resolution mode, a plurality of still images are continuously taken into the image buffer 107 while the release button is pressed. Alternatively, a predetermined number of still images are taken into the image buffer 107 by pressing the release button once.

  Next, the operations of the reference image selection unit 202, the region division unit 203, the feature amount extraction unit 204, the image structure determination unit 205, the misregistration detection unit 206, the high resolution space development unit 207, and the image composition unit of the control unit 108 will be described. This will be described in detail with reference to the flowchart of FIG. The flowchart of FIG. 2 shows a procedure of high-resolution processing for synthesizing one high-resolution still image from a plurality of still images. Here, it is assumed that the digital camera 101 is set to the high resolution mode, and a plurality of still images have already been taken and temporarily stored in the image buffer 107. Alternatively, it is assumed that a plurality of still images stored in the memory card 111 a are reproduced and temporarily stored in the image buffer 107. In the present embodiment, it is assumed that the plurality of still images are images obtained by continuously shooting the same subject at substantially the same angle of view in time. Hereinafter, it demonstrates in order according to the flowchart of FIG.

  (Step S <b> 101) When the digital camera 101 is photographing, the control unit 108 continuously photographs a plurality of still images by pressing the release button, and takes them into the image buffer 107. Alternatively, when the digital camera 101 is playing back, the control unit 108 reads a plurality of shot still images stored in the memory card 111 a into the image buffer 107 in accordance with the playback operation of the operation unit 112. In the present embodiment, it is assumed that the plurality of still images are images obtained by continuously shooting the same subject at substantially the same angle of view in time.

  (Step S <b> 102) The reference image selection unit 202 of the control unit 108 uses one image among a plurality of images temporarily stored in the image buffer 107 as a reference image and the remaining images as reference images. . And the control part 108 performs the process from step S103 to S105, and the process from step S106 to S107 in parallel. Accordingly, whichever of the processing from step S103 to S105 and the processing from step S106 to S107 may be performed first.

  (Step S103) The area dividing unit 203 of the control unit 108 divides the reference image into a plurality of areas. Here, as the divided area is smaller, an appropriate feature amount can be extracted in units of area. However, since the processing load becomes larger, depending on the processing capability of the CPU of the processing apparatus (digital camera, personal computer, etc.) It is necessary to determine an appropriate size of the divided area. For example, in the case of the digital camera 101 according to the present embodiment, as shown in FIG. 3, an image 301 on one screen is divided into 12 divided areas (blocks) of 3 blocks in the vertical direction and 4 blocks in the horizontal direction.

  (Step S104) The feature amount extraction unit 204 of the control unit 108 extracts feature amounts in units of divided regions. For example, in the case of FIG. 3, the feature amount is first extracted in the divided region of the block X (1, 1), and then the feature amount is extracted in the divided region of the block X (1, 2). In this manner, the feature amount extraction unit 204 extracts feature amounts in each divided region from the block X (1, 1) to the block X (3, 4). Here, in the present embodiment, the feature quantity extracted by the feature quantity extraction unit 204 is, for example, an edge strength or a variance value. Note that the edge strength means the number and strength of edge components contained in the block. For example, an image whose luminance changes gently has a low edge strength, and a structure with a sharp change in luminance has an edge Strength increases.

  (Step S105) The image structure determination unit 205 of the control unit 108 determines whether or not the image structure of the divided area is flat. Here, when the image structure is flat, the image has few edge portions (images with smooth luminance changes and small variations in luminance), and when the image structure is not flat, the images have many edge portions (luminance changes drastically and luminance Is an image with a large variation. In the present embodiment, the image structure determination unit 205 determines the image structure of each divided region using the feature amount extracted by the feature amount extraction unit 204 in each divided region after division.

  (Step S106) The misregistration detection unit 206 of the control unit 108 performs alignment (calculation of misregistration amount) between the standard image and the reference image.

  Here, in the present embodiment, there is a premise that there is a slight misalignment between the standard image and the reference image. This is because, for example, the shooting positions slightly differ among a plurality of shot images due to the influence of camera shake during continuous shooting, and the pixel positions of the plurality of reference images and the reference image do not completely match. . Therefore, in the digital camera 101 according to the present embodiment, the misregistration detection unit 206 obtains the misregistration amount with respect to each reference image based on the standard image.

  Next, how to determine the amount of displacement will be described. For detecting the amount of normal displacement, for example, the pattern of the subject is detected by edge extraction image processing for the base image and each reference image, and the subject of a certain pattern on the base image moves to which position on the reference image Therefore, the amount of positional deviation is determined by the resolution of the standard image and the reference image, and only the amount of deviation in pixel units can be detected. For example, it is only detected as a displacement amount of 2 pixels in the right direction and 1 pixel in the downward direction. In contrast, the misregistration detection unit 206 of the present embodiment obtains a misregistration amount in a high-resolution space dimension that is finer than the resolution of the standard image or the reference image. For example, a positional deviation amount (subpixel level positional deviation amount) that is smaller than the resolution of the standard image or reference image, such as 2.2 pixels in the right direction and 1.4 pixels in the downward direction, is obtained.

  Here, as a method for obtaining the positional deviation amount at the sub-pixel level, for example, a high-dimensional parameter estimation method using a geometric transformation method such as affine transformation can be used. In this case, for example, a parameter such as a rotational position of 0.2 degrees can be obtained, and as a result, a sub-pixel level displacement amount is obtained. Note that the sub-pixel level displacement amount of each reference image with respect to the base image may be obtained by a method other than the above.

  In this way, in step S106, the misregistration detection unit 206 calculates the misregistration amount between the standard image and the reference image.

  (Step S107) The high-resolution space development unit 207 of the control unit 108 develops a high-resolution space using one standard image, a plurality of reference images, and the amount of displacement. The high resolution space development unit 207 first develops the reference image in the high resolution space. An example of the development process of the reference image is shown in FIG. FIG. 4 shows how a reference image 401 having a resolution of 16 pixels of 4 pixels in the vertical direction and 4 pixels in the horizontal direction is developed into a high-resolution spatial image 402 having a resolution of 256 pixels of 16 pixels in the vertical direction and 16 pixels in the horizontal direction. As shown in FIG. 4, each of the 16 pixels A1 to A16 of the reference image 401 is applied to the image 402 in the high resolution space so as to jump four pixels vertically and horizontally. In this state, of the 256 pixels of the high-resolution space image 402, the pixels in the portion where the 16 pixel values of the reference image 401 are not applied are empty pixels that do not have pixel values.

  Next, the high resolution space development unit 207 performs processing for applying a plurality of reference images to empty pixels of the high resolution space image 402 according to the amount of positional deviation obtained previously. This is shown in FIG. In FIG. 5, each pixel of the three reference images 403 to 405 is applied to an empty pixel to which a pixel of the standard image is not applied according to the positional deviation amount, thereby creating a high-resolution space image 406.

  Here, when the pixels of the reference images 403 to 405 are arranged at the same pixel positions as the positions of the pixels A1 to A16 when the standard image 401 is expanded in the high resolution space, the pixel value of the standard image is given priority. The pixel value of the pixel is used. Alternatively, a value obtained by averaging the pixel values at the same pixel position of the reference image and the reference image may be used as the pixel value of the pixel, and the pixel value of the reference image is weighted higher than the pixel value of the reference image. It does not matter as a pixel value.

  Similarly, even when pixel values of a plurality of reference images are arranged at the same pixel position when a plurality of reference images are expanded in a high resolution space, a value obtained by averaging the pixel values of the same pixels of the plurality of reference images May be used as the pixel value of the pixel, or the pixel value of any reference image may be represented as the pixel value of the pixel. Alternatively, a plurality of reference images may be used for subsequent processing by handling a plurality of images as three-dimensional data.

  Through the processing from step S101 to step S107, a high-resolution space image 406 is generated in which a plurality of reference images are aligned and arranged at the sub-pixel level based on the basic image developed in the high-resolution space. This is a temporary high-resolution space image. As shown in FIG. 5, the provisional high-resolution space image 406 is obtained by simply applying a plurality of reference images to the basic image developed in the high-resolution space according to the amount of displacement. In 406, there is an empty pixel to which the reference image is not applied. Therefore, in the present embodiment, the final high-resolution spatial image is synthesized by performing an interpolation process for interpolating the empty pixels in the next processing step.

  (Step S108) The image composition unit 208 of the control unit 108 performs an interpolation process (pixels) for obtaining a pixel value of the target pixel using an interpolation filter for each filter region having a predetermined size centered on the target pixel (interpolated pixel). Value estimation process).

  For example, as shown in FIG. 6, when a filter region 501 having a size of 5 pixels × 5 pixels is used, a pixel value is obtained by interpolation processing from 25 pixels from pixel p1 to pixel p25 with the central pixel p13 as a target pixel. Here, the filter region 501 of the interpolation filter corresponds to, for example, a 5 pixel × 5 pixel portion surrounded by a thick line frame of the temporary high-resolution space image 406, and is obtained by the filter region surrounded by the thick line frame in FIG. Is the pixel value of the pixel 551 which is the target pixel of this filter area. Similarly, when the pixel value of the pixel 552 on one pixel of the pixel 551 is obtained, the entire filter region 501 having a size of 5 pixels × 5 pixels is also moved up by one pixel, and the pixel 552 is set as the target pixel in the filter region. To do. Similarly, when the pixel value of the pixel 553 is obtained, the interpolation process is performed by moving the entire filter region 501 having a size of 5 pixels × 5 pixels so that the pixel 553 becomes the target pixel of the filter region. Note that the size of the filter area 501 may not be 5 pixels × 5 pixels.

  In this way, the pixel values of the empty pixels of the temporary high resolution space image 406 are obtained in order. Note that only the pixel value of the empty pixel may be obtained, or a new pixel value may be obtained by performing the same interpolation process for the pixel to which the pixel value of the base image or the reference image has already been applied. .

  Next, the interpolation processing method will be specifically described. In the present embodiment, different interpolation processing is performed according to the image structure determined for each divided region by the image structure determination unit 205 in step S105. That is, the interpolation process is performed separately for the case where the image structure of the divided region including the target pixel on which the interpolation process is performed is flat and the case where the image structure is not flat.

  First, an interpolation process when the image structure is flat will be described. Since the edge strength is small when the image structure is flat, the pixel value of the pixel of interest p13 is obtained by simple processing that is fast. For example, in the case of the filter region 501 as shown in FIG. 6, an average value of all the pixel values (25 pixels) in the filter region 501 is simply obtained and set as the pixel value of the target pixel p13.

  Alternatively, pixels close to the target pixel p13 (p7, p8, p9, p12, p14, p17, p18, p19) and distant pixels (p1, p2, p3, p4, p5, p6, p10, p11, p15, p16, p20) , P21, p22, p23, p24, p25), the pixel value of the pixel of interest p13 may be obtained by weighting the close pixels to be heavy and the distant pixels to be light. For example, by weighting the Gaussian distribution around the pixel of interest p13, it is possible to weight the pixels close to the pixel of interest heavy and lightly weight the pixels far away.

  Next, an interpolation process when the image structure is not flat will be described. If the image structure is not flat, the pixel value of the target pixel is obtained by a smoothing method that does not impair edge information. In the smoothing method in which the edge information is not impaired, for example, as shown in FIG. 7, the filter area 501 is divided into a plurality of sub areas having different positions, and the luminance distribution is obtained for each sub area. Then, an average value of pixel values in the sub-region having the lowest luminance dispersion is obtained, and this is used as the pixel value of the target pixel p13 in the filter region 501.

  Here, the sub-region of FIG. 7 will be described in detail. FIG. 7 is a diagram illustrating an example of sub-region division when the filter region 501 having the same size of 5 pixels × 5 pixels as that in FIG. 6 is used. In FIG. 7, the filter region 501 is divided into nine types of sub-regions 502 to 510 (shaded portions). The sub-regions 502 to 509 are sub-regions composed of seven pixels within the filter region 501 and the target pixel p13. The sub-region 510 is a sub-region that is rotated 45 degrees around the center, and the sub-region 510 is a sub-region composed of nine pixels of the target pixel p13 and its surrounding pixels (p7, p8, p9, p12, p14, p17, p18, p19) It is an area.

  As described above, the image composition unit 208 divides the filter area 501 into a plurality of sub areas having different positions, and obtains a luminance distribution for each sub area. Here, for example, it is assumed that the variance obtained by the image composition unit 208 for each sub-region is the following value. The variance of the sub-region 502 is 0.1, the variance of the sub-region 503 is 0.5, the variance of the sub-region 504 is 0.7, the variance of the sub-region 505 is 0.6, the variance of the sub-region 506 is 0.4, The dispersion of the sub-region 507 is 0.5, the dispersion of the sub-region 508 is 0.3, the dispersion of the sub-region 509 is 0.2, and the dispersion of the sub-region 510 is 0.3. In this case, since the sub-region having the lowest variance is the sub-region 502, the average of the pixel values of the seven pixels in the sub-region 502 is set as the pixel value of the target pixel p13.

  Here, the reason why the filter area 501 is divided into a plurality of sub areas and the sub area having the lowest variance is selected will be described. A sub-region with low variance can be regarded as a region where pixels with similar luminance are gathered. In this sub-region, the image structure is flat and changes little, and even if averaged, the edge component is lost. It is considered difficult. Conversely, a region with high dispersion can be regarded as a region where the image structure is not flat and the luminance change is large, and if the sub-regions are averaged, the edge component is lost and blurred, resulting in poor image quality. It is done. Therefore, in the present embodiment, the image composition unit 208 calculates the variance by dividing the filter region into a plurality of sub-regions, selects the sub-region with the lowest variance, and averages the pixel values of the pixels in the sub-region. Thus, the pixel value of the target pixel p13 is obtained.

  As described with reference to FIG. 6, when an empty pixel is included in the sub-region in the filter region 501, the pixel is ignored and the variance is obtained using the pixel value of the non-empty pixel.

  In this way, when interpolation processing is performed in a divided region where the image structure is not flat, the filter region is divided into a plurality of sub regions, and averaging processing is performed in sub regions where the luminance is as flat as possible. Since the edge component is not impaired as compared with the average of the pixel values of the pixel as the pixel value of the target pixel, it is possible to perform an interpolation process that leaves the edge component.

  (Step S109) The control unit 108 stores the high-resolution space image subjected to the interpolation processing in step S108 in the memory card 111a via the memory card I / F 111. Each image data being processed in the flowchart of FIG. 2 is processed in the image buffer 107, and the finally synthesized high-resolution spatial image is also temporarily stored in the image buffer 107.

  As described above, the digital camera 101 according to the present embodiment discriminates the image structure for each divided region, and the divided region where the image structure is flat determines the average value of all the pixels in the filter region as the pixel value of the target pixel. In the divided area where the image structure is not flat, the filter area is divided into a plurality of sub areas and the variance for each sub area is obtained, and the pixel value of the sub area having the lowest variance is used. Therefore, it is possible to synthesize a high-resolution spatial image that suppresses the processing amount and does not impair edge information in a portion where the image structure is not flat.

  In the present embodiment, the digital camera 101 performs super-resolution processing for generating a single high-resolution image by combining a plurality of low-resolution images that have been captured or a plurality of low-resolution images that have already been captured. However, a dedicated image processing apparatus that performs super-resolution processing for generating one high-resolution image from a plurality of low-resolution images may be used. In this case, the image processing apparatus includes a control unit that performs processing similar to the flowchart in FIG. 2 and a dedicated hardware circuit.

  Alternatively, the program for performing the processing of the flowchart of FIG. 2 may be run on a personal computer to perform super-resolution processing. In this case, a plurality of low-resolution images are input from a storage medium such as a memory card or CD to a personal computer, and a process according to the flowchart of FIG. 2 is executed to generate a high-resolution image. It is stored in a storage medium, or displayed on a high-definition display or printed with a high-definition printer.

  Further, in the present embodiment, for the sake of easy understanding, the case where the edge strength and variance are obtained using luminance information has been described. However, in the case of a color image, the intensity of color change and each color using the color information of each RGB color. May be obtained. Further, the size and number of the divided areas and the size and weighting amount of the filter area may be appropriately changed on the setting menu screen of the digital camera. In particular, the size of the divided area and the size of the filter area may be the same or different. In any case, the smaller the size of the divided area and the smaller the filter area, the larger the processing amount and the processing load, but the image quality improves.

1 is a block diagram of a digital camera 101 according to the present embodiment. It is a flowchart which shows the process which produces | generates a high resolution image. It is explanatory drawing which shows the example of area division. It is explanatory drawing which shows expansion | deployment to the high resolution space of a basic image. It is explanatory drawing which shows expansion | deployment to the high resolution space of a reference image. It is explanatory drawing which shows the example of a filter area | region. It is explanatory drawing which shows the example of a sub area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Digital camera 102 ... Shooting optical system 103 ... Mechanical shutter 104 ... Imaging element 105 ... AFE 106 ... A / D conversion part 107 ... Image buffer 108 ... Control Unit 109 ... Memory 110 ... Display unit 111 ... Memory card I / F 112 ... Operation unit 201 ... Shooting processing unit 202 ... Reference image selection unit 203 ... Area division unit 204 ... Feature amount extraction unit 205 ... Image structure determination unit 206 ... Position displacement detection unit 207 ... High resolution space development unit 208 ... Image composition unit

Claims (6)

An image input unit for inputting images of a plurality of frames including the same subject in chronological order;
A standard image selection unit that selects a standard image from the plurality of frames of images and uses the plurality of frames of images excluding the standard image as a reference image;
An area dividing unit for dividing the reference image into a plurality of divided areas;
An image structure discriminating unit for discriminating an image structure by extracting a feature amount for each of the divided regions;
A misregistration detection unit for obtaining a misregistration amount of the reference image with respect to the reference image;
A resolution space expanding unit that expands the reference image in a high resolution space and arranges the reference image in the high resolution space in accordance with the amount of displacement;
In the high-resolution space where the resolution spatial expansion unit creates, it possesses an image synthesizing section for synthesizing a high-resolution spatial image using an interpolation method by the image structure determination unit for each of the divided areas corresponding to the image structures determined ,
The image processing unit is characterized in that the high-resolution spatial image is synthesized by using a plurality of interpolation methods in which at least one of a filter area size and a weighting amount is different . The image processing apparatus according to claim 1.
The image processing apparatus is characterized in that the image structure determination unit determines an image structure for each of the divided regions using at least one of edge intensity and pixel value dispersion. The image processing apparatus according to claim 1 or 2 ,
The image synthesis unit divides the filter area into a plurality of sub-areas to obtain a variance, and sets an average value in the sub-area having the lowest variance as a target pixel value in the filter area. apparatus. The image processing apparatus according to any one of claims 1 to 3 ,
An object extraction unit for extracting the subject portion is further provided.
An image processing apparatus characterized in that the divided area is a subject portion extracted by the object extraction unit. A lens optical system and an imaging unit are provided in the image processing apparatus according to any one of claims 1 to 4 ,
The digital camera according to claim 1, wherein the image input unit inputs images of a plurality of frames including the same subject photographed continuously in time series by the imaging unit via the lens optical system. An image processing program for executing, on a computer, a process of generating a higher resolution image than an image input by the image input unit by inputting a plurality of frames of images including the same subject in time series from the image input unit. ,
A standard image selection procedure in which a standard image is selected from the multiple frame images, and the multiple frame images excluding the standard image are used as reference images;
A region dividing procedure for dividing the reference image into a plurality of divided regions;
An area determination procedure for extracting an amount of features for each of the divided areas and determining an image structure;
A displacement detection procedure for detecting a displacement amount of the reference image with respect to the reference image;
A resolution space expanding procedure for expanding the reference image in a high resolution space and disposing the reference image in the high resolution space according to the amount of displacement;
In the high-resolution space in which the resolution spatial deployment procedures created, have a an image combining procedure for combining a high resolution spatial image using the interpolation method corresponding to the image structure in which the area determination procedure is determined for each of the divided areas,
The image synthesizing procedure synthesizes a high-resolution spatial image by using a plurality of interpolation methods in which at least one of a filter area size and a weighting amount is different .

Images