JP2022170436A - Imaging apparatus and method for controlling the same, and program - Google Patents

Abstract

To accurately detect motion vectors for shake correction.SOLUTION: An imaging apparatus has: an image pickup device; an image memory that stores an image obtained through imaging performed by the image pickup device; a cut-out unit that cuts out a partial area of the image stored in the image memory as an output image; a detection unit that detects motion vectors by using a plurality of images obtained through the imaging performed by the image pickup device; and a processing unit that performs predetermined resolution improvement processing on the images obtained through the imaging performed by the image pickup device. The detection unit detects motion vectors for the partial area cut out by the cut-out unit by using high resolution images obtained through the predetermined resolution improvement processing.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus that requires high-magnification photographing, and relates to an algorithm for improving photographing magnification and increasing resolution by enlarging and cutting out a portion of a sensor area.

Cameras equipped with an imaging sensor for monitoring purposes may require image information for several kilometers ahead, and thus require an angle of view on the telephoto side rather than the imaging area. In general, by enlarging/cutting out a part of the imaging area and photographing, it is possible to obtain an angle of view on the telephoto side of the entire imaging area. Therefore, the angle of view on the telephoto side can be obtained without using an optically high-magnification, expensive lens. On the other hand, since a part of the imaging area is enlarged and cut out, the resolution is lowered. Naturally, as the resolution decreases, the resolution of information that can be detected from the captured image also decreases, making it impossible to obtain correct information. For this reason, when detecting information from a captured image, there is a method of detecting using an image in which pixels are not thinned.

For example, Patent Document 1 discloses a method of reducing a detection range according to the size of an object and performing vector detection only on images without pixel thinning.

JP 2019-125933 A

However, Patent Document 1 has the following problems. In Patent Document 1, the detection range is reduced according to the size of the subject, and the motion vector detection required for camera shake correction control is performed only for images without pixel thinning. When a motion vector is detected using a partially enlarged/cropped image, only a few reference blocks are assigned to detect the motion vector, so the reliability of the motion vector with respect to the angle of view of the image pickup decreases. Also, if the template frame is reduced in order to detect a motion vector from an image that is partially enlarged or cropped from the imaging area, the vector frame becomes smaller than the pixels used for imaging, resulting in an increase in error vectors. Sometimes.

The present invention has been devised in view of the above problems, and when detecting a motion vector from an image obtained by enlarging or cutting out a part of the imaging area, highly accurate motion vector detection can be performed even when the number of pixels used for imaging is small. It is intended to provide the technology to

In order to solve this problem, for example, the imaging device of the present invention has the following configuration. i.e.
an imaging device;
an image memory for storing an image obtained by imaging with the imaging element;
clipping means for clipping a partial area of the image stored in the image memory as an output image;
a detection means for detecting a motion vector using a plurality of images obtained by imaging with the imaging device;
and processing means for performing a predetermined resolution enhancement process on an image obtained by imaging with the imaging device,
The detecting means detects a motion vector of the partial area cut out by the cutting out means using the high-resolution image that has undergone the predetermined high-resolution processing.

According to the present invention, a motion vector for blur correction can be detected with high accuracy.

1 is a block diagram showing a configuration example of an imaging device according to a first embodiment; FIG. The figure which shows the coordinate system which concerns on 1st and 2nd embodiment. 4 is a flowchart for explaining an image conversion operation according to the first embodiment; FIG. 4 is a view before execution of resolution enhancement processing according to the first embodiment; FIG. 4 is a view after execution of resolution enhancement processing according to the first embodiment; 4 is a flowchart for explaining an electronic correction amount conversion operation according to the first embodiment; FIG. 2 is a block diagram showing a configuration example of an imaging device according to a second embodiment; 9 is a flowchart for explaining image conversion operation according to the second embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. It should be noted that the following embodiments do not limit the invention according to the scope of claims. Although multiple features are described in the embodiments, not all of these multiple features are essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant description is omitted.

[First Embodiment]
First, matters common to each embodiment will be described. FIG. 1 is an image processing apparatus schematically showing a configuration example of a lens-interchangeable or lens-integrated digital camera and an imaging apparatus for capturing still images and moving images, as an example of an image processing apparatus according to an embodiment. 1 is a block diagram of an apparatus; FIG. The embodiments can also be applied to electronic devices (for example, smartphones) having a configuration related to imaging, and therefore are not limited to imaging devices such as digital cameras, and can be applied to various image processing devices.

In the description of the present embodiment, the vibration applied to the imaging device is referred to as "shake", and the effect on the captured image caused by the vibration applied to the imaging device is referred to as "image blur". Further, as shown in FIG. 2, in this embodiment, the horizontal rotation axis is the Yaw axis and the vertical rotation axis is the Pitch axis so as to form mutually orthogonal detection axes on a plane orthogonal to the optical axis. , the rotation axis in the optical axis direction will be described as a Roll axis.

The image processing apparatus 100 of FIG. 1 is composed of an interchangeable lens and a camera body or a lens-integrated camera, and the interchangeable lens is attached to the camera body for use. Reference numeral 150 in the drawing is a control unit that controls the entire apparatus. The control unit 150 is composed of a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM that is used as a work area.

The imaging lens 101 includes a zoom lens 102 for changing magnification, a correction optical system 103 (first image blur correction means) such as a shift lens for image blur correction, and a focus lens 104 for focus adjustment. An image of the subject is formed on the imaging device 105 by performing operations such as image blur correction.

The imaging element 105 is composed of, for example, an XY addressing CMOS (Complementary Metal Oxide Semiconductor) image sensor or the like. An image signal composed of pixels is supplied to the signal processing unit 106 .

In addition, the image sensor 105 is driven by a motor 129, which will be described later, to correct horizontal and vertical directions perpendicular to each other on a plane orthogonal to the optical axis, as well as image blur due to rotation in the optical axis direction. The above image signal is supplied to the signal processing unit 106 (second image blur correction means).

The signal processing unit 106 performs signal processing such as white balance adjustment and gamma correction on the image signal output from the image sensor 105 and stores the frame image generated as a result in the image memory 107 .

The image enlargement processing unit 108 enlarges the frame image stored in the image memory 107 at an arbitrary magnification, and supplies it to the image clipping control unit 109 and the image conversion unit 117 . Examples of the enlargement processing used by the image enlargement processing unit 108 include the bicubic method and the bilinear method, and the enlargement magnification may be controlled from a user interface, or may be controlled according to a zoom key or the like.

The image cutout control unit 109 cuts out a predetermined area from the frame image provided by the image enlargement processing unit 108 to generate a new frame image, and supplies it to the display control unit 110 and the recording control unit 112 as an output image. At this time, the image clipping control unit 109 moves the clipping position of the predetermined area according to the shake of the apparatus. As a result, the image clipping control unit 109 functions as electronic image blur correction means (third image blur correction means) capable of correcting movement of the subject position between frames (image blur) caused by shake of the apparatus. do. A series of operations performed by the signal processing unit 106, the image enlargement processing unit 108, and the image clipping control unit 109 are executed at a cycle of 60 Hz in the case of a video signal conforming to the NTSC format, for example, and moving image data is generated. be.

The display control unit 110 outputs and displays an image (camera through image) based on the video signal supplied from the image clipping control unit 109, a setting menu image, a recorded image, and other video signals processed according to the application. An image is displayed on the device 111 . A display control unit 110 drives a display device 111, and the display device 111 displays an image using a liquid crystal display element (LCD) or the like.

The recording control unit 112 controls the recording medium 113 and supplies data from the image memory 107 and the image clipping control unit 109 when an operation unit (not shown) used for instructing the start or end of recording instructs recording of a video signal. It records and reads out the video signal, moving image data, still image data, metadata, and the like. The recording medium 113 is an information recording medium such as a semiconductor memory or a magnetic recording medium such as a hard disk.

The optical parameter acquisition unit 114 acquires, from the imaging lens 101, characteristic information of the attached lens, such as the focal length of the imaging lens, the aperture value, the focus position, the amount of shift lens movement, the amount of distortion, and the effective image circle. These pieces of information may be obtained not directly from the imaging lens 101 but from values input via a user interface or the like. The acquired information is supplied to the image blur correction amount calculation unit 119 and used for image blur correction control.

The angular velocity sensor 115 is a sensor for detecting shake applied to the image processing apparatus 100 . The detected shake signal is supplied to an image blur correction amount calculation unit 119, which will be described later, and used to control image blur correction. The angular velocity sensor 115 has a horizontal rotation axis Yaw axis, a vertical rotation axis Pitch axis, and a rotation axis Roll axis in the optical axis direction so as to form mutually orthogonal detection axes on a plane orthogonal to the optical axis. Three angular velocity sensors are arranged so as to detect shake in three axial directions. The angular velocity sensor 115 detects the angular velocity of shake applied to the image processing apparatus 100 and outputs a voltage corresponding to the angular velocity.

The A/D converter (1) 116 converts the voltage output from the angular velocity sensor 115 into digital data, fetches it as angular velocity data, and supplies it to the image blur correction amount calculator 119, which will be described later.

The image conversion unit 117 performs predetermined resolution enhancement processing to convert the image provided from the image enlargement processing unit 108 from a low-resolution image due to electronic enlargement to a high-resolution image. Provide images. Details of this will be described later.

The motion vector detection unit 118 detects motion vectors in two directions, the horizontal direction and the vertical direction, which are orthogonal to each other on a plane orthogonal to the optical axis. Specifically, motion vector detection methods include a correlation method, a block matching method, and the like. Here, as an example, it is assumed that the motion vector detection unit 118 adopts the block matching method. This block matching method uses a plurality of images. First, an input image signal is divided into a plurality of blocks of appropriate size (for example, 16×16 pixels), and each block unit is a fixed block of the previous field or frame. Compute the difference between pixels in the range. Then, the block of the previous field or frame in which the sum of the absolute values of the differences is the smallest is searched, and the relative displacement of the block is detected as the motion vector of that block. As a result, vertical and horizontal movement amounts (that is, motion vectors) are obtained in units of pixels. This motion vector indicates the amount of movement per unit time of successive captured images, that is, the amount of movement of the image blur correction device. Further, the motion vector detection unit 118 performs motion vector error determination when the motion vector cannot be successfully detected. As an example of the motion vector error determination method, conditions such as the luminance signal being small and the detected value being a peak value are conceivable. The detected motion vector is supplied as a horizontal vector and a vertical vector to an image blur correction amount calculator 119, which will be described later, and used for image blur correction control.

The image blur correction amount calculation unit 119 calculates a correction amount for correcting image blur caused by shaking applied to the image processing apparatus 100 and supplies it to the correction amount division control unit 120 . Note that the image blur correction amount calculated by the image blur correction amount calculation unit 119 does not output the correction amount of each of the plurality of image blur correction means, but calculates the overall image blur correction amount of the image processing apparatus 100. It is something to do.

A correction amount division control unit 120 divides the image blur correction amount of the entire apparatus calculated by the image blur correction amount calculation unit 119 into correction amounts to be corrected by a plurality of image blur correction units. As an example, the correction amount division control unit 120 of the present embodiment uses the image blur correction amount supplied from the image blur correction amount calculation unit 119 as a correction amount for correcting the image blur correction amount by the image sensor 105 and and a correction amount for correction.

The image sensor correction amount conversion unit 121 converts the correction amount supplied from the correction amount division control unit 120 into a movement amount for appropriately correcting image blur in the image sensor 105, and outputs it as a drive target position.

The electronic correction amount conversion unit 122 converts the correction amount supplied from the correction amount division control unit 120 into a clipping position for appropriately correcting the image blur by the image clipping control unit 109 , and sends the image clipping control unit 109 to the clipping position. set.

The position detection unit 123 detects the position of the imaging device 105 and outputs a voltage corresponding to the position, which is amplified by the amplifier 124 into a signal within an appropriate voltage range. Then, it is digitized by the A/D converter (2) 125 and taken in as position data.

The control filter 126 inputs deviation data that is the difference between the drive target position of the imaging device 105 and the position data, performs various signal processing such as amplification and phase compensation, and provides the result to the pulse width modulation section 127 .

The pulse width modulation section 127 modulates the output from the control filter 126 into a waveform (that is, a PWM waveform) that changes the duty ratio of the pulse wave, and supplies the modulated waveform to the motor driving section 128 .

The motor 129 is, for example, a voice coil type motor, and is driven by the motor driving section 128 to move the imaging device 105 in a direction perpendicular to the optical axis. A feedback loop is formed in which the position of the image sensor 105 that has moved is detected by the position detection unit 123 and the next deviation data is calculated. be. Thereby, the imaging element 105 can be driven (vibrated or moved) so as to follow the drive target position.

Next, the image conversion unit 117 in FIG. 1 will be described with reference to the flowchart in FIG. It should be understood that the processing shown in FIG. 3 is repeatedly executed at a predetermined cycle such as 60 Hz in the case of a video signal conforming to the NTSC format, for example.

First, in S<b>201 , the image conversion unit 117 acquires information such as the electronic enlargement magnification from the image enlargement processing unit 108 . Next, in S202, the image conversion unit 117 acquires image information before and after the image enlargement processing performed by the image enlargement processing unit 108, which is necessary for the resolution enhancement processing. Brightness information is included as an example of image information. Next, in S203, the image conversion unit 117 determines whether or not the electronic enlargement magnification is greater than or equal to a certain threshold. The image conversion unit 117 advances the process to S<b>204 if it determines that it is equal to or greater than the threshold value, and advances the process to S<b>205 if it determines that it is within the threshold value.

In S204, the image conversion unit 117 performs resolution enhancement processing using the image information required for resolution enhancement processing acquired in S202. In this embodiment, as an example, the information acquired in S202 is used to perform resolution enhancement and filtering processing to generate a high-resolution image. Specifically, the image conversion unit 117 uses the bicubic method or the bilinear method to increase the number of pixels, thereby increasing the number of pixels per unit area. Then, the image conversion unit 117 executes a brightness enhancement filtering process to generate a brightness-enhanced image.

In S205, the image conversion unit 117 determines whether or not motion vector detection can be performed before performing the high-resolution imaging process in S204. As an example, it is determined whether or not the pixel values (values indicating luminance) of the image after the luminance enhancement process are more emphasized than before the resolution enhancement process. If it is determined to be no, then the resolution enhancement process S204 is performed again after changing the parameters related to the resolution enhancement process (values for selecting a luminance enhancement filter that performs a higher enhancement process). If the above conditions are more effective than before the resolution enhancement process is performed, the image subjected to the resolution enhancement process is provided to the motion vector detection unit 118 . Also, when performing the resolution enhancement process, the resolution enhancement rate may be changed according to the focal length, the image size used for motion vector detection, and the like.

Here, the effect of enlarging a partial area of the imaging area and detecting a motion vector in the enlarged area, which is a subject of the present embodiment, will be described. First, as described above, motion vectors must be detected from within the enlarged area. When the magnification is high, if the motion vector is detected from the entire imaging area, the motion vector outside the output angle of view is more likely to be detected, and the original motion vector cannot be detected correctly. However, even if a motion vector is detected directly from the enlarged image, the resolution of information that can be detected from the image is lowered due to the reduction in resolution, and correct information cannot be obtained. FIG. 4 shows an image before the resolution enhancement process, and FIG. 5 shows an example of an image subjected to the resolution enhancement process using luminance values as an example. 4(a), 4(b), and 4(c) show an image of the entire imaging area, an enlarged image of a partial area of the imaging area, and an enlarged image of a partial area of the imaging area after moving by one frame. indicates 5(a) and 5(b) are enlarged images of a partial area of the imaging area subjected to high resolution processing, and an enlarged partial area of the imaging area subjected to high resolution processing after moving by one frame. shows an image with When attempting to detect a motion vector for the leftmost pixel using FIGS. is small and the motion vector cannot be detected correctly, but as shown in FIGS. 5A and 5B, a difference is generated in the luminance value of each pixel with respect to the leftmost pixel in FIGS. 5A and 5B, and a motion vector can be detected. Note that the reason why the squares (block size) in FIG. 5 are smaller than in FIG. 4 is that the number of pixels has increased due to the increase in resolution, and that the block size (the number of pixels) itself has not decreased. . Also, the detected motion vector is detected from an image to which enlargement processing by the image enlargement processing unit and resolution enhancement by the image conversion unit 117 have been applied. Therefore, the motion vector detection unit 118 determines the final motion vector by dividing the horizontal and vertical components of the detected motion vector by the "magnification rate×high resolution rate".

In the above description, an example of applying brightness enhancement filtering as filtering processing for increasing the resolution has been described. Also good. When the edge information is received, the image conversion unit 117, in S205, converts the pixel value (value indicating edge degree) of the image after the edge enhancement processing obtained by applying the edge enhancement filtering processing to the resolution enhancement processing. It is determined whether or not the edge is emphasized more than the value indicating the degree of edginess before applying. If it is determined to be no, the parameters (values for selecting a higher edge enhancement filter) related to resolution enhancement processing are changed. On the other hand, when noise information is received, the image conversion unit 117, in S205, determines that the pixel value (value indicating the degree of noise) after applying the high-resolution processing is higher than the noise image before the high-resolution processing. is also reduced. If it is determined to be no, the parameter (value for selecting a filter with higher noise reduction) related to resolution enhancement processing is changed.

Next, the correction amount division control section 120 in FIG. 1 will be described. The correction amount division control unit 120 determines the correction amounts Yaw_Correct and Pithc_Correct used in the second image blur correction means and the third image blur correction amount H_Total and V_Total calculated by the image blur correction amount calculation unit 119 in FIG. It is divided into correction amounts H_hom and V_hom used in the blur correction means. Then, the correction amount division control unit 120 provides the correction amounts Yaw_Correct and Pithc_Correct to the image sensor correction amount conversion unit 121 and the correction amounts H_hom and V_hom to the electronic correction amount conversion unit 122 . As for the dividing method, the ratio between the image sensor correction amount and the electronic correction amount may be changed according to the image blurring correction amount calculated from the angular velocity sensor 115, or another method may be used.

Next, the electronic correction amount converter 122 in FIG. 1 will be described with reference to the flowchart in FIG. Note that the processing shown in FIG. 4 is repeatedly executed at a predetermined cycle such as 60 Hz in the case of a video signal conforming to the NTSC format, for example.

First, in S<b>301 , the electronic correction amount conversion unit 122 acquires the image blur correction amounts H_hom and V_hom supplied from the correction amount division control unit 120 . Then, the electronic correction amount conversion unit 122 calculates the image blur correction range in S302. The image blur correction range is determined from the effective image circle information acquired by the optical parameter acquisition unit 114 in FIG. The correction amounts H_hom and V_hom are restricted. Let H_hom_final and V_hom_final be the limited image blur correction amounts. In S<b>303 , the electronic correction amount conversion unit 122 determines the image blur correction amounts H_hom_final and V_hom_final calculated in S<b>302 as image blur correction amounts to be used in the image clipping control unit 109 .

As described above, even when a motion vector is detected from an image in which a partial area of the imaging area is enlarged or cut out and the resolution is lowered, erroneous detection of the motion vector is prevented by applying resolution enhancement processing. Appropriate image blur correction becomes possible. As a result, it is possible to provide an image blur correction device capable of realizing appropriate image blur correction control even in photographing with a high electronic magnification.

In addition, in the present embodiment, an example of using an edge enhancement filter or a brightness enhancement filter as a method of high resolution processing has been described, but other super resolution techniques such as performing high resolution processing using reinforcement learning technology may be used. Alternatively, means for detecting a motion vector using information other than the luminance value may be used, or the motion vector search range, detection area, and resolution enhancement rate may be varied according to the motion vector detection image.

[Second embodiment]
FIG. 7 is a block diagram showing a configuration example of an image processing apparatus 200 according to the second embodiment. In addition, the same code|symbol is attached|subjected to the structure similar to FIG. 1, and description is abbreviate|omitted. 7 has a configuration in which the image enlargement processing unit 108 and the image conversion unit 117 are replaced with an image enlargement processing unit 130 and an image conversion unit 131 in the configuration of FIG. Other than this, it is the same as the first embodiment.

First, the image enlargement processing unit 130 in the second embodiment will be described. The image enlargement processing unit 130 is repeatedly executed at a predetermined cycle such as 60 Hz in the case of a video signal conforming to the NTSC format, for example. The image enlargement processing unit 130 enlarges the frame image stored in the image memory 107 at an arbitrary magnification, and supplies the image clipping control unit 109 and the image conversion unit 131 with the enlarged image. Examples of enlargement processing used by the image enlargement processing unit 108 include a bicubic method and a bilinear method. The image provided by the image enlargement processing unit 130 to the image conversion unit 131 is composed of a plurality of images, an image enlarged from the optical center and an image enlarged by shifting the enlargement center coordinates by an arbitrary number of pixels.

Next, the processing of the image conversion unit 131 will be described with reference to the flowchart of FIG. Note that the processing shown in FIG. 8 is repeatedly executed at a predetermined cycle such as 60 Hz in the case of a video signal conforming to the NTSC format, for example.

First, in S<b>401 , the image conversion unit 131 acquires from the image enlargement processing unit 130 information such as electronic enlargement magnification and image information after image enlargement for a plurality of images. Brightness information is included as an example of image information. Next, in S402, the image conversion unit 131 determines whether or not the electronic enlargement magnification is greater than or equal to a certain threshold. The image conversion unit 131 advances the process to S403 if it determines that it is equal to or greater than the threshold value, and advances the process to S404 if it determines that it is within the threshold value.

In S403, the image conversion unit 131 performs resolution enhancement processing using the image information required for resolution enhancement processing acquired in S401. In this embodiment, as an example, the information of the plurality of images acquired in S401 is used to perform resolution enhancement processing. Although detailed description is omitted, the image conversion unit 131 converts pixels that do not exist in the first image that has been enlarged from the optical center by the image enlargement processing unit 130 to center coordinates. Pixels are interpolated from the second frame based on the amount of shift when the image is shifted and enlarged, an image with a larger number of pixels is generated, and a high-resolution image is generated by performing luminance enhancement filter processing.

In S404, the image conversion unit 131 determines whether or not the image can be used for motion vector detection before performing the high resolution processing in S403. As an example, it is determined whether or not the luminance value of the generated high-resolution image is emphasized compared to before performing the high-resolution processing. If it is determined no, the resolution enhancement processing S403 is performed again after changing the parameters related to the resolution enhancement processing (values for selecting a luminance enhancement filter with a higher degree of enhancement). Then, when the image conversion unit 131 determines in S403 that the above conditions are more effective than before the resolution enhancement processing is performed, the image conversion unit 131 provides the motion vector detection unit 118 with the image subjected to the resolution enhancement processing.

In the above example, the information received by the image conversion unit 131 from the image enlargement processing unit 130 is brightness information, but may be edge information or noise information. When the edge information is received, in S404, the image conversion unit 131 detects that the pixel values (values indicating the degree of edge) of the image after the edge enhancement process are more enhanced than the luminance image before the resolution enhancement process. determine whether or not there is If it is determined to be no, the parameter (value for selecting a filter that emphasizes edges more) related to the resolution enhancement process is changed. On the other hand, if noise information is received, in S404 the image conversion unit 131 converts the pixel values (values indicating the degree of noise) of the image after the noise removal process to the noise information before the resolution enhancement process. It is determined whether or not it is reduced more than the image. If it is determined to be no, the parameter (value for selecting a filter with higher noise reduction) related to resolution enhancement processing is changed.

As described above, even when a motion vector is detected from an image in which a partial area of the imaging area is enlarged or cut out and the resolution is lowered, erroneous detection of the motion vector is prevented by applying resolution enhancement processing. Appropriate image blur correction becomes possible. As a result, it is possible to provide an image blur correction device capable of realizing appropriate image blur correction control even in photographing with a high electronic magnification. Note that the application of the detected motion vector is not limited to image blur correction. For example, it can be used for various purposes such as main subject region discrimination for distinguishing between a main subject region and a background region, moving object prediction for predicting the motion of a moving subject, and imaging control using those results.

Further, in the present embodiment, a method of performing resolution enhancement processing using two images has been described as a method of resolution enhancement processing. good. Alternatively, means for detecting a motion vector using information other than the luminance value may be used, or the motion vector search range, detection area, and resolution enhancement rate may be varied according to the motion vector detection image. Further, motion vector detection may be performed on an image that has undergone resolution enhancement processing, and the timing at which resolution enhancement processing is performed is not limited to the timing described above. For example, an image before being enlarged may be subjected to resolution enhancement processing, or an image before being stored in the image memory 107 or an image from which a partial region has been cut may be subjected to resolution enhancement processing. good too. The resolution enhancement process at this time may use the technique of this embodiment and the technique of the same kind as the above technique.

Although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms without departing from the gist of the present invention can be applied to the present invention. included.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the claims are appended to make public the scope of the invention.

DESCRIPTION OF SYMBOLS 100... Image processing apparatus 101... Lens unit 105... Imaging element 107... Image memory 108... Image enlargement process part 109... Image clipping control part 110... Display control part 111... Display device 112... Recording control Unit 113 Recording medium 114 Optical parameter acquisition unit 117 Image conversion unit 118 Motion vector detection unit 119 Image blur correction amount calculation unit 120 Correction amount division control unit 121 Imaging element correction amount Conversion unit 122: electronic correction amount conversion unit

Claims (11)

an imaging device;
an image memory for storing an image obtained by imaging with the imaging device;
clipping means for clipping a partial area of the image stored in the image memory as an output image;
a detection means for detecting a motion vector using a plurality of images obtained by imaging with the imaging device;
and processing means for performing a predetermined resolution enhancement process on an image obtained by imaging with the imaging device,
The imaging apparatus, wherein the detection means detects a motion vector of the partial area cut out by the cutout means using the high-resolution image that has undergone the predetermined high-resolution processing. 2. The imaging apparatus according to claim 1, wherein said processing means determines a resolution enhancement rate based on an image size. 3. The imaging apparatus according to claim 1, wherein said predetermined resolution enhancement processing performed by said processing means is brightness enhancement filtering processing of an image. 3. The imaging apparatus according to claim 1, wherein said predetermined resolution enhancement processing performed by said processing means is edge enhancement filtering processing of an image. further comprising enlarging means for enlarging the image stored in the image memory according to a set magnification;
5. The imaging apparatus according to any one of claims 1 to 4, wherein the clipping means cuts out the image after being enlarged by the enlarging means. 6. The imaging apparatus according to claim 5, wherein said processing means performs said predetermined high-resolution processing on the image after being enlarged by said enlarging means. 7. The imaging apparatus according to claim 1, further comprising recording control means for recording said output image on a recording medium different from said image memory. 8. The imaging apparatus according to any one of claims 1 to 7, further comprising display control means for displaying the output image on the display unit. 9. The imaging apparatus according to claim 1, further comprising image blur correction means for correcting image blur using the motion vector detected by said detection means. A control method for an imaging device comprising an imaging device and an image memory for storing an image obtained by imaging with the imaging device, comprising:
a clipping step of clipping a portion of the image stored in the image memory as an output image;
a processing step of performing a predetermined resolution enhancement process on an image obtained by imaging with the imaging device;
a detection step of detecting a motion vector of the partial region cut out in the cutout step using the high-resolution image that has undergone the predetermined high-resolution processing;
A control method for an imaging device, comprising: A program for causing the processor to execute each step of the method according to claim 10 by being read and executed by a processor in an imaging device provided with an imaging device and an image memory for storing an image obtained by imaging with the imaging device.

Images