JP2016006920A - Imaging apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of correcting a motion vector while effectively sharing a resource.SOLUTION: The imaging apparatus comprises: an imaging device which converts an optical image having passed through an optical system to an image signal; vector detection means which detects a motion vector; processing means which performs geometrical deformation processing of the image signal and correction processing of the motion vector; and determination means for determining the execution of the correction processing of the motion vector by the processing means.

Description

  The present invention relates to an imaging apparatus and an image processing method.

  In an imaging apparatus such as a compact digital camera, a single-lens reflex camera, and a video camera, there is a problem that image blur due to camera shake of a photographer occurs. In Patent Literature 1, a plurality of frames of captured images are stored in a memory, images are compared (template matching) between previous and next frames, a motion vector is detected, an amount of camera shake is estimated, and image blur is suppressed. Is disclosed. Patent Document 2 discloses a technique for correcting distortion of an optical system, rolling shutter distortion of a sensor, and a rotational component of image blur by performing geometric deformation processing on the image.

  FIG. 15 shows images before and after the geometric deformation process. In general, it is desirable that the amount of camera shake for performing the image stabilization process be calculated in real time, and a motion vector is detected in the original image before distortion correction illustrated in FIG. However, since the actual image is the image after distortion correction shown in FIG. 15B, the vector detection frame 1501, the center of gravity 1502 of the vector detection frame 1501, and the motion vector 1503 are distorted, and the accuracy of the motion vector 1503 This causes a problem of lowering. Patent Document 3 discloses a technique for correcting motion vector distortion by performing coordinate transformation processing (affine transformation) in consideration of the amount of deformation of an image on a motion vector.

JP 2000-261757 A JP 2006-127083 A JP 2009-258868 A

  The calculation of the amount of deformation of the image and the coordinate transformation process are preferably performed by hardware because of the high calculation load, but there is a problem that the circuit scale becomes large. Therefore, it is conceivable as a solution to correct the distortion of the motion vector by sharing the circuit resource with the coordinate calculation process of the geometric deformation process.

  FIG. 16A shows the timing from motion vector detection processing to image stabilization control. FIG. 16B shares circuit resources and detects motion vectors, motion vector correction, image stabilization control. It shows the timing until it performs. In FIG. 16A, in order to quickly calculate the amount of camera shake, the vector value is read out in units of vector detection frames for one horizontal line, for example, during the vector detection process. Then, the vector value of the last horizontal line is read, the amount of camera shake is calculated, and image stabilization control is performed. In FIG. 16B, even if the vector detection process is completed, the motion vector correction process cannot be performed because the resources of the coordinate arithmetic circuit are not released unless the geometric deformation process is completed. Therefore, since the time when the vector value can be used is delayed as compared with the conventional case, the time during which the image stabilization control can be performed is delayed.

  In view of such a problem, an object of the present invention is to provide an imaging device capable of correcting a motion vector while effectively sharing resources.

  An imaging apparatus according to one aspect of the present invention performs an image sensor that converts an optical image that has passed through an optical system into an image signal, a vector detection unit that detects a motion vector, a geometric deformation process of the image signal, and It is characterized by comprising processing means for performing motion vector correction processing, and determination means for determining execution of the motion vector correction processing by the processing means.

  An image processing method according to another aspect of the present invention includes an image sensor that converts an optical image that has passed through an optical system into an image signal, vector detection means that detects a motion vector, and geometric deformation processing of the image signal. And an image processing method for an imaging apparatus comprising: a processing unit that performs the motion vector correction process; and a determination unit that determines execution of the motion vector correction process by the processing unit. In the determination step for determining whether the motion vector correction processing by the processing means is necessary, and in the determination step, when it is determined that the correction processing is not necessary, the processing means performs the motion vector correction processing. First, it is characterized by including a processing step of performing the correction processing of the motion vector when it is determined that the correction processing is necessary.

  An image processing method according to another aspect of the present invention includes an image sensor that converts an optical image that has passed through an optical system into an image signal, vector detection means that detects a motion vector, and geometric deformation processing of the image signal. And processing means for performing correction processing of the motion vector, determination means for determining execution of the correction processing of the motion vector for each detection frame obtained by dividing the image signal, and reading the motion vector, An image processing method for an imaging apparatus having a control unit for calculating an amount of camera shake, wherein the determination unit determines whether the motion unit needs to execute correction processing of the motion vector, and A first reading step for reading out the motion vector of the detection frame determined by the control unit as not requiring correction processing in the determination step; A deformation processing step in which a stage performs geometric deformation processing of the image signal; a correction processing step in which correction processing of the motion vector of the detection frame determined to require correction processing in the determination step; and the control means Comprises a second reading step for reading the motion vector corrected in the correction processing step.

  An imaging apparatus capable of correcting a motion vector while effectively sharing resources can be provided.

1 is a block configuration diagram of an imaging apparatus according to Embodiment 1. FIG. It is the figure which showed the outline | summary of the vector detection process. It is the figure which showed the outline | summary of the geometric deformation process. It is a block diagram of a coordinate calculation means. 3 is a flowchart of image processing according to the first exemplary embodiment. It is the figure which showed the search range in figure distortion and vector detection. It is the figure which showed the template size in figure distortion and vector detection. It is the figure which showed the distortion aberration characteristic. It is the figure which showed the read-out timing of an image pick-up element. 6 is a block configuration diagram of an image pickup apparatus according to Embodiment 2. FIG. 10 is a flowchart of image processing according to the second embodiment. 6 is a timing chart of image processing according to the second embodiment. It is the figure which showed the outline | summary of the vector correction flag. It is the figure which showed the example of edge shape. It is the figure which showed the concept of vector correction. It is a timing chart of conventional vector detection.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram of the imaging apparatus according to the present embodiment. An optical image that passes through the optical system 101 and is imaged on the image sensor 102 is converted into an image signal by the image sensor 102. An image signal output from the image sensor 102 is held in the memory 103. The memory 103 outputs the held image signal to the vector detection unit 104 and the geometric deformation unit 105. The vector detection unit 104 calculates a vector value from the image signal output from the image sensor 102 and the image signal of the previous frame output from the memory 103. The calculated vector value is sent to the system control means 107. The geometric deformation unit 105 inputs a pixel group necessary for the geometric deformation process to the memory 103 based on the geometric deformation amount in the output pixel input from the internal coordinate calculation unit 105 a. Based on the input pixel group, the memory 103 inputs data at a memory address corresponding to the pixel group, out of the stored image signals, to the geometric deformation means 105. The geometric deformation unit 105 generates an output image signal (anti-vibration image) by generating an interpolation pixel in the output pixel based on the sequentially read image signal and the amount of geometric deformation, and outputs it to the outside. . The vector value calculated by the vector detection unit 104 can be input to the coordinate calculation unit 105a. The coordinate calculation unit 105 a corrects the vector value and outputs the corrected vector value to the vector detection unit 104. An imaging parameter analysis unit (determination unit) 106 controls vector value correction processing based on imaging parameters input from the system control unit 107. A system control unit 107 controls the operation of the entire imaging apparatus. The focal length, aperture value, focus distance of the optical system 101, driving of the image sensor 102, vector values are acquired from the vector detection means 104, and lens drive control for vibration isolation of the optical system 101 is performed.

  Next, details of the vector detection means 104 will be described. The vector detection unit 104 detects a motion vector between the two input frame images based on the template arrangement determined by the system control unit 107.

  In this embodiment, a motion vector is detected using a template matching method. An image signal input from the image sensor 102 is an original image, and an image signal input from the memory 103 is a reference image. As shown in FIG. 2, a template 201 is arranged at an arbitrary position in the original image. A correlation value with each region of the reference image is calculated. At this time, since calculating the correlation value for the entire region of the reference image requires a large amount of computation, a rectangular region for calculating the correlation value on the reference image is actually used as the motion vector search range 202. Set. The position and size of the search range 202 are not particularly limited, but a correct motion vector cannot be detected unless the search range 202 includes an area corresponding to the movement destination of the template 201.

  In this embodiment, a sum of absolute difference (hereinafter abbreviated as SAD) is used as an example of a correlation value calculation method. The calculation formula of SAD is shown in Formula (1).

In Expression (1), f (i, j) represents a pixel value at the coordinates (i, j) in the template 201, and g (i, j) is an area for which a correlation value is to be calculated in the search range 202. Represents each pixel value. The area for which the correlation value is to be calculated is the same size as the template 201. In SAD, the correlation value S_SAD is obtained by calculating the absolute value of the difference between the pixel values f (i, j) and g (i, j) and obtaining the sum thereof. The smaller the value of the correlation value S_SAD, the smaller the difference in luminance value between regions, that is, the texture in the region for which the correlation value is calculated is similar to the template 201. In this embodiment, SAD is used as an example of a correlation value calculation method. However, the present invention is not limited to this, and other correlation values such as sum of squared differences (SSD) and normalized cross-correlation (NCC) are used. The calculation method may be used.

  In this embodiment, the template 201 is arranged on a lattice. The template 201 has a horizontal size of TX pixels, a vertical size of TY pixels, the search range 202 has a horizontal size of SX pixels, a vertical size of SY pixels, and the distance between the templates 201 is a horizontal size of DX pixels and a vertical size of DY pixels. . The templates 201 may be arranged in a staggered pattern so that a wide range of motion vectors can be observed with a small number of templates. Parameters such as TX, TY, DX, DY, SX, and SY can be changed to arbitrary values according to the purpose.

  Next, details of the geometric deformation process will be described. In the geometric deformation processing of this embodiment, sampling and interpolation of pixels are performed on the input image based on the output image coordinates (X ′, Y ′) of FIG. 3 so that no pixel defect occurs in the output image.

  The geometric deformation unit 105 sequentially scans pixels on the output image, and converts the pixel coordinates on the output image into pixel coordinates (X, Y) on the input image. The geometric deformation means 105 samples based on the pixel coordinates (X, Y) on the input image input from the coordinate calculation means 105a, and generates output pixel data by interpolation. As the interpolation processing, for example, bilinear interpolation processing that performs linear interpolation using four neighborhoods is assumed. Since the necessary pixel group in the vicinity of the sampling coordinate is different due to the interpolation processing, the geometrical deformation means 105 causes the pixel coordinate (X, Y) input from the coordinate calculation means 105a and the neighboring pixel necessary for the interpolation. Information on the group (X ″, Y ″) is input to the memory 103. The memory 103 reads out pixel value groups in the vicinity of the sampling pixels necessary for the interpolation processing according to the coordinate information input from the geometric deformation unit 105 and transmits the pixel value group to the geometric deformation unit 105.

  Next, a method for calculating the geometric deformation amount will be described. The coordinate calculation unit 105a combines coordinate conversions by a plurality of geometric deformations into one coordinate conversion, and performs coordinate calculation for sequentially converting the coordinates of each pixel on the input output image into a sampling image on the input image. FIG. 4 is a configuration diagram of the coordinate calculation means 105a. The coordinate calculation means 105a includes coordinate calculation means 401 and 402 corresponding to a plurality of geometric deformation elements and a coordinate movement vector synthesis means 403.

  The coordinate calculation means 401 and 402 calculate the coordinate movement vector and the coordinates after the geometric deformation process from the coordinate calculation information (respective geometric deformation parameters and coordinates before the geometric deformation process) input from the system control means 107. The coordinate calculation means 401 corrects distortion aberration, and the coordinate calculation means 402 performs projective transformation.

  The coordinate calculation means 401 uses the coordinates (X, Y) before geometric deformation processing and the aberration correction parameters Pdx, Pdy for distortion aberration, and the coordinate movement vector (drx, dry) before distortion aberration correction and the coordinates after distortion aberration correction. (Xd, Yd) is calculated. The coordinate movement vector (drx, dry) and the coordinates (Xd, Yd) are calculated from equations (2) to (4).

  The aberration correction parameters Pdx and Pdy are given by the ratio of the ideal image height rn and the real image height rd as shown in Expression (5), and are resolved in the X direction and the Y direction.

  The coordinate calculation means 402 calculates a coordinate movement vector (dhx, dhy) for projective transformation and coordinates (Xh, Yh) after the projective transformation from the coordinates (Xd, Yd) after the distortion correction and the projection transformation parameter Ph. . The coordinate vector (dhx, dhy) and the coordinate (Xh, Yh) are calculated from equations (6) to (9).

  The projective transformation parameter Ph is the projective transformation matrix H and the center of transformation (X0h, Y0h). As the projective transformation matrix H, for example, a method of calculating three axes (yaw direction, pitch direction, roll direction) rotation with respect to the optical axis as a motion parameter of the camera can be considered.

  The coordinate movement vector composition means 403 performs weight composition of the coordinates (X, Y) before the geometric deformation process, the movement vectors (drx, dry) and (dhx, dhy) obtained by the respective coordinate operations, and the geometric deformation parameter Pc. The composite coordinate vector (dcx, dcy) and the coordinate (Xc, Yc) after the geometric deformation process are calculated from the equations (10) and (11).

  The geometric deformation parameters Pc are composition weights ax, bx, ay, and by. In the present embodiment, the distortion aberration and projective transformation parameters have been described. However, a graphic distortion correction parameter by a rolling shutter and an image cut-out parameter for translational stabilization may be combined with a geometric deformation parameter.

  Next, a motion vector correction process using the coordinate calculation means 105a will be described. The vector detection means 104 inputs the vector starting point coordinates (Vsx, Vsy) and vector end point coordinates (Vex, Vey) of each vector detection frame to the coordinate calculation means 105a. The coordinate calculation means 105a performs the correction vector starting point coordinates (Vsx ′, Vsy ′) and the correction vector end point coordinates (Vex ′, Vey ′) in the flow of the equations (2) to (11), as in the geometric deformation amount calculation method. '). Alternatively, the vector detection unit 104 may hold only the vectors (Vx, Vy) and convert them to the vector start point coordinates and the vector end point coordinates using the equations (12) and (13). Sx and Sy are start coordinates of the upper left frame of the image, Tx and Ty are template sizes, Dx and Dy are template intervals, N is a horizontal frame count when the left frame is 0, and M is 0. Indicates the vertical frame count.

  Next, the image processing method of the present embodiment will be described with reference to FIG.

  In step S501, the shooting parameter analysis unit 106 analyzes the shooting parameters input from the system control unit 107, and determines whether or not to execute vector detection processing. Whether or not the vector detection process is executed is determined by a vector correction flag. When “1”, vector correction is performed, and when “0”, vector correction is not performed.

The shooting parameters input to the shooting parameter analysis unit 106 and the determination of execution of the vector detection process will be described. The shooting parameters used for determination are described in the following because there are several cases.
(I) Search Range A case where the shooting parameter is a vector search range will be described with reference to FIG. As shown in FIG. 6, a subject edge 601 at time t 1 and a subject edge 602 at time t 2 (> t 1) are arranged on a template 603. The subject edges 601 and 602 are originally the same shape, but have different shapes depending on distortion aberration of the optical system 101, rolling shutter distortion of the image sensor 102, tilt, and the like. The search range 604b in FIG. 6B is wider than the search range 604a in FIG. When the search range is wide, pattern matching is widely performed with the subject edge where the graphic distortion has occurred, and thus the vector is more easily distorted than when the search range is narrow. Therefore, the imaging parameter analysis unit 106 sets the vector correction flag to 1 when any one of the search ranges (SX, SY) at the time of vector detection is larger than a predetermined threshold (SXth, SYth).
(Ii) Template Size A case where the shooting parameter is a vector template size will be described with reference to FIG. As shown in FIG. 7, a subject edge 701 at time t1 and a subject edge 702 at time t2 (> t1) are arranged on a template 703a or 703b. The object edges 701 and 702 have different shapes as described in (i). The template 703a in FIG. 7A is larger than the template 703b in FIG. If the template is large, template matching is performed over a wide range, so that erroneous matching results are unlikely to occur. Therefore, the vector is less likely to be distorted than when the template is small. Therefore, the imaging parameter analysis unit 106 sets the vector correction flag to 1 when any one of the template sizes (TX, TY) at the time of vector detection is smaller than the predetermined threshold (TXth, TYth).
(Iii) Template Position A case where the shooting parameter is the template position will be described. The template placed at a higher image height is more susceptible to graphic distortion. Therefore, the shooting parameter analysis unit 106 sets the vector correction flag to 1 when the position of the template is higher than the position of the predetermined image height.
(Iv) Distortion Aberration Characteristic of Optical System A case where the imaging parameter is the distortion aberration characteristic of the optical system 101 will be described with reference to FIG. Curves 801a and 801b show the distortion aberration characteristics of the optical system 101 under different shooting conditions. The horizontal axis represents the image height H, and the vertical axis represents the distortion amount D (H). The greater the distortion, the greater the distortion of the vector because template matching is performed on the distorted subject. Accordingly, the imaging parameter analysis means 106 sets the vector correction flag to 1 when the absolute value of the distortion aberration characteristic at Ht at which the image height is highest in the template arrangement is larger than the predetermined threshold (Dth). . Since the distortion of the vector changes depending on the image height position, the distortion characteristics for each template are held at the plot points, and each template is compared with a predetermined threshold (DTh), and the vector correction is performed for each template. A flag may be set.
(V) Reading speed of image sensor The case where the imaging parameter is the reading speed of the image sensor 102 will be described with reference to FIG. In the case where the image sensor 102 captures an image with a rolling shutter, the readout time changes for each line, and therefore, graphic distortion occurs due to camera shake or subject shake. FIG. 9 shows the readout timing of the image sensor 102 in the time direction, and each rectangle shows readout for one image. tc represents the accumulation time, and vsize represents the number of lines (vertical image size). FIG. 9A shows a case where the difference tr between the readout time of the upper end line (v = 0) and the readout time of the lower end line (v = vsize-1) is large, that is, the readout speed of the image sensor 102 is slow. is there. FIG. 9B shows a case where the difference tr is small, that is, the reading speed of the image sensor 102 is high. As the readout speed becomes slower as shown in FIG. 9A, the figure distortion due to the rolling shutter becomes larger, and the vector becomes more distorted. Therefore, the imaging parameter analysis unit 106 sets the vector correction flag to 1 when the difference tr is larger than the predetermined threshold (TRth).

  Note that a plurality of parameters described in (i) to (v) may be used as shooting parameters. For example, by taking the logical sum of the vector correction flags, if any of the conditions that require vector correction becomes valid, control is performed so that vector correction is performed.

  In step S502, if the vector correction flag obtained in step S501 is 0, the process proceeds to step S503 of the sequence A. If the vector correction flag is not 0, that is, the process proceeds to step S504 of the sequence B.

  In step S503, the vector detection unit 104 calculates a vector value in units of vector detection frames for one horizontal line, and the system control unit 107 reads the vector value.

  In step S504, the vector detection unit 104 calculates the vector value of the entire vector detection frame.

  In step S505, the geometric deformation unit 105 performs a geometric deformation process on the image signal. At this time, the vector detection unit 104 stands by until the geometric deformation unit 105 becomes ready to input the vector value of the vector detection unit 104. The geometric deformation unit 105 transmits an enable signal to the vector detection unit 104, for example, when the vector value can be input.

  In step S506, the vector detection means 104 outputs a vector value to the geometric deformation means 105, and the coordinate calculation means 105a in the geometric deformation means 105 performs vector correction. The corrected vector value is output to the vector detection means 104. Vector correction is performed in a batch for all vector detection frames.

  In step S <b> 507, the system control unit 107 reads the corrected vector value input to the vector detection unit 104.

  In step S508, the system control unit 107 calculates camera shake information using the read vector value, and further calculates an image stabilization parameter.

  As described above, the imaging apparatus according to the present embodiment can correct a motion vector while effectively sharing resources.

  FIG. 10 is a block diagram of the imaging apparatus according to the second embodiment. The imaging apparatus according to the present exemplary embodiment has no difference from the imaging apparatus according to the first exemplary embodiment except for the vector detection unit 1004 and the image analysis unit (determination unit) 1006.

  The image analysis unit 1006 calculates a vector correction flag for each vector detection frame from the image signal output by the image sensor 102 and outputs the vector correction flag to the vector detection unit 1004. The vector detection unit 1004 calculates a vector value from the image signal output from the image sensor 102 and the image signal one frame before output from the memory 103. At this time, the vector correction flag of each vector detection frame output from the image analysis unit 1006 is added to the vector value. The vector value calculated by the vector detection means 1004 is input to the coordinate calculation means 105a in the geometric deformation means 105 based on the vector correction flag for each vector detection frame. The coordinate calculation unit 105 a corrects the vector value and outputs the corrected vector value to the vector detection unit 1004.

  Next, the image processing method of the present embodiment will be described with reference to FIG. The first embodiment is different from the first embodiment in that a vector correction flag is calculated for each frame and a vector detection frame without vector correction is read for each horizontal line, so that camera shake information is obtained in advance even during vector correction. Specifically, as shown in FIG. 12, the system control unit 107 acquires the vector value of the detection frame that is not vector-corrected at the timing 1201, and the vector value of the detection frame that is vector-corrected is 1202 for which the geometric deformation processing has been completed. Get at the timing.

  In step S <b> 1101, the image analysis unit 1006 extracts an image feature amount from the image signal from the image sensor 102. As shown in FIG. 13, a vector correction flag is calculated for each vector detection frame obtained by dividing an image signal into a plurality of parts. In FIG. 13, a frame displaying “0” indicates a frame where vector correction is not performed, and a frame displaying “1” indicates a frame where vector correction is performed.

An image analysis method and determination of execution of vector detection processing will be described. The vector correction flag is determined based on whether or not the subject within the vector detection frame is suitable for vector detection. When there is a subject suitable for vector detection, vector correction is performed to further improve the accuracy of vector detection. When there is a subject that is not suitable for vector detection, even if vector correction is performed, the effect is considered to be low, and thus vector correction is not performed. The image feature amount used for control has several cases and will be described below.
(I) Contrast (gradient) information When the contrast in the vector detection frame is low, there is no phase that produces a high correlation value, and it is easily affected by small amplitudes such as noise. There is a possibility. Therefore, the image analysis unit 1006 determines that the edge shape is unsuitable for the vector detection unit 1004 and sets the vector correction flag to 0. In addition, as a method for calculating the contrast in the vector detection frame, it may be obtained by a known algorithm, for example, an integral value of an image signal subjected to a differential filter in the vector detection frame. Further, a method for estimating the contrast of an image using a template matching result (correlation value) may be used. For example, when the difference value between the maximum correlation value CMAX and the minimum correlation value CMIN is smaller than a predetermined threshold CTH, the image analysis unit 1006 determines that the subject is a low contrast subject and sets the vector correction flag Vflag to 0.
(Ii) Edge Information FIG. 14 is a diagram showing an example of an edge shape that the vector detection unit 1004 is good at and not good at.

  FIG. 14A shows a case where one line edge (single line edge) 1401a is in the vector detection frame. Since the line edge 1401a gives a high correlation value when there is a movement in the direction of the straight edge, the vector value cannot be calculated correctly. Therefore, the image analysis unit 1006 determines that the edge shape is unsuitable for the vector detection unit 1004 and sets the vector correction flag to 0.

  FIG. 14B shows a case where a line edge (periodic edge) 1401b having periodicity is in the vector detection frame. Since the line edge 1401b frequently has a phase that gives a high correlation value, a vector value cannot be calculated correctly. Therefore, the image analysis unit 1006 determines that the edge shape is unsuitable for the vector detection unit 1004 and sets the vector correction flag to 0.

  FIG. 14C shows a case where a corner point (intersecting edge) 1401c where two or more line edges intersect is in the vector detection frame. Since the corner point 1401c concentrates in the vicinity of the phase that produces a high correlation value, a highly accurate vector value can be calculated. Therefore, the image analysis unit 1006 can determine that the edge shape is good for the vector detection unit 1004 and sets the vector correction flag to 1.

  As a method for calculating the edge shape in the vector detection frame, a known algorithm such as pattern matching, frequency analysis, feature point detection, or the like is used. Further, a method of estimating the presence or absence of a subject that is not good at vector detection using a template matching result (correlation value) may be used. For example, the vector correction flag Vflag is set to 1 when the maximum Mc value Mcmax for each pixel obtained using the shi-tomasi corner detection formula (see formula (14)) is larger than the threshold value Mcth.

  As the image feature amount, a plurality of parameters shown in (i) and (ii) may be used. For example, by taking the logical sum of the vector correction flags, if any of the conditions that require vector correction becomes valid, control is performed so that vector correction is performed.

  In step S1102, the vector detection unit 1004 calculates a vector value in units of vector detection frames for one horizontal line, and the system control unit 107 reads a vector value whose vector correction flag is 0.

  In step S1103, the geometric deformation unit 105 performs a geometric deformation process on the image signal. At this time, the vector detection unit 1004 waits until the geometric deformation unit 105 becomes ready to input the vector value of the vector detection unit 1004. The geometric deformation unit 105 transmits an enable signal to the vector detection unit 1004, for example, when the vector value can be input.

  In step S1104, the vector detection unit 1004 outputs a vector value whose vector correction flag is 1 to the geometric deformation unit 105, and the coordinate calculation unit 105a in the geometric deformation unit 105 performs vector correction. The corrected vector value is output to the vector detection means 1004.

  In step S1105, the system control unit 107 reads the vector value of the vector detection frame whose vector correction flag is 1.

  In step S1106, the system control unit 107 calculates camera shake information by a known method using the read vector value, and further calculates an image stabilization parameter.

  As described above, the imaging apparatus according to the present embodiment can correct a motion vector while effectively sharing resources.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

101 Optical system 102 Image sensor 104, 1004 Vector detection means 105 Geometric deformation means (processing means)
106 Imaging parameter analysis means (determination means)
1006 Image analysis means (determination means)

Claims (11)

An image sensor that converts an optical image that has passed through the optical system into an image signal;
Vector detection means for detecting a motion vector;
Processing means for performing geometric deformation processing of the image signal and correcting the motion vector;
An image pickup apparatus comprising: determination means for determining execution of the motion vector correction processing by the processing means.   The imaging apparatus according to claim 1, wherein the processing unit performs correction processing of the motion vector when geometric processing of the image signal is not performed.   The imaging apparatus according to claim 1, wherein the vector detection unit detects the motion vector using a template matching method.   The determination means includes a template search range used for detecting the motion vector, a template size used for detecting the motion vector, a template position used for detecting the motion vector, an aberration characteristic of the optical system, and The imaging apparatus according to claim 3, wherein execution of the motion vector correction process is determined based on at least one of a reading speed of the imaging element.   When the search range is narrower than a predetermined width, the size is larger than a predetermined size, the template position is lower than a predetermined image height position, the determination unit is configured to detect distortion of the optical system. The processing means determines not to perform the correction processing of the motion vector when at least one of the case is smaller than a predetermined value and the reading speed of the image sensor is higher than a predetermined speed. The imaging device according to claim 4.   The imaging apparatus according to claim 1, wherein the determination unit determines execution of the motion vector correction processing for each detection frame obtained by dividing the image signal into a plurality of frames.   The imaging apparatus according to claim 6, wherein the determination unit determines execution of the motion vector correction processing based on at least one of a contrast and an edge shape of the detection frame.   The determination means includes at least a case where the contrast is lower than a predetermined value, a case where a single-line edge is included in the detection frame, a case where a periodic edge is included in the detection frame, and a case where a cross edge is not included in the detection frame. 8. The imaging apparatus according to claim 7, wherein in any case, the processing unit determines not to perform the correction processing of the motion vector. It further comprises control means for reading out the motion vector and calculating the amount of camera shake,
The control unit reads the motion vector of the detection frame determined by the determination unit as not requiring correction processing before the geometric deformation processing is completed, and the determination unit determines that correction processing is necessary. The image processing method according to claim 6, wherein the motion vector of the detection frame is read after the geometric deformation process is completed. An image sensor that converts an optical image that has passed through the optical system into an image signal, a vector detection unit that detects a motion vector, a processing unit that performs a geometric deformation process of the image signal and a correction process of the motion vector, An image processing method for an imaging apparatus, comprising: a determination unit that determines execution of the motion vector correction process by the processing unit;
A determination step in which the determination unit determines whether the correction processing of the motion vector by the processing unit is necessary;
If it is determined in the determination step that correction processing is not necessary, the processing means does not perform the motion vector correction processing. If it is determined that correction processing is required, the motion vector correction processing is performed. And a processing step. An image sensor that converts an optical image that has passed through the optical system into an image signal, a vector detection unit that detects a motion vector, a processing unit that performs a geometric deformation process of the image signal and a correction process of the motion vector, An image of an imaging apparatus, comprising: a determination unit that determines execution of the motion vector correction processing for each detection frame obtained by dividing the image signal into a plurality; and a control unit that reads the motion vector and calculates an amount of camera shake. A processing method,
A determination step for determining whether or not the determination unit needs to execute correction processing of the motion vector by the processing unit;
A first reading step in which the control means reads the motion vector of the detection frame that is determined not to require a correction process in the determination step;
A deformation processing step in which the processing means performs a geometric deformation processing of the image signal;
A correction processing step of performing correction processing of the motion vector of the detection frame determined to require correction processing in the determination step;
An image processing method comprising: a second reading step in which the control means reads the motion vector corrected in the correction processing step.