JP2015056043A - Image processor, control method thereof, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy of a deviation correction parameter even when the reliability of a motion vector is low.SOLUTION: A reliability determination part 201 compares a first target block in a first image with a reference block in a second image corresponding to the first target block, and determines a reference block with a feature pattern existing. Then, the reliability determination part together with a motion vector detection part 202 sets a reference block corresponding to a second target block in the first image with the reference block with the feature pattern existing as the second target block, and calculates a correlation value between the second target block and the reference block in the first image to detect a motion vector.

Description

  The present invention relates to an image processing apparatus, a control method thereof, and a control program, and more particularly, to a method for correcting a shift of a subject image caused by a movement of an imaging apparatus that is one of image processing apparatuses.

  In general, an imaging apparatus such as a digital camera is known as one of image processing apparatuses. In the imaging apparatus, the subject image may be blurred due to the movement of the imaging apparatus due to a user's camera shake or the like during imaging. In order to prevent such blurring of the subject image, the imaging apparatus is provided with a camera shake correction function.

  The camera shake correction function is roughly classified into an optical camera shake correction function and an electronic camera shake correction function. The electronic camera shake correction function corrects subject image shifts in a plurality of images obtained as a result of imaging by image processing.

  For example, a feature pattern matching process is performed on two images obtained as a result of imaging to detect a plurality of motion vectors, and a shift correction parameter is estimated according to these motion vectors to correct a shift in a subject image. There is an imaging device (see Patent Document 1). Further, here, a highly reliable correction parameter is estimated by calculating a reliability index value for the motion vector and estimating a deviation correction parameter by excluding a motion vector with low reliability. ing.

JP 2009-258868 A

  However, in the method described in Patent Document 1, even if a motion vector with high reliability can be detected, it is determined that the reliability is low as follows, and as a result, motion vector detection is performed. May not be possible.

  FIG. 14 is a diagram for explaining an example of motion vector detection processing in a conventional imaging apparatus.

  In FIG. 14, here, detection of a motion vector for a repetitive pattern subject is shown, and a target frame (also referred to as a target image or target image) 913 serving as a reference for motion vector detection and a reference for detecting a motion vector. A frame (reference image) 914 is shown. In each of the areas 915 and 916, there is a repeating pattern for each frame.

  Using a target block (target block) 917 which is a block matching reference, block matching processing is performed on the search range 918 set in the reference frame to generate a difference absolute value sum table (SAD table). Thereby, for example, the SAD table 919 is generated. In the SAD table 919, a white portion indicates that the SAD value (that is, a correlation value) is high, and a black portion indicates that the SAD value is low.

  Since the target block 917 includes a circular pattern and the pattern is repeated on the search range 918, the SAD table 919 has a low SAD value (SAD minimum value) at a location where the circular pattern exists on the search range 918. Indicates. As a result, in the method described in Patent Document 1 described above, the reliability index value becomes small and it is determined that the reliability is low.

  However, in the illustrated example, there is a numerical pattern that is not a circular pattern on the search range 918. For this reason, although there is a pattern effective for motion vector detection in the vicinity, motion vector detection is not performed as a result of the motion vector being discarded.

  FIG. 15 is a diagram for explaining another example of motion vector detection processing in a conventional imaging apparatus.

  FIG. 15 shows the detection of a motion vector with respect to a linear object, and a target frame (target image) 901 serving as a reference for motion vector detection and a reference frame (reference image) that is a target for detecting a motion vector. ) 902 is shown. Using the target block 903 serving as a block matching criterion, a block matching process is performed on the search range 904 set in the reference frame to generate a difference absolute value sum table (SAD table).

  FIG. 16 is a diagram showing an SAD table generated by the method shown in FIG. FIG. 17 is a diagram showing an SAD table that three-dimensionally represents the SAD table shown in FIG.

  Since the target block 903 is a linear pattern, many similar patterns are detected along the linear object in the search range 904 on the reference frame 902. For this reason, a corresponding block cannot be specified in the search range 904, and as a result, a motion vector cannot be detected correctly.

  That is, in the SAD table, as shown in FIG. 17, a portion having a low SAD value (a portion having a high correlation) appears in a groove shape. For this reason, it is impossible to specify which position of these grooves is the correct corresponding block position, and it is impossible to correctly detect the motion vector.

  Even in the situation as described above, it is possible to determine the reliability of the motion vector by determining whether or not there is a groove-like correlation value distribution in the SAD table. If the reliability of the motion vector can be determined, it is possible to obtain a highly reliable correction parameter by estimating a shift correction parameter by excluding a motion vector with low reliability.

  FIG. 18 is a diagram for explaining still another example of motion vector detection processing in a conventional imaging apparatus.

  In FIG. 18, here, the detection of the motion vector related to the target block 910 on the right side of the target block 903 is shown, and again, the reliability of the detected motion vector is lowered for the same reason. Then, although there is a line segment break on the search range 911 and there is a pattern effective for motion vector detection in the vicinity, the motion vector is discarded, so that motion vector detection is not performed. End up.

  Accordingly, an object of the present invention is to provide an image processing apparatus, a control method thereof, and a control program that can improve the accuracy of a deviation correction parameter even when the reliability of a motion vector is low.

  In order to achieve the above object, an image processing apparatus according to the present invention is an image processing apparatus for obtaining a motion vector between a first image and a second image, wherein the first target in the first image is obtained. A determination unit for comparing a reference block in the second image corresponding to the first target block with a block and determining a reference block in which a feature pattern exists; and the determination unit determines that the feature pattern exists The reference block is a second target block, a reference block corresponding to the second target block is set in the first image, and the second target block and the reference block in the first image are Motion vector detection means for obtaining a correlation value and detecting a motion vector.

  A control method according to the present invention is a control method of an image processing apparatus for obtaining a motion vector between a first image and a second image, wherein the first target block and the first image in the first image A reference step in which the reference block in the second image corresponding to the target block is compared to determine a reference block in which a feature pattern exists, and the reference block in which the feature pattern is determined to exist in the determination step A reference block corresponding to the second target block is set in the first image, and a correlation value between the second target block and the reference block in the first image is determined to move. A motion vector detecting step for detecting a vector.

  A control program according to the present invention is a control program used in an image processing apparatus for obtaining a motion vector between a first image and a second image, and the first image is stored in a computer included in the image processing apparatus. A determination step of comparing a reference block in the second image corresponding to the first target block and the second target block corresponding to the first target block to determine a reference block in which a feature pattern exists; The reference block determined to exist is set as a second target block, a reference block corresponding to the second target block is set in the first image, and the second target block and the first image are set. Vector detection to detect the motion vector by obtaining the correlation value with the reference block in Characterized in that to execute a step, the.

  According to the present invention, a reference block determined to have a feature pattern is set as a second target block, a reference block corresponding to the second target block is set in the first image, and the second target block Since the motion vector is detected by obtaining the correlation value with the reference block in the first image, the motion vector with high reliability can be redetected when the motion vector is low in reliability. Thus, the accuracy of the deviation correction parameter can be improved.

It is a block diagram which shows an example of an imaging device provided with the image processing apparatus by the 1st Embodiment of this invention. It is a block diagram which shows the example about the structure of the image process part 107 shown in FIG. It is a flowchart for demonstrating operation | movement of the reliability determination part shown in FIG. It is a figure which shows an example of the SAD table produced | generated by the reliability determination part shown in FIG. It is the figure which plotted the evaluation value with respect to the deviation | shift amount of a X direction in the camera shown in FIG. FIG. 6 is a diagram in which the frequency of evaluation values that are greater than or equal to the first threshold and less than the second threshold in FIG. 5 is plotted for each amount of deviation in the X direction. It is the figure which plotted the evaluation value with respect to the deviation | shift amount of the X direction by the SAD table shown in FIG. It is the figure which plotted the frequency of the evaluation value which is more than a 1st threshold value and less than a 2nd threshold value for every deviation | shift amount of a X direction in FIG. It is a figure for demonstrating the redetection process of the motion vector with respect to the target block shown in FIG. It is a flowchart for demonstrating operation | movement of the reliability determination part in the camera provided with the image processing apparatus by the 2nd Embodiment of this invention. It is the figure which plotted the evaluation value with respect to the deviation | shift amount of the X direction by a SAD table in the camera provided with the image processing apparatus by the 2nd Embodiment of this invention. It is the figure which plotted the frequency of the minimum value which exists in the range from a 1st minimum value to an evaluation threshold value for every deviation | shift amount of a X direction in FIG. It is a figure for demonstrating the redetection process of the motion vector in a camera provided with the image processing apparatus by the 2nd Embodiment of this invention. It is a figure for demonstrating an example of the motion vector detection process in the conventional imaging device. It is a figure for demonstrating the other example of the motion vector detection process in the conventional imaging device. It is a figure which shows the SAD table produced | generated by the method shown in FIG. It is a figure which shows the SAD table which expressed the SAD table shown in FIG. 16 three-dimensionally. It is a figure for demonstrating the further another example of the motion vector detection process in the conventional imaging device.

  Hereinafter, an example of an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  Here, a case will be described in which the image processing apparatus according to the embodiment of the present invention is provided in an imaging apparatus such as a digital camera, and the shift of a subject image is corrected in an image obtained as a result of imaging by the imaging apparatus.

[First Embodiment]
FIG. 1 is a block diagram illustrating an example of an imaging apparatus including an image processing apparatus according to the first embodiment of the present invention.

  The illustrated imaging apparatus is, for example, a digital camera (hereinafter simply referred to as a camera) and includes a control unit 101. The control unit 101 is, for example, a CPU, and controls the entire camera 100 by reading an operation program from the R0M 102, developing the operation program in the RAM 103, and executing the program.

  The ROM 102 is a rewritable nonvolatile memory, and stores parameters necessary for the operation of the camera in addition to the above-described operation program. The RAM 103 is a rewritable volatile memory, and is used as a temporary storage area for various data output in the operation of the camera 100. Here, the RAM 103 temporarily stores a target image, a reference image before correction, and the like necessary for detecting a shift of a subject image in an image (referred to as a reference image) caused by the movement of the camera 100. The

  The imaging unit 105 includes an imaging element such as a CCD or CMOS sensor, for example. The imaging unit 105 outputs an analog image signal corresponding to the optical image formed on the imaging element via the optical system 104 by photoelectric conversion to the A / D conversion unit 106. The A / D conversion unit 106 performs A / D conversion processing on the analog image signal to obtain a digital image signal (image data), and stores the image data in the RAM 103.

  In the illustrated camera 100, the image processing unit 107 performs various types of image processing on image data obtained as a result of imaging. For example, the image processing unit 107 performs a correction process for correcting a shift in the subject image caused by the movement of the camera 100.

  FIG. 2 is a block diagram showing an example of the configuration of the image processing unit 107 shown in FIG.

  The illustrated image processing unit 107 receives a reference image (second image) that is a target for detecting a shift in the subject image and a target image (first image) that serves as a reference for correction. The Note that the target image and the reference image are read from the RAM 103 by the control unit 101 and given to the image processing unit 107.

  The image processing unit 107 includes a motion vector detection unit 202, and the motion vector detection unit 202 performs a reference block (here, a reference block) in a reference image for each target block (here, a first target block) in the target image. The motion vector of the subject image is detected for each first reference block). For example, the motion vector detection unit 202 compares subject images in a region near the target image and the reference image. Here, the motion vector detection unit 202 detects a motion vector according to the movement of the same pattern between the target image and the reference image using a so-called block matching method.

  The reliability determination unit 201 determines whether or not the motion vector of each block detected by the motion vector detection unit 202 has been detected in a linear or linear object (linear part). If the motion vector is not detected in the linear subject, the reliability determination unit 201 outputs the motion vector as it is. On the other hand, if the motion vector is detected in a linear object, the reliability determination unit 201 performs re-detection of the motion vector if there is a pattern effective for motion vector detection in the vicinity, and an effective stripe pattern is detected. If not, the motion vector is excluded.

  Referring to FIG. 15 described above, here, the target image 901 includes 6 × 6 = 36 target blocks 901a. The motion vector detection unit 202 detects 36 motion vectors corresponding to each of the target blocks 901a.

  In FIG. 15, since there is no linear subject for the 18 target blocks 901a located in the right half, the reliability determination unit 201 outputs the motion vector detected by the motion vector detection unit 202 as it is. On the other hand, since there are linear objects for the 18 target blocks 901a located in the left half, the reliability determination unit 201 performs redetection of the motion vector or excludes the motion vector.

  In the example illustrated in FIG. 15, for the six target blocks 901a in the third column from the left, there is a line segment break in the corresponding search range 904, so the reliability determination unit 201 performs redetection of the motion vector. For the other 12 target blocks 901a, motion vectors are excluded.

  The affine coefficient calculation unit 203 calculates an affine coefficient, which is a coordinate conversion coefficient used for correcting each region (block), using the motion vectors of each of the plurality of target blocks included in the target image (coordinate conversion coefficient calculation). As the affine coefficient calculation method, for example, the method described in Patent Document 1 is used. Then, the affine transformation unit 207 corrects the positional deviation of the reference image using the affine coefficient, and outputs the corrected reference image.

  FIG. 3 is a flowchart for explaining the operation (reliability determination) of the reliability determination unit 201 shown in FIG. Based on the motion vector for each block from the motion vector detection unit 202, the reliability determination unit 201 performs the following processing.

  When reliability determination is started, the reliability determination unit 201 first performs one-dimensional line determination to be described later for the column direction (step S101), and then performs one-dimensional line determination to be described later for the row direction (step S102). ). Then, the reliability determination unit 201 determines whether both the column direction and the row direction are lines (line segments) (step S103).

  If it is determined that neither the column direction nor the row direction is a line (YES in step S103), the reliability determination unit 201 completes the reliability determination without updating the motion vector (step S104). On the other hand, if it is determined that at least one of the column direction and the row direction is a line (NO in step S103), the reliability determination unit 201 determines whether the column direction is a broken line and the row direction is not a line. (Step S105).

  If it is determined that the column direction is a broken line and the row direction is not a line (YES in step S105), the reliability determination unit 201 resets the target block in the column direction and redetects the motion vector (step S106). ). And the reliability determination part 201 complete | finishes reliability determination. If it is determined that at least one of the column direction is not a broken line and the row direction is a line is satisfied (NO in step S105), the reliability determination unit 201 has a column direction that is not a line and the row direction is broken. It is determined whether or not the line is a line (step S107).

  If it is determined that the column direction is not a line and the row direction is a broken line (YES in step S107), the reliability determination unit 201 resets the target block in the row direction and redetects the motion vector (step S107). S108). And the reliability determination part 201 complete | finishes reliability determination. On the other hand, when it is determined that at least one of the column direction being a line and the row direction not being a broken line is satisfied (YES in step S107), the reliability determination unit 201 excludes the motion vector (step S109). And the reliability determination part 201 complete | finishes reliability determination.

  Here, the one-dimensional line determination in the column direction in step S101 described in FIG. 3 will be described.

  FIG. 4 is a diagram illustrating an example of the SAD table generated by the reliability determination unit 201 illustrated in FIG.

  The SAD table (that is, the first correlation value map) shown in FIG. 4 is equivalent to the SAD table shown in FIG. 16, and the SAD (that is, correlation value) minimum value in the column is shown in black. For example, the SAD minimum value of the column whose deviation amount in the X direction is “−10” is “1” 5 in the row where the deviation amount in the Y direction is “0”. In addition, the SAD minimum value of the column in which the deviation amount in the X direction is “3” is “2” in the row in which the deviation amount in the Y direction is “3”.

  Further, here, the SAD minimum value of the column is normalized by the SAD average value of the column, and the value is used as the evaluation value. For example, since the SAD average value of the column in which the deviation amount in the X direction is “−10” is “88”, the evaluation value is 15/88 = 0.17. Further, since the SAD average value of the column in which the deviation amount in the X direction is “3” is “89”, the evaluation value is 2/89 = 0.02.

  FIG. 5 is a diagram in which evaluation values are plotted with respect to the amount of deviation in the X direction in the camera shown in FIG.

  When the SAD minimum value is normalized for each column as described above, an evaluation value for each column is obtained as shown in FIG. Note that here, when obtaining the evaluation value, normalization, that is, obtaining the evaluation value by dividing the SAD minimum value by the SAD average value for each column, the evaluation value is obtained by other methods. Also good. For example, the evaluation value may be obtained by dividing the SAD minimum value by the integrated value of the SAD value for each column, the maximum value, or the difference value between the maximum value and the minimum value. Furthermore, the evaluation value is obtained by dividing the SAD minimum value by the average value, integrated value, maximum value, or difference value between the maximum value and the minimum value for each column corresponding to each shift amount in the X direction. You may do it.

  The reliability determination unit 201 illustrated in FIG. 2 determines that the SAD table has a groove when the ratio of evaluation values that are greater than or equal to the first threshold and less than the second threshold in the one-dimensional line determination in the column direction is greater than or equal to the third threshold. It is determined that the line is a line. Now, it is assumed that the first threshold is “0”, the second threshold is “0.3”, and the third threshold is “0.5”. In the example illustrated in FIG. 5, the number of evaluation values that are greater than or equal to the first threshold and less than the second threshold is “21”, and the ratio is 21/21 = 1. And since the said ratio is more than a 3rd threshold value, the reliability determination part 201 will determine as a line. When it is determined that the line is a line, the reliability determination unit 201 determines whether the line is a broken line or a complete line.

  FIG. 6 is a diagram in which the frequency of evaluation values that are greater than or equal to the first threshold and less than the second threshold in FIG. 5 is plotted for each amount of deviation in the X direction.

  In FIG. 6, since the grooves are distributed over the entire amount of deviation in the X direction (that is, the distribution density is high), the reliability determination unit 201 determines that the above line is a complete line and not a broken line. judge.

  Also for the SAD table 912 shown in FIG. 18, the one-dimensional line determination in the column direction is performed in the same manner as the SAD table shown in FIG.

  FIG. 7 is a diagram in which evaluation values with respect to the amount of deviation in the X direction are plotted based on the SAD table 912 shown in FIG.

  In the example illustrated in FIG. 7, the number of evaluation values that are greater than or equal to the first threshold and less than the second threshold is “14”, and the ratio is 14/21 = 1. And since the said ratio is more than a 3rd threshold value (here 0.5), the reliability determination part 201 will determine as a line. When it is determined that the line is a line, the reliability determination unit 201 determines whether the line is a broken line or a complete line.

  FIG. 8 is a diagram in which the frequency of evaluation values that are greater than or equal to the first threshold and less than the second threshold in FIG. 7 is plotted for each amount of deviation in the X direction.

  In FIG. 8, since the distribution of the grooves is interrupted when the amount of deviation in the X direction = 3, the reliability determination unit 201 determines that the above line is an interrupted line. Further, the corresponding shift amount in the Y direction is “3” from FIG. Therefore, the reliability determination unit 201 sets the line break position (X, Y) = (3, 3) as the shift amount of the column target block.

  In the one-dimensional line determination in the column direction, an evaluation value is obtained for the X-direction deviation amount, but in the one-dimensional line determination in the row direction, an evaluation value for the Y-direction deviation amount is obtained. Processing in the row direction is the same as in the column direction.

  By the way, for the target block 903 shown in FIG. 15, since the reliability determination unit 201 determines that the column is not a complete line and the row is not a line, as described in the flowchart shown in FIG. In this case, the motion vector is excluded.

  On the other hand, for the target block 910 shown in FIG. 18, since the reliability determination unit 201 determines that the column is not a broken line and the row is not a line, as described in the flowchart shown in FIG. In this case, the motion vector is detected again.

  FIG. 9 is a diagram for explaining motion vector redetection processing for the target block 910 shown in FIG.

  When the motion vector is re-detected, the column target block 301 (here, the second target block) is newly reset on the reference image 902 (here, the second image), and further, the target image The column search range 302 is reset on 901 (here, the first image). At this time, the column target block 301 is reset and shifted on the target image 901 by the column target block shift amount (3, 3). The column search range 302 is reset on the reference image 901 by being shifted by the same shift amount.

  Here, the reference block 303 is a block having a minimum SAD value with respect to the column target block 301, and the SAD table 304 is a SAD table corresponding to the reference block 303. Here, the motion vector 305 is re-detected as shown in the figure.

  For other target blocks, the starting point of the motion vector is on the target image 901. For this reason, the starting point of the motion vector (first motion vector or second motion vector) is set to the center of the reference block 303 on the target image 901 so that the starting point of the redetected motion vector 305 is also on the target image 901. The

  As described above, the corrected reference image is output from the image processing unit 107 and recorded on the recording medium 108 as a corrected reference image. The recording medium 108 is, for example, a recording device such as an internal memory provided in the camera 100 or a memory card or HDD that is detachably attached to the camera 100.

  Note that if the subject image shift is not corrected in the image obtained as a result of imaging, the image processing unit 107 performs predetermined conversion processing and encoding processing on the image data output from the A / D conversion unit 106. To record on the recording medium 108.

  As described above, in the first embodiment of the present invention, it is possible to correct a shift of the reference image with respect to the target image caused by the movement of the camera 100 when the reference image is captured among the target image and the reference image. .

  Here, the image processing unit 107 determines whether or not the motion vector detected by the block matching is a linear subject, and uses the motion vector detected as not being a linear subject as it is. On the other hand, if the object is a linear object, the image processing unit 107 performs re-detection of the motion vector. Then, the image processing unit 107 excludes the motion vector when a highly reliable motion vector cannot be obtained.

  As a result, there is a high possibility that a highly reliable motion vector is detected again even when the motion vector is low in reliability when the motion vector is detected in a linear object, and the accuracy of the deviation correction parameter is improved. Can be improved.

[Second Embodiment]
Next, an example of a camera provided with an image processing apparatus according to the second embodiment of the present invention will be described.

  The configuration of the camera according to the second embodiment is the same as that of the camera shown in FIG. The configuration of the image processing unit is the same as the configuration of the image processing unit shown in FIG. 2, but the function of the reliability determination unit 210 is different.

  FIG. 10 is a flowchart for explaining the operation of the reliability determination unit in the camera including the image processing apparatus according to the second embodiment of the present invention.

  When the reliability determination is started, the reliability determination unit 201 first performs a one-dimensional repetition determination described later for the column direction (step S201), and then performs a one-dimensional repetition determination described later for the row direction (step S202). ). Then, the reliability determination unit 201 determines whether the pattern is a repetitive pattern (that is, a feature pattern including feature points) in both the column direction and the row direction (step S203).

  If it is determined that neither the column direction nor the row direction is a repetitive pattern (YES in step S203), the reliability determination unit 201 completes the reliability determination without updating the motion vector (step S204). On the other hand, if it is determined that at least one of the column direction and the row direction is a repetitive pattern (NO in step S203), the reliability determination unit 201 determines whether or not the column direction is a repetitive pattern and is interrupted (step S205). ).

  If it is determined that the column direction is a repetitive pattern and there is a discontinuity (YES in step S205), the reliability determination unit 201 resets the target block in the column direction and redetects the motion vector (step S206). Then, the reliability determination unit 201 determines whether or not the row direction is a repeated pattern and is interrupted (step S207).

  If it is determined that at least one of the column direction is not a repetitive pattern and that there is no interruption is satisfied (NO in step S205), the reliability determination unit 201 proceeds to the process of step S207.

  If it is determined that the row direction is a repetitive pattern and there is a break (YES in step S207), the reliability determination unit 201 resets the target block in the column direction and redetects the motion vector (step S208). Then, the reliability determination unit 201 determines whether or not a motion vector is detected (step S209).

  If it is determined that at least one of the row direction is not a repetitive pattern and that there is no interruption is satisfied (NO in step S207), the reliability determination unit 201 proceeds to the process of step S209.

  If no motion vector is detected (NO in step S209), reliability determination unit 201 excludes the motion vector (step S210) and ends the reliability determination. On the other hand, when a motion vector is detected (YES in step S209), reliability determination unit 201 ends the reliability determination.

  Here, the one-dimensional repetition determination in the column direction in step S201 described with reference to FIG. 10 will be described.

  FIG. 11 is a diagram in which evaluation values are plotted with respect to a deviation amount in the X direction according to the SAD table in a camera including an image processing apparatus according to the second embodiment of the present invention.

  Here, the SAD minimum value is detected for each column of the SAD table in the same manner as the one-dimensional line determination in the column direction. The evaluation value is the SAD minimum value for each column. FIG. 11 plots evaluation values with respect to the amount of deviation in the X direction with respect to the SAD table 919 shown in FIG.

  In the one-dimensional repetition determination in the column direction, if another minimum value exists in the range from the first minimum value (smallest minimum value) of the evaluation values to the evaluation value threshold, the reliability determination unit 201 is a repeated pattern. judge. In the example illustrated in FIG. 11, since another minimum value exists in the range from the first minimum value to the evaluation value threshold, the reliability determination unit 201 determines that the pattern is a repetitive pattern. When it is determined that the pattern is a repetitive pattern, the reliability determination unit 201 determines whether the pattern is a discontinuous repeat pattern or a complete repeat pattern as described above.

  FIG. 12 is a diagram in which the frequency of the minimum value existing in the range from the first minimum value to the evaluation threshold in FIG. 11 is plotted for each amount of deviation in the X direction.

  As shown in FIG. 11, since the minimum value appears every evaluation value period, the deviation amount in the X direction also appears every evaluation value period. As shown in FIG. 12, since the distribution of the repetitive pattern is discontinued at the X-direction deviation amount = 3, the reliability determination unit 201 determines that the repetitive pattern is a discontinuous repetitive pattern. Then, the reliability determination unit 201 sets the interruption position (X, Y) = (3, 0) as the shift amount of the column target block.

  In the one-dimensional repetition determination in the column direction, an evaluation value is obtained for the X-direction displacement amount, but in the one-dimensional repetition determination in the row direction, an evaluation value for the Y-direction displacement amount is obtained. The processing in the row direction is the same as that in the column direction.

  When the one-dimensional repetitive pattern determination in the column direction and the row direction is performed as described above, for example, for the target block 917 shown in FIG. 14, the column direction is determined to be a discontinuous repetitive pattern and the row direction is determined to be a complete repetitive pattern. The As a result, as described in the flowchart shown in FIG. 10, the reliability determination unit 201 redetects the motion vector in step S206.

  FIG. 13 is a diagram for explaining motion vector redetection processing in a camera including an image processing apparatus according to the second embodiment of the present invention.

  FIG. 13 shows motion vector redetection processing for the target block 917 shown in FIG. 14. When the motion vector is redetected, a new column target block 401 is reset on the reference image 914. Further, the column search range 402 is reset on the target image 913. At this time, the column target block 401 is reset on the target image 913 by being shifted by the column target block shift amount (3, 0). The column search range 402 is reset by being shifted on the reference image 913 by the same shift amount.

  Here, the reference block 403 is a block having a minimum SAD value with respect to the column target block 401, and the SAD table 404 is a SAD table corresponding to the reference block 403. Here, the motion vector 405 is re-detected as shown in the figure.

  For other target blocks, the starting point of the motion vector is on the target image 913. For this reason, the starting point of the motion vector is set to the center of the reference block 403 on the target image 913 so that the starting point of the redetected motion vector 405 is also on the target image 913.

  As described above, the corrected reference image is output from the image processing unit 107 and recorded on the recording medium 108 as a corrected reference image. Note that, as in the first embodiment, the image processing unit 107 applies the image data output from the A / D conversion unit 106 when the subject image shift is not corrected in the image obtained as a result of imaging. Then, a predetermined conversion process and an encoding process are performed and recorded on the recording medium 108.

  As described above, also in the second embodiment of the present invention, it is possible to correct the shift of the reference image with respect to the target image caused by the movement of the camera 100 when the reference image is captured among the target image and the reference image. it can.

  Here, the image processing unit 107 determines whether the motion vector detected by the block matching is a repetitive pattern, and uses the motion vector detected as not being a repetitive pattern as it is. On the other hand, if the pattern is a repetitive pattern, the image processing unit 107 redetects the motion vector. Then, the image processing unit 107 excludes the motion vector when a highly reliable motion vector cannot be obtained.

  As a result, there is a high possibility that a highly reliable motion vector is redetected even when the motion vector is low in reliability when the motion vector is detected in the repetitive pattern, thereby improving the accuracy of the deviation correction parameter. be able to.

  As is clear from the above description, in the example shown in FIG. 2, the motion vector detection unit 202 and the reliability determination unit 201 function as a determination unit and a motion vector detection unit. In the example illustrated in FIG. 1, at least the control unit 101 and the image processing unit 107 constitute an image processing apparatus.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a control method, and this control method may be executed by the image processing apparatus. In addition, a program having the functions of the above-described embodiments may be used as a control program, and the control program may be executed by a computer included in the image processing apparatus. The control program is recorded on a computer-readable recording medium, for example.

  Each of the above control method and control program has at least a determination step and a motion vector detection step.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed.

DESCRIPTION OF SYMBOLS 101 Control part 104 Optical system 105 Image pick-up part 106 A / D conversion part 107 Image processing part 108 Recording medium 201 Reliability determination part 202 Motion vector detection part 203 Affine coefficient calculation part 207 Affine conversion part

Claims (7)

An image processing apparatus for obtaining a motion vector between a first image and a second image,
Determining means for comparing a first target block in the first image and a reference block in the second image corresponding to the first target block, and determining a reference block having a feature pattern;
The reference block determined by the determining unit to determine that the feature pattern exists is set as a second target block, a reference block corresponding to the second target block is set in the first image, and the second block is set. Motion vector detection means for detecting a motion vector by obtaining a correlation value between a target block and a reference block in the first image;
An image processing apparatus comprising:   The determination unit generates a first correlation value map that represents a correlation value between the first target block and a reference block in the second image, and the feature pattern according to the first correlation value map The image processing apparatus according to claim 1, wherein it is determined whether or not the reference block including   The determination means determines that the reference block having the feature pattern exists when the first correlation value map has a line shape and the line is interrupted, and is located at an interrupted portion of the line. The image processing apparatus according to claim 2, wherein the reference block is set as the second target block.   When the first correlation value map has a plurality of local minimum values and there is a region where the distribution density of the local minimums is lower than a predetermined threshold, the determination unit includes the reference pattern. The reference block located in a region where the distribution density of the local minimum is lower than a predetermined threshold is set as the second target block. Image processing device.   When there are a plurality of the first target blocks, the motion vector detection means, for each of the first target blocks, the first motion vector corresponding to the first target block or the second target When the second motion vector corresponding to the block is detected and the first motion vector or the second motion vector is detected, the starting points of the first motion vector and the second motion vector are the same. The image processing apparatus according to claim 1, wherein the image processing apparatus is positioned in an image. A control method of an image processing apparatus for obtaining a motion vector between a first image and a second image,
A determination step of comparing a first target block in the first image and a reference block in the second image corresponding to the first target block to determine a reference block in which a feature pattern exists;
The reference block determined as having the feature pattern in the determination step is set as a second target block, a reference block corresponding to the second target block is set in the first image, and the second block is set. A motion vector detection step of detecting a motion vector by obtaining a correlation value between a target block and a reference block in the first image;
A control method characterized by comprising: A control program used in an image processing device for obtaining a motion vector between a first image and a second image,
In the computer provided in the image processing apparatus,
A determination step of comparing a first target block in the first image and a reference block in the second image corresponding to the first target block to determine a reference block in which a feature pattern exists;
The reference block determined as having the feature pattern in the determination step is set as a second target block, a reference block corresponding to the second target block is set in the first image, and the second block is set. A motion vector detection step of detecting a motion vector by obtaining a correlation value between a target block and a reference block in the first image;
A control program characterized by causing

Images