TWI390454B - Shaking detection device and method - Google Patents

Description

Shake detection device and method

The present invention relates to a sway detecting device and a method, and more particularly to a technique for detecting a swaying correction function required for the image quality of a captured image caused by shaking of the photographic device to detect the swaying.

In order to prevent the image quality caused by shaking of the camera, the camera and the video camera have a shake correction function. The shaking of the camera at the time of shooting is usually a vibration in the range of about 1 Hz to 15 Hz. Even if such a sway occurs, the elemental technology for the swaying correction function required to capture an image without image swaying is the technique of detecting swaying and the technique of correcting swaying.

The former technology of detecting swaying is roughly divided into a motion vector detection method and an angular velocity detection method. The motion vector detection method is a method in which the image data captured by a CCD (Charge Coupled Device) or the like is electronically processed to detect the movement vector of the camera. On the other hand, the angular velocity detecting method uses a Gyrosensor or the like to detect the angular velocity. The method of detecting the swaying by electronic means generally uses a method of obtaining the image sway (image movement amount) by obtaining the correlation between the two image data captured by the interval time difference (for example, refer to Patent Document 1).

Patent Document 1: Japanese Laid-Open Patent Publication No. Hei 10-173992.

In general, when the amount of image movement is obtained by observing the correlation of the plurality of image data, a block of a certain size (hereinafter referred to as a reference block) cut by the center of the image of the reference is detected. Compare the moves in the image of the object to where. At this time, while scanning the image of the reference block on the image of the comparison target, the degree of similarity of the two images at each scanning position is calculated, and it is judged that the reference block is moved to the scanning position with the highest similarity.

In addition, the latter's correction sway technology is roughly divided into optical and electronic. The optical sway correction system physically shifts the optical axis in response to the detected movement of the camera. On the other hand, the electronic sway correction system prepares a CCD larger than the actual shooting area, and uses one of them as a shooting area. That is, in response to the detected movement of the camera, the image of the CCD is cut, and the area (photographing area) is moved up, down, left, and right to correct the shaking motion.

Using the motion vector detection method as a technique for detecting sloshing, and using the electronic method as a technique for correcting sloshing, the movement of the camera can be detected by the software processing only to correct the sway, thereby facilitating the small size of the product. Chemical. However, the correction accuracy and processing speed of the shaking at this time are both in the electronic processing content.

In general, if the correction accuracy is to be improved, the contents of the electronic processing become complicated, the processing load becomes heavy, and the processing speed is lowered. Conversely, in order to improve the processing speed and simplify the processing content, the error detection of the image movement amount is increased, and the correction precision is lowered. In particular, it is very difficult to correctly detect the amount of image movement with a small amount of calculation.

A key factor in the error detection of the amount of image movement is the simple method of "requiring the correlation of multiple images". For example, when the same pattern exists in a plurality of places in the object to be photographed, a plurality of identical patterns appear in the captured image data. At this time, even if a simple comparison calculation is used to detect the amount of image movement, a plurality of points are considered to be in agreement, and the correct alignment state cannot be obtained.

In contrast, there is also a method of calculating the amount of image movement using a calculation method that is more complicated than the calculation. However, the number of pixels of the recent CCD has rapidly increased, and therefore, the amount of information of the captured image has become very large. Therefore, the complex calculation of the image data requires a very long time. When the calculation time becomes longer, the responsiveness of the corresponding shaking detection becomes worse.

In view of the above-mentioned problem, the applicant has applied for the invention of the sway correction device for the purpose of correcting the swaying correction with good responsiveness (for example, see Patent Document 2). In Patent Document 2, a histogram is generated from a reference image (reference image) and a comparison target image (scanned image), and the amount of image movement is obtained by a comparison process of a histogram.

Patent Document 2: Japanese Laid-Open Patent Publication No. 2004-343483.

However, according to the technique described in Patent Document 2, in the case where the same pattern appears in a plurality of places in the image data to be photographed, there is still a possibility of matching in accordance with the histogram. Therefore, it is difficult to know exactly what the correct alignment state is, and there is a limit to improving the detection accuracy of the image movement amount.

Further, the technique described in Patent Document 2 does not consider at all the movement of the image pattern due to the moving object in the image when the comparison between the reference image and the scanned image is performed to obtain the amount of image movement. . Therefore, if a moving object exists in the image of the object that detects the amount of movement, it is greatly affected by the movement of the moving object, and the possibility of detecting the amount of image movement is improved. Especially when detecting the shaking of the animation, there is a problem that the detection accuracy of the image moving amount becomes worse than the shaking of the detected still image.

The present invention has been made to solve the problem, and an object thereof is to improve the detection accuracy of an image movement amount.

In order to solve the above problems, the present invention is directed to processing a projection data column from image data, and performing filtering processing between the processing of the evaluation value of the image similarity according to the correlation calculation of the data column, and filtering the projection data. The column is converted into a eigenvalue data column containing a plurality of low frequency components having a wavelength above a certain length.

In another aspect of the present invention, when the projection data column is obtained, the image data is divided into a plurality of fields, and a projection data column is obtained for each segmentation field. Further, in another aspect of the present invention, when the evaluation value of the complex number is obtained, the eigenvalue data column is divided into a plurality of data columns, and the evaluation value of the complex number is obtained for each of the divided data columns.

According to the invention having the above configuration, since the wavelength of the data column to be subjected to the correlation calculation is relatively long, when the correlation value of the individual scanning positions is obtained by scanning the data sequence, the correlation value becomes large. The number of places that are considered to be consistent with the image will be reduced. Accordingly, the number of candidates to be matched according to the comparison of the images can be minimized, and the detection accuracy of the image movement amount can be improved.

According to another feature of the present invention, even if the moving object as the object of the object exists in the image of the object detecting the amount of image movement, the range of influence of the movement of the moving object on the image detection may be limited to the segmentation field. In size. For example, assume that a moving object that exists in a place in an image at a certain point in time moves to another place in the image of the next time point. At this time, if the mobile system spans different divided areas, in the respective divided areas, when the image at a certain time point is correlated with the image of the next time point, no moving image is regarded as being consistent. The situation. That is, the possibility of treating a moving object as an anastomosis according to the comparison of images can be minimized. Accordingly, even if a moving object exists in the image of the object that detects the amount of movement, it is not easily affected by the movement of the image, and the detection accuracy of the image movement amount can be improved.

Hereinafter, an embodiment of the present invention will be described based on the drawings. Fig. 1 is a functional area diagram showing an example of a configuration of an important part of the shake detecting device of the present embodiment. In addition, in the second to fourth embodiments, an area chart of the entire configuration of the digital video camera 10 to which the shake detecting device of the present embodiment is applied is applied.

As shown in Fig. 2, the digital video camera 10 is configured to include an optical system 11 including a shutter 11a, a lens 11b, and an aperture 11c, an imaging element 12 such as a CCD or a CMOS, and a programmable gain amplification. a control circuit (Programmable Gain Amplifier, PGA) 13, an A/D converter 14, a signal processing unit 15, a shake correction unit 16, and an AE/AF processing unit 17 that performs automatic exposure adjustment and automatic focus adjustment processing, And a controller 18 that controls the entirety of the digital video camera 10.

In the digital video camera 10 thus constructed, incident light to the optical system 11 is imaged on the imaging element 12. The photographic element 12 photoelectrically converts the incident incident light to produce an analog photographic signal corresponding to the incident light. The photographic signal generated here is supplied to the A/D converter 14 after being increased in Gain by the PGA 13, and converted into digital image data. In this way, the gain of the analog photography signal is increased by the PGA 13 to reduce the disadvantage of the coarsening of the Gradation caused by the quantization in the A/D converter 14.

The image data obtained by the A/D converter is supplied to the signal processing unit 15. The signal processing unit 15 includes color interpolation processing (Color Interpolation), color correction processing (Color Correction), and RGB (red, green, and blue primary color signals) to YCbCr (luminance signal, blue color difference) for the input image data. Various signal processing such as conversion processing of the signal and the red color difference signal are outputted, and the result is output to the shake correction unit 16. The sway correction unit 16 includes the sway detection device of the present embodiment configured as shown in Fig. 1, and detects sway (detection of image movement amount) and sway correction based on the brightness data received by the signal processing unit 15 Processing.

However, the digital video camera 10 can also be constructed as shown in FIG. In the example of Fig. 3, the processing sequence of the signal processing unit 15 and the shake correction unit 16 is reversed. In other words, in the example of Fig. 2, the sway correction unit 16 performs sway correction processing using the image data processed by the signal processing unit 15 for signal processing. On the other hand, in the example of Fig. 3, the sway correction unit 16 is used. The shake correction processing is performed using the image data before the signal processing by the signal processing unit 15.

Further, the digital video camera 10 can also be constructed as shown in FIG. In the example of FIG. 4, the digital video camera 10 does not include the shake correction unit 16, and an external device (for example, a personal computer) 20 other than the coefficient video camera 10 is provided with the shake correction unit 16. That is, in the example of FIG. 4, the image data generated by the signal processing unit 15 is via a communication network or? The medium or the like is input to the external device 20. Then, the shake correction processing is performed by the shake correction unit 16 provided in the external device 20.

Fig. 5 is an explanatory diagram showing an outline of the sway correction processing performed by the sway correction unit 16. In this embodiment, the motion vector detection method is used as a technique for detecting sloshing, and an electronic method is used as a technique for correcting sway. That is, as shown in Fig. 5, the photographic element 12 of the photographic area 22 which is larger than the actual output image area 21 is prepared, and a part of the photographic area 22 is cut to be used as the output image area 21. In this case, the image movement amount 23 is detected by the shake detecting device of the embodiment, and the cut image area (output image area) 21 is moved up and down and left and right corresponding to the detected image movement amount 23, thereby Get an output image with less shaking.

Fig. 6 is an explanatory diagram showing the relationship between the size of the input image and the maximum correction amount (maximum image movement amount) of the image shake. As shown in Fig. 6, the size of the input image (photographing area 22) is such that the width in the horizontal direction (the number of pixels) is W, and the height in the vertical direction (the number of pixels) is H. When the position of the output image area 21 is at the center of the photographing area 22, the maximum correction amount U of the left and right in the horizontal direction is U=W×m (m is a fixed number, for example, m=0.1). Further, the maximum amount of swaying V in the vertical direction is set to V = H × n (n is a fixed number, for example, n = 0.1).

On the other hand, if the maximum correction amount (the values of m and n) of the sway is set too large, the amount of calculation of the image movement amount detection described below increases, and the processing speed becomes slow. Conversely, if the maximum amount of correction for shaking is too small, the shake correction will become invalid. Therefore, it is preferable that the maximum correction amount of the shaking is an appropriate value which is not too large or too small. m = n = 0.1 is an example of an appropriate value.

Next, the configuration of the shake detecting device according to the present embodiment will be described based on Fig. 1 . As shown in Fig. 1, the function configuration of the shake detecting device of the present embodiment includes a projection data column creating unit 1, a filter processing unit 2, an evaluation value column creating unit 3, and a video moving amount determining unit 4. The projection data column creation unit 1 performs a certain calculation on the image data of the two-dimensional plural pixel values having the horizontal direction and the vertical direction, thereby obtaining a projection data column having a one-dimensional complex data value.

Fig. 7 and Fig. 8 are explanatory views of the processing contents of the projection data column creating unit 1. As shown in Fig. 7 and Fig. 8, in the present embodiment, a projection data column for the horizontal direction and a projection data column for the vertical direction are obtained. When the projection data column in the horizontal direction is obtained, as shown in Fig. 7, the image data is divided according to the horizontal direction (for example, divided into three), and the divided plural fields (upper field, horizontal middle field, The next field) is to obtain the projection data column separately, which is ideal.

The calculation of the projection data column in the horizontal direction is as shown in Fig. 15 (Form 1). This (1st formula) is an expression for finding a projection data in one horizontal direction of an x coordinate. That is, a plurality of pixel values in the vertical direction of an x coordinate are all added, thereby obtaining a horizontal projection data. This is done in the horizontal direction of all the x coordinates, in order to obtain the horizontal projection data column. Accordingly, the width (number of data) of the projection data column in the horizontal direction becomes W.

Further, when the projection data column in the vertical direction is obtained, as shown in Fig. 8, the image data is divided in the vertical direction (for example, divided into three), and the divided plural fields (left field, longitudinal direction) It is ideal to obtain the projection data column separately in the field and in the field. The calculation of the projection data column in the vertical direction is as shown in Fig. 16 (second equation). This (second type) is an expression for finding a projection material located in a vertical direction of a certain y coordinate. That is, a certain pixel value of a y coordinate in the horizontal direction is added, and all of them are added to obtain a vertical projection data. This is done in the vertical direction of all the y coordinates, so as to obtain the vertical projection data column. Accordingly, the width (number of data) of the data column in the vertical direction becomes H.

The filter processing unit 2 performs a predetermined filtering process on the projection data sequence obtained by the projection data column creating unit 1, thereby obtaining a plurality of feature value data columns containing low frequency components having a wavelength longer than a certain length. The filtering process is respectively for the plurality of projection data columns obtained in the divided regions (the field projection data column in the horizontal direction, the projection data column in the horizontal middle field, the projection data column in the lower region, and the vertical direction) The left field projection data column, the vertical middle field projection data column, and the right field projection data column are performed by six projection data columns.

FIG. 9 and FIG. 10 are explanatory diagrams of processing contents when the filter processing unit 2 obtains the feature value data column. As shown in Fig. 9, for the horizontal projection data column of width W (the number of data is W), a one-dimensional filter of a certain size (for example, 4 U in length) is overlaid, thereby obtaining the eigenvalue data column in the horizontal direction. . For example, this filter is set to a filter having a coefficient value of +1 in the interval of the center width of 2U and a coefficient value of -1 in the section of the width U on both sides.

The filter processing unit 2 performs scanning of the one-dimensional filter on the horizontal projection data line, and accumulates coefficient values such as +1 or -1 for the horizontal projection data column. That is, if the xth data of the horizontal projection data column is the position of interest, the data value of the section of the width 2U centered on the attention location is multiplied by 1 time, and the width U of the two sides is located The data values are multiplied by -1 times, and all are summed, and the result is set as the xth eigenvalue data. Scan the position of interest while scanning the filter in sequence.

However, since the filter having a width of 4 U cannot be covered in the vicinity of both ends of the projection data column in the horizontal direction, the width (number of data) of the horizontal characteristic value data column is W-4U+1. Moreover, the horizontal eigenvalue data column mainly has a low frequency component having a wavelength of 4 U or more.

Moreover, as shown in FIG. 10, for a vertical projection data column of a width H (the number of data is H), a one-dimensional filter of a certain size (for example, a length of 4 V) is overlaid to obtain an eigenvalue in the vertical direction. Data column. For example, this filter is set to have a coefficient value of +1 in the interval of the center width of 2V, and a filter having a coefficient value of -1 in the interval of the width V on both sides.

The filter processing unit 2 performs scanning of the one-dimensional filter on the vertical projection data line, and accumulates coefficient values such as +1 or -1 for the data column to be vertically projected. That is, if the yth data of the vertical projection data column is the attention position, the data value of the section of the width 2V centered on the attention position is multiplied by 1 time, and the width V of the both sides is located The data values are multiplied by -1 times, and all are summed, and the result is set as the yth eigenvalue data. Scan the position of interest while scanning the filter in sequence.

However, since the filter having a width of 4 V cannot be covered in the vicinity of both ends of the projection data column in the vertical direction, the width (number of data) of the vertical eigenvalue data column is H-4V+1. Further, the vertical eigenvalue data column mainly has a low frequency component having a wavelength of 4 V or more.

Although the filter shown here is only an example, only a simple numerical value such as a coefficient of -1 or +1 can be used, and the calculation can be simplified and high-speed filtering processing can be performed.

Next, the evaluation value is used as the component 3, and the feature value data column obtained by the image data of a certain time (for example, the frame t) is as early as the time when the timing is earlier (for example, the frame (for example) The image data of the t-1)) is extracted from the eigenvalue data column obtained as described above, and the extracted characteristic value data column is extracted, and the evaluation value column composed of the evaluation value of the complex number is obtained. Specifically, the scanning characteristic value data column of the scanning frame (t-1) is scanned in the range of the characteristic value data column of the frame t, and the extracted characteristic value data column and the characteristic value at each scanning position are calculated. The correlation between the data columns, in order to obtain the evaluation value of the plural.

Fig. 11 and Fig. 12 are explanatory diagrams showing the processing contents of the evaluation value column creation unit 3. The processing for obtaining the evaluation value column is performed separately for the complex eigenvalue data sequence obtained by the projection data column in each of the fields divided by the projection unit 1. Figure 11 shows the content of processing for one of the feature value data columns. As shown in Fig. 11, when the evaluation value column is obtained, it is preferable to divide the eigenvalue data column into a plurality of data columns and obtain an evaluation value column for each of the divided data columns.

In this embodiment, one of the feature value data columns in the frame t is allowed to be overlapped into a plurality of numbers (for example, divided into three). Here, it is not necessary to make them overlap. The divided feature value data columns are referred to as "divided feature value data columns".

The image data of the frame (t-1) is also subjected to the same processing to obtain a segmentation feature value data column, and the data column of the data length X is extracted from the segmentation feature value data column. For example, X data values are extracted from the central portion of the segmentation feature value data column. The extracted feature value data column is referred to as "extracted feature value data column". The data length X of the extracted feature value data column is arbitrary, but the horizontal direction can be set to X=U and the vertical direction to X=V.

On the other hand, if the length X of the extracted feature value data column is too long, it is easily affected by the moving object which is the object to be detected as the object in the image. Conversely, if the data length X is too short, the accuracy of the evaluation value column will decrease. Therefore, it is preferable to set the data length X of the extracted feature value data column to an appropriate value that is not too long but not too short. X = U (V) is an example of a good value.

Further, the data length of the divided feature value data column is as follows. When the segmentation feature value data column is created from the horizontal direction feature value data column, since the maximum correction amount of the sway in the horizontal direction is ±U (width 2U), the feature value data column is divided into the segmentation feature values. The data in the data column becomes X+2U. On the other hand, when the divided feature value data column is created from the vertical eigenvalue data column, since the maximum correction amount of the sway in the vertical direction is ±V (width 2V), the eigenvalue data column is divided into Make the data length of the segmentation feature value data column X+2V.

The evaluation value creation unit 3 performs correlation calculation between the divided feature value data column of the frame t thus created and the extracted feature value data column of the frame (t-1), thereby obtaining an evaluation value column. Specifically, while scanning the extracted feature value data column of the frame (t-1) within the range of the segmented feature value data column of the frame t, the data column and segmentation of the extracted feature value at each scanning position are calculated. The correlation between the eigenvalue data columns is used to obtain the evaluation value of the complex number.

The calculation for obtaining the evaluation value column is as shown in Fig. 17 (Form 3). Fig. 12 is an image representation of the calculation shown in this (Formula 3). In Fig. 12 and Fig. 17 (Form 3), the data value of the segmentation feature value data column is represented by M _{( j + i )} ; and the data value of the extracted feature value data column is represented by T _{( j )} . Here, j represents the position of the data in the extracted feature value data column (j=1~X); i represents the scanning position on the segmentation feature value data column of the extracted feature value data column (the horizontal direction is i=-U~+U) The vertical direction is i=-V~+V).

As shown in Fig. 12 and (Form 3), the correlation value at a scan position i is obtained by extracting the data value T _{( j )} of the feature value data column and the data value M _{( j + of the} segmentation feature value data column). The sum of the squares of the differences between _{i )} is obtained. Scanning the scanning position i in the horizontal direction -U~+U and the vertical direction -V~+V, and performing such correlation calculations in sequence. Accordingly, (2U+1) evaluation values are obtained in the horizontal direction and (2V+1) evaluation values are obtained in the vertical direction by a divided feature value data column.

Here, the horizontal eigenvalue data is listed as three, and is divided into three, so that there are nine horizontally-divided eigenvalue data columns. Since (2U+1) evaluation values are obtained from each of them, the evaluation values in the horizontal direction are all 9 × (2U + 1). The scan positions i at the time of obtaining these evaluation values are all the candidate image shift amounts in the horizontal direction. Similarly, the evaluation values in the vertical direction are all 9 × (2V + 1). The scan positions i when these evaluation values are obtained are all the candidate image shift amounts in the vertical direction.

The image movement amount determining unit 4 selects a larger evaluation value from among the plurality of evaluation values obtained by the evaluation value column forming unit 3, and determines the scanning position i when the selected evaluation value is obtained as The amount of image movement. According to (3rd formula), the smaller the evaluation value, the higher the correlation between the images, and the higher the evaluation as the candidate movement amount. Therefore, the scan position i corresponding to the smaller evaluation value is selected as the most likely amount of movement. Although the method of selecting the smallest evaluation value is also considered here, it is considered that there is more or less noise, and the evaluation value corresponding to the most likely movement amount is not limited to the minimum.

For this reason, in the present embodiment, when the amount of image movement in the horizontal direction is obtained, an evaluation value equal to or less than a certain threshold value is selected from each of the (9U+1) evaluation values for each of the nine divided feature value data columns. And extract the candidate movement amount i corresponding to it. Accordingly, the candidate movement amount i is extracted from the nine divided feature value data columns. At this time, there is a case where the candidate movement amount i of the same value is extracted from the different divided feature value data columns, and the candidate movement amount i is extracted by only one divided feature value data column. The image movement amount determination unit 4 determines the candidate movement amount i having the highest extraction frequency among the values of the candidate movement amounts i extracted from the nine divided feature value data columns, and determines the image movement amount in the final horizontal direction.

The same is true for the amount of image movement in the vertical direction. In other words, for each of the nine divided feature value data columns, an evaluation value equal to or less than a certain threshold value is selected from among (2V+1) evaluation values, and the candidate movement amount i corresponding thereto is extracted. Then, among the values of the candidate movement amounts i extracted from the nine divided feature value data columns, the candidate movement amount i having the highest extraction frequency is determined as the final horizontal image movement amount.

In the present embodiment, as described above, the projection data sequence is subjected to filtering processing, and is converted into a plurality of eigenvalue data columns having low frequency components having a wavelength longer than a certain length (for example, 4 U in the horizontal direction and 4 V in the vertical direction). As a result, the wavelength of the eigenvalue data item to be subjected to the correlation calculation becomes longer, and the appearance period of the data column having the same tendency becomes longer. Accordingly, the scan data column and the evaluation value obtained for each scan position are smaller than the threshold value, and the number of images that can be regarded as being consistent can be reduced.

Figure 13 is an image representation of the relationship between the wavelength of the horizontal eigenvalue data column and the number of coincident locations. For example, as shown in Fig. 13(a), when the wavelength of the eigenvalue data column is U, the scan amount at the time of obtaining the evaluation value as shown in Fig. 12 is 2 U, and therefore, the wavelength is 2 U at the scan amount. In the following, the data column having the same tendency will appear repeatedly in the scanning range, and it will be judged that the divided feature value data column M and the extracted feature value data column T are identical in the complex position. As shown in Figure 13(b), the same is true for the wavelength of 2U.

On the other hand, as shown in FIGS. 13(c) and (d), when the wavelength of the eigenvalue data column is longer than the scan amount 2U, the divided eigenvalue data column M and the extracted eigenvalue data column T are regarded as being identical. There is only one place. As described above, with the filtering process of the present embodiment, the number of candidates regarded as coincident in the comparison of the eigenvalue data columns can be minimized, and the detection accuracy of the image movement amount can be improved. On the other hand, in the present embodiment, a large number of low-frequency components having a wavelength of a horizontal direction characteristic value data row of 4 U or more are included in the filtering process, but this is merely an example. As can be seen from Fig. 13, as long as it is a one-dimensional filter which requires a large number of eigenvalue data lines including a low frequency component having a wavelength longer than the scanning amount 2U, it can be applied to this.

Further, in the present embodiment, not only the image data is divided into a plurality of numbers to obtain the feature value data column, but also the evaluation value column (the candidate for the image movement amount) is obtained by dividing the image data into plural numbers. Accordingly, even if the object that detects the amount of image movement, that is, the moving object that is the object to be photographed, exists in the image, the influence of the movement of the moving object on the detection of the amount of image movement can be mitigated.

For example, in Fig. 8, it is assumed that in the image of the frame (t-1), the moving object existing in the left field moves to the right field in the image of the frame t. At this time, in the left field and the right field, when the correlation between the extracted feature value data column of the frame (t-1) and the segmented feature value data column of the frame t is obtained, the movement of the image of the moving object is evaluated. The value does not become small, and it is not detected between the images before and after the time, and the scanning position (the amount of candidate image movement) that the moving object is regarded as consistent.

In addition, in the image of the frame (t-1), the moving object existing in the left field is also in the left field in the image of the frame t, and only a little moves. At this time, in the left field, as for the movement of the image of the moving object, the evaluation value becomes smaller, and the scanning position between the images before and after the time is considered to be consistent with respect to the moving object. However, in other longitudinal and right fields, between the images before and after the time, the scanning position where the moving object is regarded as consistent is not detected. That is, the range of images produced by moving objects can be limited to the left field. Further, even if the movement of the moving object is detected as the candidate image movement amount in the left field, the image movement amount determining unit 4 is detected by the movement of the moving object when the image movement amount is finally determined. The amount of candidate image movement will also be screened out due to majority decision.

As described above, by dividing the image into complex numbers, it is possible to minimize the possibility that the comparison of the images is considered to be coincident with the moving object. Accordingly, even if a moving object exists in the image of the object that detects the amount of image movement, it is less susceptible to the movement of the moving object, and the detection accuracy of the image movement amount can be improved.

Further, in the case of the present embodiment, the image is divided into plural numbers when the projection data column is obtained, and the eigenvalue data column is also divided into plural numbers when the evaluation value column is obtained. Of course, such a division is ideal, but not both must be divided.

Further, in the embodiment described in the present embodiment, after the video is divided into three parts to obtain a projection data sequence, filtering processing is performed, and then the eigenvalue data column is divided into three to obtain an evaluation value column. On the other hand, in the first place, the image is divided into nine parts to obtain a projection data sequence, and then the filtering process is performed, and the evaluation value column is obtained from the feature value data column thus obtained. However, as shown in Fig. 9 and Fig. 10, since the data length is shortened after the filtering process, it is better to filter the projection data column of the shorter data length after dividing into nine copies. The projection data column of the longer data length after being divided into three copies is subjected to filtering processing.

The above-described sway detection device of the present embodiment can be applied to, for example, a person who detects "large area movement amount" in the invention described in Patent Document 2. That is, as shown in FIG. 14, the image is multiplexed by a plurality of stages and reduced by 1/2 at a time, thereby generating image data of a plurality of levels including the input image, and reducing the image by 1/8 of the uppermost layer having the largest reduction ratio. The sway detection method of this embodiment is applied. Then, using the amount of image movement obtained in this way, the lower layer of the first level (1/4 reduction image), the lower layer of the second level (1/2 reduction image), and the lower level of the third level are sequentially obtained. The amount of image movement of the layer (input image). For the content of the detection processing of the image movement amount in the bit layer below the first level and the third level, please refer to the description of Patent Document 2.

Further, the person described in the above embodiment is an example in which the shake detecting device of the present embodiment is applied to the digital video camera 10, but the present invention is not limited thereto. For example, a mobile phone equipped with an animated photographing function can be applied as long as it has an animated photographing function. Further, although there is no animation photographing function, the shake detecting device of the present embodiment can be applied to a device that captures a plurality of images and corrects the shaking of the still image.

The shake detecting device of the present embodiment can be applied to shake correction of an animation and shake correction of a still image, and can also be applied to, for example, image stitching of a mosaic image. Here, the mosaic image refers to a wide-view image obtained by joining a plurality of images.

Further, the sway detection method of the present embodiment described above can be realized by any of a hardware configuration, a DSP, and a software. For example, the shake detecting device of the present embodiment can be implemented by being assembled into an instant processing software such as the digital video camera 10.

Further, when configured by a hardware configuration, a DSP or the like, the functional configuration of the shake detecting device described in the above embodiment can be mounted on a semiconductor wafer or a substrate module. Moreover, it is not necessary to mount the entire function of the sway detecting device to a semiconductor wafer or a substrate module, and it may be separately mounted on a plurality of wafers or a plurality of substrates.

Further, the above (1st formula) to (3rd formula) are merely examples, and are not limited thereto. For example, if the calculation of the projection data column is performed, as long as the calculation of the one-dimensional data column can be obtained from the two-dimensional image data, the calculations other than (first type) and (second type) may be used. In addition, the calculation of the evaluation value column is performed, and the calculation of the correlation between the image data before and after the time can be obtained, and the calculation other than the vertical type (3rd type) is also possible.

In addition, in the above-mentioned embodiment, only the embodiment of the present invention is embodied, and the technical scope of the present invention is not limited thereto. That is, the present invention may be embodied in various forms without departing from the spirit or its main features.

Industrial availability

The sway detecting device and method of the present invention can be used for an photographic device having a swaying correction function required to prevent image quality of a captured image caused by shaking of the photographic device.

1. . . Projection data

2. . . Filter processing unit

3. . . Evaluation value is listed as a part

4. . . Image movement amount determination unit

10. . . Digital video camera

11. . . Optical system

11a. . . shutter

11b. . . lens

11c. . . aperture

12. . . Photography component

13. . . Programmable gain amplification control circuit

14. . . A/D converter

15. . . Signal processing unit

16. . . Shake correction

17. . . AE/AF processing department

18. . . Controller

20. . . External device

twenty one. . . Output image area

twenty two. . . Photography area

twenty three. . . Image movement

Fig. 1 is a view showing a functional area of an example of a configuration of an important part of the shake detecting device of the present embodiment.

Fig. 2 is a view showing a general configuration example of a digital video camera to which the shake detecting device of the present embodiment is applied.

Fig. 3 is a view showing an overall configuration example of a digital video camera to which the shake detecting device of the present embodiment is applied.

Fig. 4 is a view showing an area of a general configuration example of a digital video camera to which the shake detecting device of the present embodiment is applied.

Fig. 5 is an explanatory diagram showing an outline of a sway correction process performed by the sway correction unit according to the embodiment.

Fig. 6 is an explanatory diagram showing the relationship between the size of the input image and the maximum correction amount (maximum image movement amount) of the image shake.

Fig. 7 is an explanatory view showing a process of creating a horizontal projection data column in the present embodiment.

Fig. 8 is an explanatory view showing a process of creating a projection data column in the vertical direction in the embodiment.

Fig. 9 is an explanatory diagram showing a process of creating a horizontal direction feature value data column in the present embodiment.

Fig. 10 is an explanatory view showing a process of creating a vertical direction feature value data column in the present embodiment.

Fig. 11 is an explanatory diagram showing a process of creating an evaluation value column in the present embodiment.

Fig. 12 is an explanatory view showing a process of creating an evaluation value column in the present embodiment.

Fig. 13 is an explanatory diagram showing the relationship between the wavelength of the feature value data column and the number of positions in which the image is matched in the present embodiment.

Fig. 14 is an explanatory diagram showing an application example of the shake detecting method in the present embodiment.

Fig. 15 is a diagram showing the calculation formula of the projection data column in the horizontal direction.

Fig. 16 is a diagram showing the calculation formula of the projection data column in the vertical direction.

Fig. 17 is a diagram showing the calculation formula of the evaluation value column.

1. . . Projection data

2. . . Filter processing unit

3. . . Evaluation value is listed as a part

4. . . Image movement amount determination unit

Claims (7)

A sway detecting device characterized by comprising: a projection data column creating portion for performing a certain calculation on image data having a plurality of pixel values in two dimensions in a horizontal direction and a vertical direction, thereby obtaining a one-dimensional a projection data column of the plurality of data values; and a filter processing unit that performs filtering processing on the projection data column to obtain a eigenvalue data column including a plurality of low frequency components having a certain wavelength or more; and the evaluation value is used as a forming unit For the image data at a certain point in time, the feature value data column obtained by the projection data as the formation portion and the filter processing portion, and the image data from the time point earlier than the above-mentioned time point, according to the projection data The feature value data column obtained by the column processing unit and the filter processing unit extracts a extracted feature value data column, and scans the extracted feature value data column within the range of the feature value data column, and simultaneously calculates each scan Corresponding to the above-mentioned extracted feature value data column and the above-mentioned feature value data column, thereby obtaining the evaluation value of the complex number; and image shifting Amount determining portion, a large correlation value selected by the evaluation of the complex among the evaluation values, and an evaluation value in order to achieve the scanning position is selected as the basis, to determine an amount of image movement. For example, the sway detection device described in item 1 of the patent application scope, The projection data column creation unit divides the image data into a plurality of fields, and obtains the projection data sequence for each of the divided fields; and the processing of the filter processing unit and the evaluation value column creation unit is The plurality of projection data columns obtained by dividing the field are separately performed. The sway detection device according to the first or second aspect of the invention, wherein the evaluation value column creation unit divides the eigenvalue data column into a plurality of data columns, and obtains the above-described divided data columns. The plural evaluation value. The sway detection device according to the first or second aspect of the invention, wherein the filter processing unit is configured to obtain a characteristic value data string including a plurality of low frequency components having a wavelength longer than the predetermined length Dimensional filter, and the coefficient value is only one-dimensional filter composed of +1 and -1 to filter the above-mentioned projection data column. A sway detection method characterized by comprising: a first step of performing a certain calculation on image data having a plurality of pixel values in two dimensions in a horizontal direction and a vertical direction, thereby obtaining a plurality of data in one dimension a projection data column of values; and a second step of filtering the projection data column to obtain a eigenvalue data column containing a plurality of low frequency components having a wavelength greater than a certain length; and the third step, using the time The eigenvalue data column obtained from the image data of the point, and Extracting a extracted feature value data column from the feature value data obtained from the image data at a time point earlier than the above-mentioned time point, and scanning the extracted feature value within the range of the feature value data column The data column simultaneously calculates the correlation between the extracted feature value data column at each scanning position and the above-mentioned feature value data column, thereby obtaining the evaluation value of the complex number; and the fourth step, selecting a relevant large evaluation from the above plurality of evaluation values The value is determined based on the scanning position at which the selected evaluation value is obtained. The sway detection method according to the fifth aspect of the invention, wherein the first step is to divide the image data into a plurality of fields, and obtain the projection data column for each division field; and the second step and The processing of the third step described above is performed for each of the plurality of projection data sequences obtained for each of the above-described divided fields. The method for detecting a sway detected in the fifth or sixth aspect of the patent application, wherein the third step is to divide the eigenvalue data column into a plurality of data columns, and obtain the plural number for each divided data column. Evaluation value.