JP6026752B2 - Object detection from image profiles - Google Patents

Description

(Related application)
This application is one of three applications filed at the same time by the same inventor and assignee with the following titles: “FAST ROTATION ESTIMATION”, “Local Image rotation derived from motion estimation (IMAGE ROTATION FROM LOCAL MOTION ESTIMATES), and "Object detection derived from image profile (OBJECT DETECTION FROM IMAGE PROFILES)".

(Background)
It would be desirable to have an efficient solution for finding rotational changes between consecutive video frames or between image pairs. Such can be applied to obfuscate measurement and compensation, panorama generation, image stabilization and / or object tracking to list a few of the many examples. Alternative solutions based on mutual information include Hough transform, Radon transform, Fourier transform, and polar transform. However, since these alternative solutions are resource consuming, it is desirable to have a more efficient solution.

  A common problem in panorama generation is, for example, the occurrence of rotation between two frames that are to be joined or connected. An alternative is to resample the image using high precision rotation estimation prior to splicing, but this is computationally expensive and requires a large amount of memory. It would be desirable to have an application for panorama generation with a camera that detects and / or corrects rotation before splicing or joining adjacent image segments.

  Image rotation usually involves extra computation and processing time and is difficult to succeed in real time. Image resampling that compensates for rotation before the images are spliced is slow and computationally expensive.

A plot based on a 256 × 256 image showing the observation that the slope of the integral projection vector obtained from a line passing through the center of the binary image at different angles has one higher positive spike and one higher negative spike FIG. FIG. 6 is a plot showing energy spikes concentrated with corresponding coefficients. FIG. 7 is a plot diagram showing the sum of differences of convolved absolute vectors for different variable vector lengths. FIG. 6 is a plot showing rotation angles estimated with power values as a function of vector length for a 512 × 512 image. FIG. 6 is a plot showing rotation calculated and estimated for a simulated 3 degree rotation. FIG. 6 is a plot showing a simulated example where the rotation angle is changed from 0.2 degrees to 5 degrees in steps of 0.2 degrees. It is a figure which shows roughly the relative direction of a local window when a camera rotates. FIG. 6 shows an example of angle estimation that can be performed for points that take into account calculated horizontal and vertical permutations. FIG. 4 shows a profile difference plot against an aligned profile used to detect object motion. FIG. 6 shows a plot of an aligned and moving image profile used for object detection and / or tracking.

(Detailed description of embodiment)
Techniques are provided for performing a method for detecting rotation from matching corner regions of an acquired image within an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the frame of the sequence of image frames, a global XY alignment is performed on the first frame and the second frame. A local XY alignment is determined in at least two matching corner regions of the aligned image frame pair. Based on the difference between the local XY alignments, the global rotation of the second frame relative to the first frame is determined. Further actions are performed based on the determined global rotation.

  Another technique is provided for performing a method for detecting rotation from a matching region of an acquired image in an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the frame of the image sequence, a global XY alignment of the first frame and the second frame is performed. A local XY alignment is determined in at least two matching regions of the aligned image pair. The difference between delta X and delta Y is determined between at least two matching regions. From the difference between delta X and delta Y, a global interframe rotation is calculated. Further operations are performed based on the calculated global inter-frame rotation.

  The technique is further provided for performing a method for cross-correlating integral projection vectors in an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the sequence of image frames, horizontal and vertical integral projection vector gradients for each of the first and second frames are determined. The integral projection vector gradient is normalized. The positions of the major maximum and minimum peaks of the integral projection vector gradient are determined. Based on the normalized distance between the major maximum and minimum peaks, a global image rotation is determined. Further actions are performed based on the determined global image rotation.

  Another technique is provided for performing a method of cross-correlating integral projection vectors within an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the sequence of image frames, horizontal and vertical integral projection vectors are determined for each of the first and second frames. The first integral projection vector is continuously convolved with a variable length vector. The second integral projection vector is convolved with a fixed length vector. An absolute sum of differences between the filtered vectors is determined based on the convolution of the first and second integral projection vectors. A global rotation of the second frame relative to the first frame is determined based on the length of the variable vector N leading to the minimum absolute sum of the differences. Further actions are performed based on the determined global rotation.

  The technique is further provided for performing a method for determining a moving object in an integral projection vector in an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the frame of the image sequence, a global XY alignment of the first frame and the second frame is performed. At least one portion of each of the X and Y integral projection vectors for which the aligned global vector manifests significant local differences is determined. Based on the X and Y positions of at least one portion of the X and Y integral projection vectors, the position, relative velocity, and / or approximate region of at least one moving object is determined within the sequence of image frames. The Further actions are performed based on the determined position, relative velocity and / or approximate area of the at least one moving object.

  Another technique is provided for performing a method for determining a moving object in an integral projection vector in an image acquisition device. A sequence of image frames is acquired. The degree of rotation between frames is determined. Following the frame of the image sequence, a global alignment of the first frame and the second frame is performed. At least one portion of the integral projection vector is determined in which the aligned global vector manifests significant local differences. Based on the position of each of the at least one portion of the integral projection vector, an approximate size and relative velocity of at least one moving object in the sequence of image frames is determined. Further actions are performed based on the determined approximate size and / or relative velocity of at least one moving object.

  In any of these techniques, a further action is to concatenate adjacent frames of the panoramic image; and / or predict the position of the tracked object in a later frame; and / or Providing an image with reduced blur than at least one of the first and second frames; and / or providing a more stable video sequence by removing one or more unwanted motion artifacts. be able to.

  In any of these techniques, the difference in delta X and delta Y between at least two matching corner regions can be determined. The global rotation determination can be further based on the difference between delta X and delta Y.

  In any of these techniques, horizontal and vertical integral projection vector gradients can be determined for each of the first and second frames following the image frames in the sequence. The integral projection vector gradient can be normalized. The positions of the major maximum and minimum peaks of the integral projection vector gradient can be determined. The global rotation determination can further be based on the normalized distance between the major maximum and minimum peaks.

  In any of these techniques, horizontal and vertical integral projection vectors can be determined for each of the first and second frames following the sequence of image frames. The first integral projection vector can be continuously convolved with a variable length vector. The second integral projection vector can be convolved with a fixed length vector. An absolute sum of the differences can be determined between the filtered vectors based on the convolution of the first and second integral projection vectors. The determination of the global rotation can further be based on the length of the variable vector N that leads to the minimum of the absolute sum of the differences.

  In any of these techniques, a global XY alignment of the first frame and the second frame can be performed following the frame of the sequence of image frames. In at least two matched corner regions of the aligned image pair, a local XY alignment can be determined. Based on the difference between the local XY alignments, a global rotation for the second image frame relative to the first image frame can be determined. Further actions can further be based on the determined global rotation.

  The technology, among other applications of these advantageous technologies, is continuous video frames (see, for example, US Patent Specification US 2009/03030334, incorporated by reference), or continuous, according to the embodiments described herein. Other image pairs such as preview frames (see, eg, US patent specification US 2010/0060727 incorporated by reference), adjacent image segments of panoramic images (eg, US patent application Ser. No. 12/636 incorporated by reference). , 608), a pair of low light and normal light images (eg incorporated by reference) that are combined to produce a sharper and better exposed image on a handheld or other portable camera Between US patent specification US 2009/167893) It can be efficiently performed at high speed for a typical rotation values appearing. These techniques can be accomplished with advantageous results without using multiple image rotations at different angles.

  A method of estimating rotation between images or frames after being aligned horizontally and vertically is also provided. This technique uses an integral projection technique, and in some examples, with respect to successive video frames, or differently exposed image pairs, or spectrally different image pairs, or image pairs acquired with different cameras. Can be applied. One integral projection (IP) vector can be continuously convolved with a variable length vector, and the second IP vector can be convolved with a fixed vector. The sum of the absolute differences of these filtered vectors is calculated, and the length of the variable vector that leads to the minimum over the selected range is made by the rotation between the two images or frames Enables estimation of.

  The use of convolving one integral projection vector with a variable length vector allows estimation of rotation between images or successive frames compared to convolving another integral projection vector with a fixed vector. Refinement of the rotation estimation is provided by combining the results of the movement estimation in the local window based on the integral projection method and the search method using the result of the IP vector convolution comparison.

Fast rotation estimation method

A further technique is provided for estimating the rotation between two images. The two images can be two consecutive frames or image pairs and need not necessarily be acquired with the same parameters or sensors. Some examples of such image pairs have been described above. In one embodiment, the rotation estimation can be performed on one channel from each image. For example, a Y channel, a gray level image, a green channel, and the like may be selected. The horizontal and vertical integral projection vectors are then obtained by thresholding the image (eg, the median or other average value of one image, or a smaller or larger value depending on the application). In this embodiment, the calculation is performed for pixels from both binary images. Some rotation estimation methods use a line detection procedure (eg, “Orientation estimation combining visual and gyroscopic measurements” by JONATA BLOMSTER, incorporated by reference, Master's degree project, Stockholm, Sweden, April 2006. 06, XR-EE-SB 2006: 012). Such a line is a common feature in most images. If applied on successive frames, the sensed lines can help to estimate the rotation between frames by using their orientation.

  Most rotation estimation techniques use complex Hough transforms (eg, “Image Registration Method: Survey” by B. ZITVA, J. FLUSSER, incorporated by reference, Image and Vision Computing 21 (2003). ) Pages 977-1000. In the image gradient region, the line segment indicates a change in luminance level. These changes can be captured in the binary image by using appropriate thresholds.

  The displacement per frame is compared by comparing the horizontal and vertical integral projection (IP) vectors of the binary image (eg, “high dynamics derived from manual manipulation imaging by GREG WARD, incorporated by reference. "Fast and robust image registration for compositing range photos", see Graphic Tool Journal, 2003, 8 (2): 17-30) or comparing them with gradient images (eg see both) US patent application Ser. Nos. 12 / 879,003 and 12 / 636,629, incorporated by reference). A vertical IP vector is obtained by the sum of all the rows of the image, while a horizontal IP vector is obtained by the sum of all the columns. Rotating the sensed line at different angles develops its contribution in the IP vector differently.

  The technique according to an advantageous embodiment is that the gradient of the integral projection vector obtained from lines passing through the center of the binary image at different angles has one higher positive spike and one higher negative spike. Is used. FIG. 1 shows an example for a 256 × 256 image. The amplitude of the spikes decreases with the rotation angle by a normalization step using a total projection of non-null binary pixels. Furthermore, the distance from the center position of the vector increases with the rotation angle and depends on the image size.

Techniques according to certain embodiments ignore ripples between spikes and outside them. They are set to zero in certain embodiments by imposing a threshold that depends on the maximum vector value (eg, one tenth thereof). The spike energy is intensified by a corresponding factor (see FIG. 2). As the inventor made, an approximation can be made for a binary image with less noise at different angles and image sizes. If the image has C rows, the radians rotation is designated as “a” while using a horizontal projection profile, and we have N = [a × C]. The absolute standardized amplitude of both of the corresponding spikes is
1 / (N + 1)
Where N is the distance between significant spikes and [x] is the integer part of x.

  In one embodiment, the integral projection (IP) vector (or profile) is a variable length vector [-100. . . 01] (the number of N, zero is variable) is continuously convolved, and its output is divided by N + 1 (ie, the values corresponding to the different angle values as shown in FIG. 2). In this embodiment, the second IP vector is convolved with a fixed vector [−1 1] (that is, corresponding to 0 degrees in FIG. 2). The sum of the absolute differences of these filtered vectors is calculated, and the length of the variable vector N that leads to the minimum over the selected range is related to the rotation between the two images or frames (FIG. 2). reference).

  Referring now to FIGS. 3a-3b, a minimum is obtained for N = 29 hitting a vector (see FIG. 3a) corresponding to 3.2 degrees for a 512 × 512 image (see FIG. 3b). . The estimated rotation increases somewhat linearly with the filter length for a typical rotation value and is inversely proportional to the image size. Convolution calculations advantageously include operations that are not resource intensive. The sum of absolute differences can be exchanged for the sum of squared differences. Usually it obtains curves with deeper local minima, but it has an increased number of complexity. Furthermore, both horizontal and vertical projection profiles can be used for better accuracy. In this case, the sum of the minimum of the horizontal and vertical sums of absolute differences can be used in certain embodiments to indicate an appropriate filter length. Subpixel registration techniques can also be used in certain embodiments as well to improve the accuracy of rotation estimation.

  To reduce the computational complexity, the profile size can be arbitrarily or selectively reduced in certain embodiments. The length of the profile at maximum resolution can also be arbitrarily or selectively clipped. In a further embodiment, a descent algorithm can be used that includes not performing a full search for all investigated rotation values to reduce the number of calculations. Local minima can be avoided by calculating the error with different interval lengths. The section length can be reduced as long as it is sufficiently small. At each step, the rotation with the smallest error can be selected as the center point of the next interval of the three rotation values. FIG. 4 is a plot showing the estimated rotation calculated for another simulated 3 degree rotation. The calculated value (red circle) is a subset of possible rotation angles.

  The range of the rotation value depends on the image size. In the 512 × 512 image example, for a 5 degree angle, the filter length included in certain embodiments corresponds to about 8% of the image size. If the filter length is more than one fifth of the image size, lower accuracy is expected. The accuracy depends on the details of the image. FIG. 5 shows a simulated example where the rotation angle is changed from 0.2 to 5 degrees in steps of 0.2 degrees. The absolute maximum rotation estimation error between the true rotation and the estimated rotation is 0.2. However, this error can be several times higher for some rotation values in the case of a noisy or blurred binary map with missing details. For example, it is common to have several blurred frames in a video sequence. In these cases, the rotation estimate may not be as reliable. Thus, in certain embodiments, advantageous focus measurements can be performed on both consecutive frames (eg, “Estimation of focus measurements in multi-focus image fusion” by WEI HUANG, ZHONGLIANG JING, incorporated by reference, (See Pattern Recognition Letters, March 2007, v.28 n.4, p.493-500). Rotation estimation can be selectively performed only if there is a sufficiently small difference between focus measurements (eg, less than 20%).

  In certain embodiments, it is determined whether the rotation is less than a certain value. The exact sign of the rotation may not be determined or determined by itself as not achieving a large result unless a very small rotation is detected. For example, it is desirable to know the evidence only when a very small rotation is detected to join images or take a picture for panoramic photography. In other applications, it is desirable to know the exact signature of the rotation for a photograph with less detail and important absolute rotation values.

In a non-limiting example, accurate evidence can be obtained in the following manner using the available images. In certain embodiments, the second binary image is rotated back at the estimated angle and its vertical and horizontal integral projection vectors are calculated. A cosine similarity factor between the horizontal integral projection of the first binary image and the horizontal integral projection of the rotated second binary image is calculated (eg, incorporated by reference, http: // /En.wikipedia.org/wiki/Cosine_similarity). If A and B are the mentioned vectors, the cosine similarity (q) is expressed as follows using the dot product and magnitude:

  If this value is less than a threshold (eg 0.999), the trace of rotation is changed, otherwise it remains the same.

  Alternatively, an indication of the angle of rotation can be obtained from the motion calculated in the window for global motion. These displacements of movement can be obtained by the use of known methods, such as the integral projection method described in the same assignee application including US 2008/03097769, which is incorporated by reference. Their signature determines the rotation signature. The simulated differential sign rotation change leads to be an indication of a change in horizontal and vertical displacement, for example as shown in FIG.

  In other embodiments, variable length convolution comparison is applied for integral projection vectors obtained from differently exposed image pairs, eg, visible / near infrared image pairs or pairs of images exposed at different time periods. (See, eg, US patent application Ser. No. 12 / 941,983). The proposed method based on the integral projection vector can be used, for example, to improve the rotation estimate obtained when the computation is performed on a local window, as in FIG. The above assumption is that the center of rotation is the center of the image or frame. However, the center of rotation can be arranged differently (eg, closer to the image center or closer to other corner points). Angle estimation can be made for points that take into account the calculated horizontal and vertical displacements. In certain embodiments, it is assumed that these displacement estimates are accurate and the frames are aligned.

ここで、ｂ及びａは、それぞれ、一のフレーム（ｘｉ，ｙｉ）及び他のフレーム（ｘｒ，ｙｒ）でのフレーム中心についての隅ウィンドウ中心画素によって形成された角度である（図７）。（ｘｒ，ｙｒ）の値は、（ｘｉ，ｙｉ）及び、局所の水平・垂直の変位を計算して得られる。かかる地点に対して対応する回転値は、ベクトルを形成することができ、回転の中心が画像中心に対して一致又は近接していれば、近い値を有しなければならない。そうでなければ、回転中心は、最小値に推定された角度ベクトル変化を備えた点として選択されることができる。 FIG. 7 shows an example in which the origin (0, 0) can be arranged at an arbitrary point. The rotation r is obtained by the following formula:

Here, b and a are angles formed by the corner window center pixels with respect to the frame center in one frame (xi, yi) and the other frame (xr, yr), respectively (FIG. 7). The value of (xr, yr) is obtained by calculating (xi, yi) and the local horizontal / vertical displacement. The corresponding rotation value for such a point can form a vector and should have a close value if the center of rotation is coincident or close to the image center. Otherwise, the center of rotation can be selected as the point with the angle vector change estimated to the minimum value.

  The average of all four calculated rotation values can be used as an initial approximation. An advantageous integral projection method according to certain embodiments uses a rotation value that is less and closer to such initial approximation to improve that value (see, eg, FIG. 4). The calculated center and rotation values are used for rotation compensation in the registration process. A binary image can be obtained by imposing a threshold after applying an edge detection procedure to the original image.

Image rotation derived from local motion estimation for panorama joint (PANOMALA STITCHING)

Particular embodiments are based on motion estimation used for panorama generation and / or image stabilization. These use the motion measured in four (4) small windows to give an indication of the amount of rotation in the current frame. If the amount of rotation does not reach the specified value, the panoramic image can be captured. That is, in a specific embodiment, when the amount of rotation is too large, it is determined that a panoramic image cannot be captured. Using this condition, the registration and joining of captured panoramic images is advantageously viewed and feels very natural.

  In certain embodiments, the low complexity extension of panorama generation techniques has been described in US patent application Ser. No. 12 / 636,629, incorporated by reference. The output of the motion estimation is used to measure rotation and then used to capture an image for splicing when the rotation is below a minimum value and / or a defined threshold. This avoids heavier computations involving rotations that exceed a threshold and / or are not at a minimum, and provide a panoramic image that does not include a minimum amount of rotation or a minimum amount of rotation.

  Motion estimation that can be used advantageously in panorama generation and video image stabilization is described in US Pat. Nos. 7,773,118, 7,697,778, which is assigned to the same applicant and is incorporated herein by reference. 7,676,108, 7,660,478, 7,639,889, 7,636,486, and / or 7,639,888, and / or US published patent application US2010- No. 0329582, US2010-0328472, US2010-0201826, US2010-0201827, US2009-0179999, US2009-0309769, US2010-0303343, US2009-0167893, US2009-0080796, US2007-0296833 021 581 and / or US2008-0309770 and / or U.S. Patent Application Nos. 12 / 956,904, 12 / 941,983, 12 / 820,086, and / or 12 / 820,034. It can be applied according to the specific description found anywhere in the specification. The basis of many of these existing patents and patent applications is the measurement of motion between successive image frames. In video image stabilization, motion is typically measured between all successive image frames, known as global motion estimation, and moreover used to check local object movement. Measured between four small (4) windows.

  In certain embodiments, the difference between the local motion estimate and the global value is used to measure the rotation between image frames. In the absence of rotation between image frames, the local motion estimation should usually be the same as the global value. In the presence of rotation, the local window records different movements that arise from relative movement. For example, under counterclockwise rotation, the two windows on the left will have a different relative movement compared to the two windows on the right. The two windows on the left will report moving down with a lower relative motion value than the right window reporting a higher movement value as they move up.

  Similar relative motion occurs horizontally where the two top windows and the two bottom windows experience diametrically opposite relative motion. In certain embodiments, the difference between the measured motions for the individual local windows at x and y is found to determine the rotation angle. Subsequently, the rotation angle is calculated using the distance of the center of each local window from the center of the entire image frame. The motion measured from all image frames is obtained because it is concentrated at the center of the image. Thus, the distance to the center of the local window relative to the center of all images, along with the relative motion, can be used to determine the angle of motion.

As an example, ydisp [0] can represent global movement in the vertical direction and ydisp [1] can represent movement from one of the windows. Then, the movement by rotation is
ydispRot [1] = ydisp [0] −ydisp [1]. The following tangent function is used for subtraction to remove the moving element and the rotation angle.
Roty [1] = arctan (ydispRot [1] / distanceY)
Here, distanceY is the horizontal distance from the image center with respect to the center of the local window. The above calculations can be performed for all frames and all windows. The calculation results can be averaged to provide an indication of the amount of rotation in angles.

  Thereafter, for example, during panorama capture, the algorithm may be configured to acquire a new image at any time for registration if the image is overlapped by a desired amount (eg, 20% to 40%). it can. Subsequently, in certain embodiments, a new image will be captured for registration if the overlap is within the defined region and the rotation is below the defined angle. If the algorithm does not meet the criteria before the minimum overlap is reached, the image may be present in the particular embodiment captured, as was the case without checking for rotation. This captures the image at a low rotation prior to splicing, creating a better panorama without unnecessarily complicated rotational measurements and resampling, while making the splicing operation more efficient Give an opportunity. In the technique according to a particular embodiment, the calculated motion estimation is used to perform a lightweight analysis of the rotation that can improve the joint and panorama.

Object detection from image profiles

By using an image profile generated for panorama generation or video image stabilization, moving objects are detected according to a particular embodiment in the image frame. This is done by aligning the image profiles after the initial motion estimate has been evaluated. The absolute difference between profiles will subsequently be due to movement from the object or subject or region of interest.

  If the two profiles are realigned, large differences in the profiles will typically correspond to the movement of the object. The plots shown in FIGS. 8 and 9 illustrate the realigned profile after the interframe shift is measured. The portion surrounded by the rectangle in FIG. 9 emphasizes the region where the profile has the highest difference. When the difference between these profiles is plotted, the location of the peak helps identify moving objects. This position can then be used to point to the area of the moving object, or it can be used with other object detection processes to monitor object movement. The result of obtaining the difference can be typed as shown in FIGS. If the object movement is smaller, the profile difference is more noisy, so the position of the largest profile difference may tend to give unreliable results.

  For each frame, the result of the thresholding operation on the difference profile is stored according to a particular embodiment. If the profile does not have a strong peak, thresholding can be reverted to an all-zero result in certain embodiments, in which case the rejected position is assigned a previous reliable result. be able to. If the object moves actively, the profile difference tends to be high and the position of the moving object is easier to detect.

  The functionality of the panorama and video image stabilization profile can be used to detect moving objects as described above. This can assist panoramic joints to avoid joints through objects / subjects, and can also be useful alone for scene analysis applications in quickly detecting the position of moving objects. .

  While typical drawings and specific embodiments of the present invention have been described and illustrated, it should be understood that the scope of the invention is not limited to the specific embodiments discussed. Accordingly, the embodiments are to be regarded as illustrative rather than restrictive, and modifications can be made from these embodiments by those skilled in the art without departing from the scope of the invention. Should be understood.

  In addition, in the method that can be performed here according to the preferred embodiment and described above, the operations have been described in a selected letterpress order. However, such an order is selected and ordered for typographic convenience, unless a special order is clearly described or a special order may be deemed necessary by one skilled in the art. It is not intended to imply any particular order for performing the operations.

  Further, all references cited above and below, as well as the background section, the summary section of the invention, the summary section and the brief description section of the drawings, all refer to alternative embodiments by reference, It is incorporated into the detailed description of the preferred embodiments.

The following documents are incorporated by reference as indicating features that may be implemented in alternative embodiments:
U.S. Patent Nos. 7,639,888, 7,636,486, 7,639,889, 7,660,478, 7,773,118, and 7,864,990 , And 7,620,218; and
US Published Patent Applications US2008 / 0219581, US2007 / 0296833, US2008 / 0309769, US2010 / 0238309, US2009 / 0167893, US2008 / 0309770, US2009 / 0080796, US2009 / 0303433, US2009 / 0179999, US2010 / 0329582, US2010 / 0328472, US2010 / 0201826, US2008 / 0231713, US2008 / 0309770, US2009 / 0263022, and US2009 / 0303342; and
U.S. Patent Application Nos. 12 / 572,930, 12 / 636,608, 12 / 941,983, 12 / 944,701, and 12 / 879,003; and
B. ZITVA and J.I. “Image Registration Method: Survey” by FLUSSER, Image and Vision Computing 21 (2003), pages 977-1000 KUGLIN CD And HINES DC “Phase Correlation Image Alignment Method”, 1975, IEEE, Cybernetics and Society International Conference, In Proc. ;
M.M. MA, A.A. VAN GENDEREN, P.M. "Sign bit-only phase normalization for rotation and scale invariant template matching" by BEUKELMAN, 16th Annual Workshop on Circuits, Systems and Signal Processing, ProRisc 2005 (pages 641-646);
CHEN TING's “Video Stabilization Algorithm Using Block-Based Parameter Motion Models”, Stanford University (CA), EE392J Project Report
G. S. PEAKE and T.E. N. TAN “General Algorithm for Document Skew Angle Estimation”, IEEE International Conference Image Process. 2 (1997) 230-233;
GREG WARD "Fast and robust image registration for compositing high dynamic range photos derived from manual operation photography", Graphic Tool Journal, 2003, 8 (2): 17-30;
See Wikipedia: Cosine similarity,
URL: http://en.wikipedia.org/wiki/Cosine_similarity;
JONATA BLOMSTER, "Orientation Estimation Combining Visual and Gyroscopic Measurements", MSc Project, Stockholm, Sweden, 04/06/2006, XR-EE-SB 2006: 012;
YI WAN and NING WEI, "Fast Algorithm for Recognizing Transformed, Rotated, Reflected, and Scaled Objects Only From Their Projection", IEEE Signal Processing Letters, January 2010, vol. 17, no. 1, pages 71-74; and
“Estimation of focus measurement in multi-focus image fusion” by WEI HUANG and ZHONGLIANG JING, Pattern Recognition Letters, March 2007, v. 28 n. 4, p. 493-500.

Claims (16)

A method for determining whether a moving object is detected in a digital image frame within an image acquisition device, comprising:
Obtaining a sequence of image frames;
A global vector aligned by performing a global XY alignment of a first frame and a second frame in the sequence of image frames (the second frame follows the first frame in the sequence of image frames); Generating,
Determining at least one portion of each of the X integral projection vector and the Y integral projection vector;
The position, relative velocity, approximate region, or these of at least one moving object in the sequence of image frames based on at least part of the X and Y positions of at least one part of the X and Y integral projection vectors Determining a combination of
Determining the degree of rotation between the frames; and
Performing further actions based on the determined position, relative velocity, or approximate region of at least one moving object, or a combination thereof ;
further,
Determining horizontal and vertical integral projection vector gradients for the first frame and the second frame;
Normalizing the horizontal and vertical integral projection vector gradients;
Determining the positions of major maximum and minimum peaks of the horizontal and vertical integral projection vector gradients; and
Determining the global rotation based on a normalized distance between major maximum and minimum peaks, wherein the further action is based on the determined global rotation . The further operations include concatenating adjacent frames of the panoramic image, predicting the position of the tracked object in a later frame, and blurring reduced more than at least one of the first frame and the second frame. The method of claim 1, comprising: providing an image, providing a more stable video sequence by removing one or more unwanted motion artifacts, or a combination thereof. Determining a delta X and delta Y difference between at least two matching corner regions of the first frame and the second frame of the sequence of image frames; and
Determining a global rotation based on the difference between the delta X and delta Y;
Further including
The method of claim 1, wherein the further action is based on the determined global rotation. Determining horizontal and vertical integral projection vectors for the first frame and the second frame;
Continuously convolving the first integral projection vector with a variable length vector to generate a first filtered vector;
Convolving the second integral projection vector with a fixed length vector to produce a second filtered vector;
Determining a sum of differences between the first and second filtered vectors; and
Determining a global rotation based on the length of the variable vector N leading to the minimum of the difference sum;
Further including
The method of claim 1, wherein the further action is also based on the determined global rotation. Determining local XY alignment in at least two matched corner regions of the aligned image pair; and
Determining a global rotation of the second frame relative to the first frame based on a difference between the local XY alignments;
The method of claim 1, wherein the further action is also based on the determined global rotation. A method for determining whether a moving object is detected in a digital image frame within an image acquisition device, comprising:
Obtaining a sequence of image frames;
A global alignment of the first frame and the second frame in the sequence of image frames (the second frame follows the first frame in the sequence of image frames) is performed to obtain an aligned global vector Generating,
Determining at least one portion of the integral projection vector;
Determining an approximate size and / or relative velocity of at least one moving object in the sequence of image frames based on at least a portion of the position of each of the at least one portion of the integral projection vector;
Determining the degree of rotation between the frames; and
Performing further actions based on the determined approximate size and / or relative velocity of at least one moving object ;
further,
Determining horizontal and vertical integral projection vector gradients for the first frame and the second frame;
Normalizing the horizontal and vertical integral projection vector gradients;
Determining the positions of major maximum and minimum peaks of the horizontal and vertical integral projection vector gradients; and
Determining the global rotation based on a normalized distance between major maximum and minimum peaks, whereby the further operation is also based on the determined global rotation . The further operations include concatenating adjacent frames of the panoramic image, predicting the position of the tracked object in a later frame, and blurring reduced more than at least one of the first frame and the second frame. 7. The method of claim 6 , comprising: providing an image at or providing a more stable video sequence by removing one or more unwanted motion artifacts, or a combination thereof. Determining a delta X and delta Y difference between at least two matching corner regions of the first frame and the second frame of the sequence of image frames; and
Determining a global rotation based on the difference between the delta X and delta Y;
Further including
The method of claim 6, wherein the further action is also based on the determined global rotation. Determining horizontal and vertical integral projection vectors for the first frame and the second frame;
Continuously convolving the first integral projection vector with a variable length vector to generate a first filtered vector;
Convolving the second integral projection vector with a fixed length vector to produce a second filtered vector;
Determining a sum of differences between the first and second filtered vectors; and
Determining a global rotation based on the length of the variable vector N leading to the minimum of the difference sum;
Further including
The method of claim 6 , wherein the further action is also based on the determined global rotation. Performing a global XY alignment of the first frame and the second frame;
Determining local XY alignment in at least two matched corner regions of the aligned image pair; and
Determining a global rotation of a second frame relative to the first frame based on a difference between the local XY alignments;
Further including
The method of claim 6 , wherein the further action is also based on the determined global rotation. Lenses, shutters and image sensors to obtain digital images;
A processor; and
A memory having embedded code for programming the processor to perform a method for determining whether a moving object is detected in a digital image frame;
The method
Obtaining a sequence of image frames;
A global vector aligned by performing a global XY alignment of a first frame and a second frame in the sequence of image frames (the second frame follows the first frame in the sequence of image frames); Generating,
Determining at least one portion of the integral projection vector; and
Determining the size and / or relative velocity of at least one moving object in the sequence of image frames based on at least a portion of the position of each of the at least one portion of the integral projection vector;
Determining the degree of rotation between the frames; and
Performing further actions based on the determined approximate size and / or relative velocity of at least one moving object ;
further,
Determining horizontal and vertical integral projection vector gradients for the first frame and the second frame;
Normalizing the horizontal and vertical integral projection vector gradients;
Determining the positions of major maximum and minimum peaks of the horizontal and vertical integral projection vector gradients; and
Determining the global rotation based on a normalized distance between major maximum and minimum peaks, wherein the further action is based on the determined global rotation . The further operations include concatenating adjacent frames of the panoramic image, predicting the position of the tracked object in a later frame, and blurring reduced more than at least one of the first frame and the second frame. 12. An image acquisition device according to claim 11 , comprising providing an image, providing a more stable video sequence by removing one or more unwanted motion artifacts, or a combination thereof. Determining a delta X and delta Y difference between at least two matching corner regions of the first frame and the second frame of the sequence of image frames; and
Determining a global rotation based on the difference between the delta X and delta Y;
Further including
The further work, the image acquisition apparatus of claim 1 1, wherein the based on the global rotation the determined. Determining horizontal and vertical integral projection vectors for the first frame and the second frame;
Continuously convolving the first integral projection vector with a variable length vector to generate a first filtered vector;
Convolving the second integral projection vector with a fixed length vector to generate a second filtered vector; and
Determining a sum of differences between the first and second filtered vectors; and
Determining a global rotation based on the length of the variable vector N leading to the minimum of the difference sum;
Further including
12. The image acquisition device according to claim 11 , wherein the further operation is also based on the determined global rotation. Performing a global XY alignment of the first frame and the second frame;
Determining local XY alignment in at least two matched corner regions of the aligned image pair; and
Determining a global rotation of a second frame relative to the first frame based on a difference between the local XY alignments;
12. The image acquisition device according to claim 11 , wherein the further operation is also based on the determined global rotation. A storage medium having embedded code for programming a processor to determine whether a moving object is detected in a digital image frame,
The code is
Obtaining a sequence of image frames;
A global alignment of the first frame and the second frame in the sequence of image frames (the second frame follows the first frame in the sequence of image frames) is performed to obtain an aligned global vector Generating,
Determining at least one portion of the integral projection vector;
Determining an approximate size and / or relative velocity of at least one moving object in the sequence of image frames based on at least a portion of the position of each of the at least one portion of the integral projection vector;
Determining the degree of rotation between the frames;
Performing further actions based on the determined approximate size and / or relative velocity of at least one moving object ;
further,
Determining horizontal and vertical integral projection vector gradients for the first frame and the second frame;
Normalizing the horizontal and vertical integral projection vector gradients;
Determining the positions of major maximum and minimum peaks of the horizontal and vertical integral projection vector gradients; and
A processor characterized in that the further action is based on the determined global rotation by including determining a global rotation based on a normalized distance between major maximum and minimum peaks. One or more non-transitory storage media that can be read.

Images