BR112018000825B1 - METHOD IMPLEMENTED BY COMPUTER AND NON-TRANSITORY COMPUTER READABLE MEDIUM - Google Patents

Abstract

VIDEO STABILIZATION. In general, the subject can be incorporated into methods, systems and program products to identify, by a computing system and using first and second frames of a video, a transformation that indicates the movement of a camera with respect to the frames. The computing system generates a modified transformation so that the transformation is less representative of recent movement. The computing system uses the transformation and the modified transformation to generate a second transformation. The computing system identifies an early distortion that would be present in a stabilized version of the second frame. The computer system determines an amount by which to reduce a stabilization effect. The computing system applies the second transformation to the second frame to stabilize the second frame, where the stabilization effect has been reduced based on the determined amount by which to reduce the stabilization effect.

Description

TECHNICAL FIELD

[001] This document generally relates to video stabilization.

FUNDAMENTALS

[002] Video recording used to be the domain of dedicated video recording devices, but it is more common to find everyday devices such as mobile phones and tablets that are capable of recording videos. A problem with most portable recording devices is that these devices suffer from video shake, where a user's involuntary movements while holding the recording device affect the quality of the video.

[003] Shaking the recording device can result in equally jerky video unless the shaking is compensated for, for example, by a video stabilization mechanism. Optical video stabilization can lessen the vibration present in the video by mechanically moving recording device components such as a lens or image sensor. Optical video stabilization devices, however, can add material and manufacturing costs to a recording device. Additionally, optical video stabilization devices can add size to a recording device, and it is often desirable for recording devices to be small.

SUMMARY

[004] This document describes techniques, methods, systems, and other mechanisms for stabilizing video.

[005] As an additional description for the embodiments described below, this description describes the following embodiments.

[006] Modality 1 is a computer-implemented method. The method comprises receiving, by a computing system, first and second frames of a video that has been captured by a recording device such as a camera. The method comprises identifying, by the computer system and using the first and second frames of the video, a mathematical transformation that indicates the movement of the camera in relation to a scene captured by the video (that is, a scene represented in the video) since when the first frame was captured to when the second frame was captured. The method comprises generating, by the computer system, a modified mathematical transformation by modifying the mathematical transformation that indicates the movement of the camera relative to the scene, so that the mathematical transformation is less representative of the movement that has just begun. The method comprises generating, by the computer system using the mathematical transformation and the modified mathematical transformation, a second mathematical transformation that can be applied to the second frame to stabilize the second frame. The method comprises identifying, by the computer system, an anticipated distortion that would be present in a stabilized version of the second frame resulting from applying the second mathematical transformation to the second frame, based on the difference between: (i) an amount of distortion in the direction horizontal direction resulting from applying the second mathematical transformation to the second frame, and (ii) an amount of distortion in the vertical direction resulting from applying the second mathematical transformation to the second frame. The method comprises determining, by the computer system, an amount to reduce a stabilization effect that results from applying the second mathematical transformation to the second frame, based on a degree to which the anticipated distortion exceeds an acceptable change in distortion that has been calculated. from distortion in multiple frames of the video that preceded the second frame. The method comprises generating, by the computer system, the stabilized version of the second frame by applying the second mathematical transformation to the second frame, where a stabilization effect of applying the second mathematical transformation to the second frame has been reduced based on the amount determined by which reduce the stabilization effect.

[007] Mode 2 is the method of mode 1, characterized by the fact that the second frame is a video frame that immediately follows the first video frame.

[008] Modality 3 is the method of modality 1, characterized by the fact that the mathematical transformation that indicates camera movement includes a homograph transformation matrix.

[009] Modality 4 is the method of modality 3, in which the modification of the mathematical transformation includes the application of a low-pass filter to the homography transformation matrix.

[010] Mode 5 is the method of mode 3, where the early distortion is based on a difference between a horizontal zoom value in the second math transformation and a vertical zoom value in the second math transformation.

[011] Modality 6 is the method of modality 1, characterized by the fact that modifying the mathematical transformation includes modifying the mathematical transformation so that the modified mathematical transformation is more representative of the movement that occurred over a long period of time than the than the mathematical transformation.

[012] Modality 7 is the method of modality 1, characterized by the fact that the determination of the amount to reduce the stabilization effect that results from applying the second mathematical transformation to the second frame is still based on a determined movement speed of the camera from the first frame to the second frame exceeding an acceptable change in camera movement speed that was calculated based on the camera movement speed between multiple frames of video preceding the second frame.

[013] Modality 8 is the method of modality 1, characterized by the fact that generating the stabilized version of the second frame includes zooming in on a version of the second frame that was generated by applying the second mathematical transformation to the second frame.

[014] Mode 9 is the method of mode 1, characterized by the fact that the operations also comprise shifting an enlarged region of the version of the second frame horizontally or vertically to prevent the enlarged region of the second frame from presenting an invalid region.

[015] Mode 10 is addressed to a system that includes a recordable medium with instructions stored therein, the instructions, when executed by one or more processors, cause the performance of operations according to the method of any of the modes 1 to 9.

[016] Particular implementations may, in certain cases, realize one or more of the following advantages. The video stabilization techniques described here can compensate for motion in more than two degrees of freedom (e.g., more than just horizontal and vertical movement), for example, compensating for motion in eight degrees of freedom (e.g., aspects of translation, rotation aspects, zooming aspects, and non-rigid rolling shutter distortion). The video stabilization techniques described here may work as video is being captured by a device and may not require future frame information. In other words, video stabilization techniques may be able to use information from just past frames in stabilizing a more recently recorded frame, so that the system can store a stabilized video stream as the video stream is captured. (for example, without storing multiple frames of unstabilized video, such as without storing more than 1, 100, 500, 1000, or 5000 frames of unstabilized video in a video that is currently being recorded or has been recorded). Consequently, the system may not need to wait to stabilize a video until the entire video has been recorded. The described video stabilization techniques can be low in complexity and therefore can be performed on devices with modest processing power (eg some smart phones). Furthermore, the video stabilization techniques described here may be able to operate in situations where frame-by-frame motion estimation fails in the first step.

[017] Details of one or more implementations are shown in the attached drawings and in the description below. Other features, objects and advantages will be evident from the description and drawings, and from the claims.

DESCRIPTION OF THE DRAWINGS

[018] Figure 1 shows a diagram of a video stream being stabilized by a video stabilization process.

[019] Figures 2A-2B show a flowchart of a process to stabilize the video.

[020] Figure 3 is a block diagram of computing devices that can be used to implement the systems and methods described in this document, as a client or as a server or a plurality of servers.

[021] Similar reference symbols in the various drawings indicate similar elements.

DETAILED DESCRIPTION

[022] This document generally describes video stabilization. Video stabilization can be performed by identifying a transform between a newly received frame of video and a previously received frame of video (where the transform indicates movement of the camera relative to a scene from frame to frame), modifying this transform with based on information from past frames, generating a second transformation based on the transform and the modified transform, and applying the second transform to the currently received frame to generate a stabilized version of the currently received frame. This process is described generally with reference to Figure 1 and then in greater detail with reference to Figure 2.

[023] Figure 1 shows a diagram of a video stream being stabilized by a video stabilization process. The figure includes three frames from a video 110a-c. These frames can be successive, so that frame 110b can be the frame that was captured immediately after frame 110a was captured, and frame 110c can be the frame that was captured immediately after frame 110b was captured. This document may occasionally refer to two frames of a video as a first frame of a video and a second frame of a video, but the "first" notation does not necessarily mean that the first frame is the starting frame throughout the entire video.

[024] The frames 110a-c are shown positioned between or near the lines 112a-b, which indicate a position of the scenes represented by the frames in relation to each other. Lines are provided in this figure to show that the camera was moving when it captured frames 110a-c. For example, the camera was pointing further down when it captured frame 110b than it did when it captured frame 110a, and it was pointing further up when it captured frame 110c than it did when it captured frames 110a-b.

[025] A computing system identifies a mathematical transformation (box 120) that indicates the movement of the camera from the first frame 110b to the second frame 110c. Identification can be performed using frames 110b-c (as illustrated by the arrows in the figure), where frame 110c can be the most recently captured frame. These two frames 110b-c may be received from a camera sensor or camera module that is connected to the computing system, or may be received from a remote device that has captured the video frames 110b-c. Identifying the math transformation can include generating the math transformation. The math transformation can be a homograph transformation matrix, as described in further detail with respect to box 210 in Figures 2A-B.

[026] A computing system creates a modified transformation (box 125) by modifying the initial transformation (box 120) such that the modified transformation (box 125) is less representative than the initial motion transformation it just started. In other words, generating the modified transform comprises modifying the initial transform based on information from one or more video frames prior to the first and second frames 110b-c. The modified transformation can be a low-pass filtered version of the initial transformation. This results in a modified mathematical transformation (box 125) that is more representative than the initial transformation (box 120) of movement that has taken place over a long period of time, as opposed to movement that has recently started.

[027] As an example, the modified mathematical transformation (box 125) may more strongly represent a panning motion that has been occurring for several seconds than an oscillation that started a fraction of a second ago. Modifying the transform in this way takes into account earlier frames of the video, as illustrated by the arrows in Figure 1 pointing from frames 110a-b to box 122. For example, transforms calculated using earlier frames can be used to identify which movements have been going on for a longer period of time and which movements have started recently. An exemplary way of using previous frames to compute the modified transformation might be to apply a low pass filter to the homograph transformation matrix, as described in further detail with respect to box 220 in Figures 2A-B.

[028] Box 130 shows a second transformation that is generated from the initial transformation (box 120) and the modified transformation (box 125). The second transform can be a difference between the initial transform and the modified transform. Generating the second transformation from the initial transformation and the modified transformation is described in further detail with respect to box 230 in Figures 2A-B.

[029] Box 132 shows how a computing system identifies an early distortion in a stabilized version of the second frame 110c which would result from applying the second transformation to the second frame 110c, based on the difference between (i) an amount of distortion in the horizontal direction that would result from applying the second mathematical transformation to the second frame 110c, and (ii) an amount of distortion in the vertical direction that would result from applying the second mathematical transformation to the second frame 110c. The calculation of early distortion is described in further detail with respect to box 250 in Figures 2A-B.

[030] Box 134 shows how a computing system determines an amount by which to reduce a stabilization effect that results from applying the second mathematical transformation to the second frame 110c, based on a degree to which the anticipated distortion exceeds a distortion change acceptable. The acceptable distortion shift can be calculated using multiple frames of video preceding the second frame 110c, as illustrated by the arrows in Figure 1 pointing from frames 110a-b to box 134. As an example, the distortion in multiple frames preceding can be analyzed, and if the distortion that would result from stabilizing the current frame 110c significantly deviates from an amount by which the distortion is changing from frame to frame, the computing system can reduce the stabilization of the current frame 110c to prevent the distortion from being very apparent to a viewer of the video. Determining the amount to reduce video stabilization is described in further detail with respect to boxes 250 and 260 in Figures 2. The computing system's use of the amount determined to reduce video stabilization may include generating a second modified transform (box 140) using the given amount.

[031] The computing system generates the stabilized version of the second frame 110c (box 150) by applying the second modified transformation (box 140) to the second frame 110c. Since the modified second transformation (box 140) has been modified by the determined amount to reduce the stabilization, the computing system's generation of the stabilized version of the second frame (box 150) is considered to have been reduced by the determined amount to reduce the stabilization effect.

[032] In some implementations, determining the amount by which to reduce the stabilization effect is additionally or alternatively based on a given speed of movement of the camera relative to the scene from the first frame 110b to the second frame 110c exceeding a change acceptable camera speed relative to the scene. The acceptable change in camera speed can be calculated from the multiple frames of video preceding the second frame 110c, as described in greater detail in relation to boxes 240 and 260 in Figures 2A-B.

[033] In some implementations, generating the stabilized version of the second frame includes zooming into a version of the second frame that is generated by applying the second math transformation to the second frame. The computing system can shift a zoomed region horizontally, vertically, or both to prevent the zoomed region from having an invalid region that might appear at the edges of the stabilized second frame. Doing so is described in greater detail in connection with boxes 280 and 290 in Figures 2A-B.

[034] Figures 2A-B show a flowchart of a process to stabilize the video. This process is represented by boxes 210 to 290, which are described below. The operations described in association with these boxes may not have to be performed in the order below or shown in Figures 2A-B.

[035] In box 210, the computing system estimates a matrix representing frame-by-frame motion ("H_interframe") using two video frames as input. This frame-by-frame motion matrix can be a homograph transformation matrix. A homograph transformation matrix could be a matrix that could represent the movement of a scene or a camera that was capturing a scene between two frames of a video. As an example, each frame of a video can display a two-dimensional image. Suppose a first frame took a picture of a square starting straight in front of the square, so the square has equal length sides at ninety degree angles in the video frame (in other words, it looks square). Suppose now that the camera was moved to the side (or the square itself was moved) so that the next frame of video shows the square as distorted with some sides longer than others and with angles that are not ninety degrees. The location of the square's four corner points in the first frame can be mapped to the location of the four corner points in the second frame to identify how the camera or scene has changed from one frame to the next.

[036] The mapping of these corner points to each other in the frames can be used to generate a homograph transformation matrix that represents the movement from the camera's point of view in relation to the scene that is recording. Given this homography transformation matrix, a first frame can be used with the generated homography transformation matrix to recreate the second frame, for example moving pixels in the first frame to different locations according to known homography transformation methods.

[037] The homograph transformation matrix that is described above can represent not only translational movement, but also rotation, zooming, and non-rigid rolling shutter distortion. In this way, the application of the homograph transformation matrix can be used to stabilize the video with respect to motion in eight degrees of freedom. To compare, some video stabilization engines only stabilize images to account for translational motion (eg up/down and left/right motion).

[038] The homography transformation matrix described above can be a 3x3 homography transformation matrix, although other types of homography matrices can be used (and other mathematical representations of movement from one frame to another, even if not a homography matrix or even if not a matrix can be used). The 3x3 matrix (referred to as H_interface) can be determined as follows. First, a computing system finds a set of feature points (usually corner points) in the current image, where these points are denoted [x'_i, y'_i], i = 1 ... N (N is the number resource points). Then the corresponding feature points are found in the previous frame, where the corresponding feature points are denoted [x_i, y_i]. Note that the points are described as being in the GL coordinate system (that is, the x and y ranges from -1 to 1 and with the frame center as the origin). If the points are in the image pixel coordinate system where x varies from 0 to the image width and y varies from 0 to the image height, then the points can be transformed to the GL coordinate system or the resulting matrix can be be transformed to compensate.

[039] The matrix described above H_interframe is a 3x3 matrix that contains 9 elements: h1, h2, h3 •H_interframe = h4, h5, h6 h7, h8, h9 H_interfame is the transformation matrix that transforms [x_i, y_i] into [ x'_i, y'_i], as described below. • z_i '* [x'_i, y'_i, 1]' = H_interframe * [x_i, y_i, 1]' [x'_i, y'_i, 1] 'is a 3x1 vector that is the transpose of vector [ x'_i, y'_i, 1]. [x_i, y_i, 1]' is a 3x1 vector which is the transpose of the vector [x_i, y_i, 1]. z_i' is a scale factor.

[040] Given a set of corresponding feature points, an example algorithm for estimating the matrix is described in the following computer vision book in Algorithm 4.1 (page 91) and Algorithm 4.6 (page 123): "Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)," available at ftp://vista.eng.tau.ac.il/dropbox/aviad/Hartley,%20Zisserma n%20- %20Multiple%20View%20Geometry%20in%20Computer%20 Vision.pdf

[041] In box 220, the computing system estimates a lowpass transformation matrix (H_lowpass). The low-pass transformation matrix can be further combined with the H_interframe matrix to generate a new matrix (H_compensation) that can be used to remove the results of unintentional "high frequency" video camera motion. If the system tried to remove all motion (in other words, it didn't do the low-pass filtering described here), the user might not be able to voluntarily move the camera and make the scene described by the video move as well. As such, computing systems generate low-pass transformation to filter out high-frequency motions. High frequency motions can be those motions that are irregular and not represented across many frames, such as back and forth motions with a short period. Conversely, low-frequency motions can be motions that are represented across multiple frames, such as a user panning a video camera for several seconds.

[042] To perform this filtering, the computing system generates a lowpass transformation matrix (H_lowpass) that includes weighted values to emphasize low-frequency movements that occurred over a series of long periods. The lowpass transformation matrix can be the result of applying a lowpass filter on the H_interframe matrix. Each element in the low-pass transformation matrix is generated individually, on an element-by-element basis, from (1) its own time series of the previous frame's low-pass transformation matrix, (2) the H_interframe matrix representing the motion between the previous frame and current frame, and (3) a smoothing ratio that is specified by the user. In other words, the elements in the matrix that are weighted with notable values could be the elements that represent motion that has been present in the H_interframe matrix through many frames. The equation to generate H_lowpass can be represented as follows: • H_lowpass = H_previous_lowpass * transform_damping_ratio + H_interframe * (1 - transform_damping_ratio) This equation is an example of a two-stroke infinite impulse response filter.

[043] At box 230, the computing system calculates a compensation transformation matrix (H_compensation). The compensation matrix can be a combination of the lowpass matrix (H_lowpass) and the frame-by-frame motion matrix (H_interframe). Combining these two matrices generates a matrix (H_compensation) which is needed to keep the movement from one frame to another, but only the movements that have occurred for a reasonable amount of time, excluding recent "involuntary" movements. The H_compensation matrix can represent the difference in movements between H_interfame and H_lowpass, so while H_lowpass could be applied to the last frame to generate a modified version of the last frame that represents the voluntary movements that occurred between the last frame to the current frame, H_compensation can be applied to the current frame to generate a modified (and stabilized) version of the current frame that represents the voluntary movements that occurred between the last frame and the current frame. In coarse language, by applying H_compensation, the current frame removes involuntary motion from that frame. Specifically, given this computed H_compensation matrix, the system should be able to take the current video frame, apply the H_compensation matrix to that frame with a transform process, and get a newly generated frame similar to the current frame, but excluding any sudden motion. is small. In other words, the system tries to keep the current frame as close as possible to the last frame, but allow for long-term "voluntary" movements.

[044] The compensation transformation matrix can be generated with the following equation: • H_compensation = Normalize (Normalizes (Normalizes (H_lowpass) * H_previous_compensation) * Inverts (H_interframe))

[045] The H_previous_compensation matrix is the H_constrained_compensation matrix that is calculated later in this process, but the one that was calculated for the previous frame. Invert() is the inverse matrix operation that is used to generate the original version of the last frame when inverting the transform. Combining the original version of the last frame with the low-pass filter matrix allows for voluntary movements. Combining with H_previous_compensation offsets the previous compensation value.

[046] Normalize() is an operation that normalizes the 3x3 matrix by its second singular value. The normalization process is performed because some of the steps in the process can result in a transformation that, for lack of better words, would not make much sense in the real world. As such, a normalization process can ensure that a reasonable result is obtained from each step of the process. Normalization is performed for each step of the process, so that a strange output from one step does not pollute the remaining steps of the process (for example, imagine if the strange output provided a value close to zero that pulled the output of the rest of the steps towards also be close to zero). For reasons that are discussed below, additional processing can improve the results of the video stabilization process.

[047] In box 240, the computing system calculates a speed reduction value. The speed reduction value can be a value that is used to determine how much to reduce the video stabilization when the camera moves too quickly and the video stabilization becomes undesirable because frame-by-frame movement can be unreliable. To calculate the amount by which video stabilization can be reduced, the movement speed between frames is first calculated. In this example, the computing system generates the speed of the center of the frame. Velocity in x direction is pulled from element in row 1 column 3 in H_interframe as follows (box 242): • speed_x = H_interframe [1, 3] * aspect_ratio Velocity in y direction is pulled from element in row 2, column 3 into the H_interframe array, as follows (box 242): • speed_y = H_interframe [2, 3]

[048] The aspect_ratio described above is the frame_width divided by the frame_height. These speed tags can only explain translational movement between two frames, but speed can explain rotation, zooming, or other types of movement in other examples.

[049] The system can then determine the speed of the low-pass movement that accounts for the long-term speed of the camera (or the scene) and excludes sudden and rapid "involuntary" movements. This is done by taking the current velocity and combining it with the previously calculated low-pass velocity, and further applying a damping ratio that inversely weighs the current velocity against the previously calculated low-pass velocity, for example, as follows: •lowpass_speed_x = lowpass_speed_x_previous * speed_damping_ratio + speed_x * (1 - speed_damping_ratio)

[050] This equation effectively generates a low-pass velocity by taking the previously calculated velocity and reducing it by an amount that is specified by a damping ratio. The reduction is compensated for by the current speed. In this way, the current video speed affects the overall lowpass_speed value, but is not a unique factor in the lowpass_speed value. The above equation represents an infinite impulse response filter. The same process can be performed for velocity y to generate the lowpass velocity, for example as follows: • lowpass_speed_y = lowpass_speed_y_previous * speed_damping_ratio + speed_y * (1 - speed_damping_ratio)

[051] The damping ratio in this process is defined by a user, and an example value is 0.99.

[052] The process then combines these values to generate a single low-pass velocity representation that explains motion in the x and y directions, for example, using the following equation (box 244): • lowpass_speed = sqrt(lowpass_speed_x * lowpass_speed_x + lowpass_speed_y * lowpass_speed_y)

[053] This calculated low-pass speed essentially represents the long-term movement speed between frames. In other words, lowpass_speed counts less for recent changes in speed and weighs the long-term speed trend more heavily.

[054] With the low-pass speed calculated, the system can calculate the speed reduction value. In some examples, the speed reduction value is a value between 0 and 1 (other threshold values are possible) and the system may generate the speed reduction value based on how the low-pass speed compares to a threshold low and a high threshold. If the low-pass speed is below the low threshold, the speed reduction value can be set to the threshold value 0. If the low-pass speed is above the high threshold, the speed reduction value can be set to the threshold value 1. If the low-pass speed is between the two thresholds, the computing system can select a speed reduction value that represents the scaled value of the low-pass speed between the thresholds, for example, where the reduction value value lies between the threshold values 0 and 1. The calculation of the speed rejection value can be represented with the following algorithm (box 246): • If lowpass_speed <low_speed_threshold, then speed_reduction = 0 • Else if lowpass_speed> high_speed_threshold, then speed_reduction = max_speed_reduction • Otherwise, speed_reduction = max_speed_reduction * (lowpass_speed - low_speed_threshold)/(high_speed_threshold - low_speed_threshold)

[055] With this algorithm, the low_speed_threshold, high_speed_threshold and max_speed_reduction are all specified by the user. Example values include low_speed_threshold = 0.008; high_speed_threshold = 0.016; and max_speed_reduction = 1.0.

[056] At box 250, the computing system calculates a distortion reduction value. The computing system can calculate a distortion reduction value because the compensation transform can create very severe distortion when applied to a video frame. In other words, video stabilization may not appear to be realistic, for example, because distortion caused by stretching the image in one direction more than the other can occur very quickly and could appear unusual to a user.

[057] To calculate the amount of distortion reduction, the computing system can first calculate the compensation zoom factor by looking at the values of the zoom factors in the H_compensation matrix, as follows: •zoom_x = H_compensation [1,1] which is the element from row 1 column 1 in the H_compensation matrix • zoom_y = H_compensation [2,2] which is the element from row 2 column 2 in the H_compensation matrix

[058] The zoom factors can be the factors that identify how the transform stretches the image by one dimension.

[059] The computing system can then determine the difference between the two zoom factors, to determine the degree to which the image is being distorted, stretching more in one direction than the other, as follows (box 252): • distortion = abs(zoom_x - zoom_y)

[060] A low-pass filter is applied to the distortion, to place a limit on the rate at which distortion is allowed, and therefore to make sure that sudden changes in distortion have been minimized, using the following formula (box 254) : • lowpass_distortion = previous_lowpass_distortion * distortion_damping_ratio + distortion * (1 - distortion_damping_ratio)

[061] Put another way, the algorithm is organized to allow the amount of distortion to change slowly. In the above formula, the distortion_damping_ratio is the damping ratio of the distortion IIR filter that is specified by the user. An example value is 0.99.

[062] With the low low-pass distortion calculated, the computing system can calculate the distortion reduction value. In some examples, the distortion reduction value is a value between 0 and 1 (other threshold values are possible), and the system may generate the distortion reduction value based on how the low-pass distortion compares to a low threshold and a high threshold. If the low-pass distortion is below the low threshold, the low-pass reduction value can be set to the threshold value 0. If the low-pass distortion value is above the high threshold, the low-pass reduction value distortion can be set to threshold value 1. If the low-pass distortion value is between the two thresholds, you can select a value that represents the scaled value of the low-pass distortion between the thresholds (for example, where the value of resulting distortion reduction lies between threshold values 0 and 1). The calculation of the distortion reduction value can be represented with the following algorithm (box 256): • If lowpass_distortion < low_distortion_threshold, then distortion reduction = 0 • Else if lowpass_distortion > high_distortion_threshold, then max_distortion_reduction • Otherwise, distortion_reduction = max_distortion_reduction * (lowpass_distortion - low_distortion_threshold)/(high_distortion_threshold - low_distortion_threshold)

[063] With this algorithm, low_distortion_threshold, high_distortion_threshold, and max_distortion_reduction are all user-specified. Example values include low_distortion_threshold = 0.001, high_distortion_threshold = 0.01, and max_distortion_reduction = 0.3.

[064] In box 260, the computing system reduces a video stabilization strength based on the determined speed reduction value and the distortion reduction value. To do this, the computing system calculates a reduction value, which in this example is identified as the maximum of the speed reduction value and the distortion reduction value (box 262), as follows: • reduction = max (speed_reduction , distortion_reduction)

[065] In other examples, the reduction value can be a combination of these two values representing a portion of each value (for example, values can be added or multiplied together and possibly multiplied by a predetermined number such as 0, 5). The reduction value can fall at or between the threshold values of 0 and 1, and the closer the reduction value is to 1, the more the computing system can reduce the strength of the image stabilization.

[066] The computing system can then modify the compensation transformation matrix to generate a reduced compensation transformation matrix (box 264). The computing system can do this by multiplying the offset transformation matrix by subtracting the offset value from one. In other words, if the reduction value is very close to one (indicating that there is a large reduction in image stabilization), the values in the compensation matrix could be significantly decreased because they would be multiplied by a number close to zero. The numbers in the modified compensation transformation matrix are then added to an identity matrix that has been multiplied by the reduction value. An example equation follows: • H_reduced_compensation = Identity * reduction + H_compensation * (1 - reduction)

[067] In box 270, the computing system may constrain the compensation so that the resulting video stabilization does not show invalid regions of the output frame (eg, regions that are outside the frame). As a little background, because compensation can warp the image that results from the image stabilization process, this image can exhibit invalid regions that are essentially outside the image at its edges. To ensure that these invalid regions are not shown, the computing system can zoom in on the image to crop the sides of the image that may include the invalid regions.

[068] Going back to the processes in box 270, if the camera is moved quickly and significantly, stabilization can block the display of an old location because rapid and significant movement can be filtered out, which can introduce the invalid regions described above in the display of a stabilized frame. In this case, the restriction process described below can ensure that video stabilization essentially stops completely controlling the region of the frame that is displayed if the video is about to display an invalid region. This determination as to whether the stabilization needs to give up some control of the frame can start by initially defining the corner points of the output image and determining whether those corner points fall outside a pre-specified crop region. A maximum amount of offset and zoom can be set to twice the crop ratio, where the crop ratio can be specified by a user (e.g. to be 15% on each side, or 0.15 in the equation below): • max_compensation = cropping_ratio * 2

[069] The computing system can then use the H_reduced_compensation matrix to transform the 4 corners of a unit square into the GL coordinate (x01, y01) = (-1, -1), (x02, y02) = (1, -1 ), (x03 (x1, y0)) = (-1,1), (x04, y04) = (1, 1) for the 4 corner points (x1, y1), (x2, y2), (x3, y3), (x4, y4). (Note that the video frame does not have to be a square unit, but the frame unit dimensions are mapped to a square unit in the GL coordinate). More specifically, we use the following formula to transform (x0i, y0i) into (xi, yi): • dzi * [xi, yi, 1]' = H_reduced_compensation * [x0i, y0i, 1]'

[070] In this example, [x0i, y0i, 1]' is a 3x1 vector that is the transpose of vector [x0i, y0i, 1]. [xi, yi, 1]' is a 3x1 vector that is the transpose of the vector [xi, yi, 1]. zi is a scale factor.

[071] The computing system can then identify the maximum amount of shift in each direction (left, right, up, and down) from the corners of each transformed video frame to the unit square edge, as follows: • max_left_displacement = 1 + max (x1, x3) • max_right_displacement = 1 - min (x2, x4) • max_top_displacement = 1 + max (y1, y2) • max_bottom_displacement = 1 - min (y3, y4)

[072] If any of the identified offsets exceed the maximum amount of offset (which is twice the crop ratio as described above and which would indicate that the invalid regions are in the displayed region of the magnified unit square region), then the corner points of the frame are shifted by the same amount away from the edge of the unit square, so invalid regions will not be displayed. The equations for shifting the corner points follow accordingly: • If max_left_displacement> max_compensation, shift the 4 corner points to the left by max_left_displacement - max_compensation • If max_right_displacement> max_compensation, shift the 4 corner points to the right by max_right_displacement - max_compensation • If max_top_displacement > max_compensation, shift the 4 corner points up by max_top_displacement - max_compensation • If max_bottom_displacement > max_compensation, shift the 4 corner points down by max_bottom_displacement - max_compensation.

[073] The offset of the corner points is an identification that invalid regions would have been shown, even if the screen had been cropped (box 272).

[074] After all the shift operations above, the 4 new corner points can be denoted (x1', y1'), (x2', y2'), (x3', y3'), (x4', y4' ). The computing system then computes the constrained compensation transformation matrix H_constrained_compensation, which maps the four corners of a unit square to the GL coordinate (x01, y01) = (-1, -1), (x02, y02) = (1, -1) (x03, y03) = (-1,1), (x04, y04) = (1,1) for the 4 constrained corner points (x1', y1'), (x2', y2'), (x3', y3'), (x4', y4'), as follows: • zi' * [xi', yi' , 1]' = H_constrained_compensation * [x0i, y0i, 1]'

[075] In this example, [x0i, y0i, 1]' is a 3x1 vector that is the transpose of vector [x0i, y0i, 1]. [xi', yi', 1]' is a 3x1 vector that is the transpose of the vector [xi', yi' , 1]. zi' is a scale factor. Given 4 pairs of points [x0i, y0i, 1]' and [xi', yi', 1]', an example algorithm for estimating the matrix is described in the following computer vision book on algorithm 4.1 (page 91): "Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)," available at ftp://vista.eng.tau.ac.il/dropbox/aviad/Hartley, %20Zisserma n%20-%20Multiple%20View%20Geometry%20in%20Computer %20Vision.pdf

[076] The H_constrained_compensation is then saved as H_previous_compensation, which can be used in calculations to stabilize the next frame as described above regarding box 230.

[077] At box 280, the computing system modifies the constrained compensation matrix so that the stabilized image is enlarged to crop the edge. In some examples, the computing system first identifies a zoom factor as follows: • zoom_factor = 1/(1 - 2 * cropping_ratio)

[078] In doing so, the computing system doubles the cutoff ratio (for example, by doubling the 15% value from 0.15 to 0.3), subtracting the resulting value from 1 (for example, to obtain 0.7) and then dividing 1 by that result to get the zoom factor (for example, 1 divided by 0.7 to equal a zoom factor of 1.42). The computing system can then slice certain features out of the constrained compensation matrix to increase the amount one way, as follows: • H_constrained_compensation [3,1] = H_constrained_compensation [3,1]/zoom_factor • H_constrained_compensation [3,2 ] = H_constrained_compensation [3,2]/zoom_factor • H_constrained_compensation [3,3] = H_constrained_compensation [3,3]/zoom_factor

[079] At box 290, the computing system applies the modified constrained offset matrix to the current frame to generate a cropped and stabilized version of the current frame. An example way of applying the constrained compensation matrix (H_constrained_compensation) on the input frame to produce the output image can be described as follows. • z'* [x', y', 1]' = H_constrained_compensation * [x, y, 1]' • [x, y, 1]' is a 3x1 vector representing a coordinate in the input frame • [x' , y', 1]' is a 3x1 vector representing the coordinate in the output frame • z 'is a scale factor • H_constrained_compensation is a 3x3 matrix containing 9 elements: h1, h2, h3 • H_constrained_compensation = h4, h5, h6 h7, h8, h9

[080] In further detail, for each pixel [x, y] in the input frame, find the position [x', y'] in the output frame using the transform above, and copy the pixel value of [x, y ] in the input frame to [x', y'] in the output frame. Another way is for each pixel [x', y'] in the output frame, find the position [x, y] in the input frame using the inverse transform, and copy the pixel value of [x, y] into the image from input to [x', y'] in the output frame. These operations can be performed on Graphics Processing Unit (GPT) computing systems efficiently.

[081] The process described here for boxes 210 to 290 can then be repeated for the next frame, with some of the values from processing the current frame being used for the next frame.

[082] In many implementations, operations that are performed "in response to" or "as a consequence of" another operation (for example, a determination or an identification) are not performed if the previous operation is unsuccessful (for example, if the determination was not performed). Operations that run "automatically" are operations that run without user intervention (for example, intermediate user input). Features in this document that are described with conditional language may describe implementations that are optional. In some examples, "broadcast" from a first device to a second device includes the first device putting data onto a network for reception by the second device, but may not include the second device receiving the data. On the other hand, "receiving" from a first device can include receiving data from a network, but cannot include the first device transmitting the data.

[083] "Determine" by a computing system may include the computing system requesting another device to perform the determination and provide the results to the computing system. In addition, "displaying" or "presenting" by a computing system may include the computing system sending data to cause another device to display or present the related information.

[084] In various implementations, operations that are described as being performed on an array mean operations that are performed on that array or a version of that array that has been modified by an operation described in this disclosure or an equivalent thereof.

[085] Figure 3 is a block diagram of the computing devices 300, 350 that can be used to implement the systems and methods described in this document, as a client or as a server or a plurality of servers. Computing device 300 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Computing device 350 is intended to represent various forms of mobile devices, such as personal digital assistants, cell phones, smart phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are examples only and should not limit implementations described and/or claimed in this document.

[086] The computing device 300 includes a processor 302, memory 304, a storage device 306, a high-speed interface 308 that connects to memory 304 and high-speed expansion ports 310, and a low-speed interface 312 that connects to connects to the low-speed bus 314 and the storage device 306. Each of the components 302, 304, 306, 308, 310 and 312 are interconnected using various buses and can be mounted on a common motherboard or in other ways, as required. appropriate. Processor 302 may process instructions for execution within computing device 300, including instructions stored in memory 304 or storage device 306 to display graphical information for a GUI on an external input/output device, such as the display 316 coupled to a high-speed interface 308. In other implementations, multiple processors and/or multiple buses may be used, as appropriate, along with multiple memories and memory types. Furthermore, multiple computing devices 300 may be connected, with each device providing portions of the necessary operations (eg, as a bank of servers, a group of blade servers, or a multiprocessor system).

[087] The memory 304 stores information within the computing device 300. In one implementation, the memory 304 is a volatile memory unit or units. In another implementation, memory 304 is a non-volatile memory unit or units. Memory 304 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[088] Storage device 306 is capable of providing mass storage for computing device 300. In one implementation, storage device 306 may be or contain a computer-readable medium, such as a floppy disk device, a device hard disk drive, an optical disk device, or a tape device, a flash memory or other similar solid-state memory device, or a variety of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly incorporated into an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as memory 304, storage device 306, or memory in processor 302.

[089] The high-speed controller 308 manages bandwidth-intensive operations for the computing device 300, while the low-speed controller 312 manages low-bandwidth-intensive operations. This allocation of roles is just an example. In one implementation, the high-speed controller 308 is coupled to memory 304, display 316 (e.g., via a graphics processor or accelerator), and to high-speed expansion ports 310, which can accept multiple expansion cards (not shown). In the implementation, the low-speed controller 312 is coupled to the storage device 306 and the low-speed expansion port 314. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet , wireless Ethernet), can be attached to one or more input/output devices such as a keyboard, pointing device, scanner or a network device such as a switch or router, for example via a network adapter .

[090] The computing device 300 can be implemented in several different ways, as shown in the figure. For example, it can be implemented as a standard 320 server, or multiple times in a group of these servers. It may also be implemented as part of an off-the-shelf server system 324. Furthermore, it may be implemented on a personal computer, such as a laptop computer 322. Alternatively, components from the computing device 300 may be combined with other components in a mobile device (not shown), such as device 350. Each of these devices can contain one or more of computing devices 300, 350, and an entire system can be comprised of multiple computing devices 300, 350 that communicate with each other. other.

[091] The computing device 350 includes a processor 352, memory 364, an input/output device such as a display 354, a communication interface 366, and a transceiver 368, among other components. Device 350 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components 350, 352, 364, 354, 366 and 368 is interconnected using various buses, and various of the components may be mounted on a common motherboard or in other ways as appropriate.

[092] The processor 352 can execute instructions within the computing device 350, including instructions stored in memory 364. The processor can be implemented as a chipset of chips that includes multiple and separate analog and digital processors. Furthermore, the processor can be implemented using any of several architectures. For example, the processor can be a CISC (Complex Instruction Computer Set) processor, a RISC (Reduced Instruction Computer Set) processor, or a Minimal Instruction Computer Set (MISC) processor. The processor may provide, for example, coordination of the other components of device 350, such as control of user interfaces, applications executed by device 350, and wireless communication by device 350.

[093] The processor 352 can communicate with a user through the control interface 358 and the display interface 356 coupled to a display 354. The display 354 can be, for example, a TFT screen (Top Transistor Liquid Crystal Display) Thin Film) or an OLED (Organic Light Emitting Diode) or other appropriate display technology. Display interface 356 may comprise appropriate circuitry to drive display 354 to present graphical and other information to a user. The control interface 358 can receive commands from a user and convert them for submission to the processor 352. Additionally, an external interface 362 can provide in communication with the processor 352 so as to allow near area communication from the device. 350 with other devices. External interface 362 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and various interfaces may also be used.

[094] The memory 364 stores information within the computing device 350. The memory 364 may be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units . Expansion memory 374 may also be provided and connected to device 350 through expansion interface 372, which may include, for example, a SIMM (Single In-Line Memory Module) card interface. Such expansion memory 374 can provide extra storage space for device 350, or it can also store applications or other information for device 350. Specifically, expansion memory 374 can include instructions for performing or supplementing the processes described above and can include secure information too. Thus, for example, expansion memory 374 can be provided as a security module for device 350 and can be programmed with instructions that allow secure use of device 350. In addition, secure applications can be provided via SIMM cards, along with additional information such as how to securely place identifying information on the SIMM card.

[095] Memory may include, for example, flash memory and/or NVRAM memory, as discussed below. In one implementation, a computer program product is tangentially embedded in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods as described above. The information carrier is a computer-readable or machine-readable medium, such as memory 364, expansion memory 374, or memory in processor 352 that may be received, for example, through transceiver 368 or external interface 362.

[096] The device 350 can communicate wirelessly through the communication interface 366, which can include digital signal processing circuitry, when necessary. Communication interface 366 can provide communications under various modes or protocols, such as GSM voice calls, SMS, EMS or MMS, CDMA, TDMA, PDC, WCDMA, CDMA2000 or GPRS, among others. Such communication may occur, for example, via radio frequency transceiver 368. In addition, short range communication may occur, such as using a Bluetooth, WiFi, or other transceiver (not shown). Additionally, GPS (Global Positioning System) receiver module 370 can provide additional wireless navigation and location related data to device 350 which can be used as appropriate by applications running on device 350.

[097] Device 350 may also communicate audibly using audio codec 360, which may receive spoken information from a user and convert it into usable digital information. Audio codec 360 may also generate sound that is audible to a user, such as through a loudspeaker, for example, in a device headset 350. This sound may include voice call sound, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device 350.

[098] The computing device 350 can be implemented in several different ways, as shown in the figure. For example, it may be implemented as a cell phone 380. It may also be implemented as part of a smart phone 382, personal digital assistant, or other similar mobile device.

[099] In addition, computing device 300 or 350 may include USB (Universal Serial Bus) flash drives. USB flash drives can store operating systems and other applications. USB flash drives can include input/output components such as a wireless transmitter or USB connector that can be inserted into a USB port on another computing device.

[100] Various implementations of the systems and techniques described herein can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), hardware, firmware, software, and/or combinations thereof. These various implementations may include implementation in one or more computer programs that are executable and/or interpretable in a programmable system including at least one programmable processor, which may be general or special purpose, coupled to receive data and instructions from, and for transmitting data and instructions to, a storage system, at least one input device and at least one output device.

[101] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and/or object-oriented programming language and/ or in assembly/machine language. As used herein, the terms "machine-readable medium", "computer-readable medium", refer to any computer program product, apparatus and/or device (e.g., magnetic disks, optical disks, memory, logical devices (PLD)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives the machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

[102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer with a display device (e.g., a CRT (cathode ray tube) monitor or LCD (liquid crystal display) to display information to the user and a keyboard and pointing device (for example, a mouse or a trackball) by which the user can provide input to the computer. Other types of devices can be used to provide interaction with a user as well; for example , the feedback provided to the user can be any form of sensory feedback (for example, visual feedback, auditory feedback, or haptic feedback); and user input can be received in any form, including acoustic, speech, or haptic input.

[103] The systems and techniques described here can be implemented on a computing system that includes a backend component (e.g., such as a data server), or that includes a middleware component (e.g., an application server). ), or that includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of these back-terminal, middleware, or front-terminal components. System components may be interconnected by any form or means of digital data communication (eg a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), peer-to-peer networks (with ad hoc or static members), grid computing infrastructures, and the Internet. .

[104] The computer system may include both clients and servers. A client and server are usually distant from each other and typically interact over a communication network. The client and server relationship arises by virtue of computer programs running on respective computers and having a client-server relationship with one another.

[105] Although some implementations have been described in detail above, other modifications are possible. In addition, other mechanisms for realizing the systems and methods described in this document can be used. Furthermore, the logic flows depicted in the figures do not require the particular order shown, or the sequential order, to obtain desirable results. Other steps may be provided, or steps may be removed, from the described flows, and other components may be added or removed from the described systems. Accordingly, other implementations are within the scope of the following claims.

Claims (12)

1 . Computer-implemented method, CHARACTERIZED by the fact that it comprises: receiving, by a computer system, the first and second frames of a video that was captured by a camera; identify, by the computer system and using the first and second frames of the video, a mathematical transformation that indicates the movement of the camera in relation to a scene captured by the video when the first frame was captured to when the second frame was captured; generate, by the computer system, a modified mathematical transformation by modifying the mathematical transformation so that the mathematical transformation is less representative of the movement that has recently started; generating, by the computer system using the mathematical transformation and the modified mathematical transformation, a second mathematical transformation that can be applied to the second frame to stabilize the second frame; identify, by the computer system, an anticipated distortion that would be present in a stabilized version of the second frame resulting from applying the second mathematical transformation to the second frame, based on the difference between: (i) an amount of distortion in the horizontal direction resulting from the application from the second mathematical transformation to the second frame; and (ii) an amount of distortion in the vertical direction resulting from applying the second mathematical transformation to the second frame; determine, by the computer system, an amount by which to reduce a stabilization effect that results from applying the second mathematical transformation to the second frame, based on a degree to which the anticipated distortion exceeds an acceptable distortion change that has been calculated from the distortion in multiple video frames preceding the second frame; and generating, by the computer system, the stabilized version of the second frame by applying the second math transformation to the second frame, where a stabilization effect of applying the second math transformation to the second frame has been reduced based on the determined amount to reduce to the stabilization effect, where the second frame is a video frame that immediately follows the first video frame, where the mathematical transformation indicating camera movement includes a homograph transform matrix, and where the transformation modification Mathematics includes applying a low-pass filter to the homography transformation matrix. 2. The computer-implemented method of claim 1, CHARACTERIZED by the fact that the anticipated distortion is based on a difference between a horizontal stretch value in the second mathematical transformation and a vertical stretching value in the second mathematical transformation. 3. Computer-implemented method, according to claim 1, CHARACTERIZED by the fact that the low-pass filter is generated according to: I HJowpasS = H_previous_lüwpass * traπsform_damping_ratio + Hjntθrframe * (1 - transformjdamping_ratio) where H_previous_lowpass is a previous frame lowpass transform matrix H, the interframe matrix represents the movement between the previous frame and the current frame, and transform_damping_ratio is a specified damping ratio. 4. Computer-implemented method, according to claim 1, CHARACTERIZED by the fact that the determination of the amount by which to reduce the stabilization effect that results from the application of the second mathematical transformation to the second frame is still based on a certain speed of camera movement from the first frame to the second frame exceeding an acceptable change in camera movement speed that was calculated based on a camera movement speed between the multiple frames of video that preceded the second frame. 5. Computer-implemented method according to claim 1, CHARACTERIZED by the fact that generating the stabilized version of the second frame includes zooming in on a version of the second frame that was generated by applying the second mathematical transformation to the second frame. 6. The computer-implemented method of claim 5, further comprising shifting an enlarged region of the second frame version horizontally or vertically to prevent the enlarged region of the second frame from displaying an invalid region. 7. Non-transient computer-readable medium, CHARACTERIZED by the fact that it comprises instructions that, when executed by a computer, cause the computer to perform operations including: receiving, by a computer system, first and second frames of a video that was captured by a camera; identify, by the computer system and using the first and second frames of the video, a mathematical transformation that indicates the movement of the camera in relation to a scene captured by the video from when the first frame was captured to when the second frame was captured; generating, by the computer system, a modified mathematical transformation by modifying the mathematical transformation, such that the mathematical transformation is less representative of movement that has recently started; generating, by the computer system using the mathematical transformation and the modified mathematical transformation, a second mathematical transformation that is capable of being applied to the second frame to stabilize the second frame; identify, by the computer system, an anticipated distortion that would be present in a stabilized version of the second frame that results from applying the second mathematical transformation to the second frame, based on a difference between: (i) an amount of distortion in the horizontal direction resulting from applying the second mathematical transformation to the second frame; and (ii) an amount of distortion in the vertical direction that results from applying the second mathematical transformation to the second frame; determining, by the computer system, an amount by which to reduce a stabilization effect that results from applying the second mathematical transformation to the second frame, based on a degree to which the anticipated distortion exceeds an acceptable distortion change that has been calculated from of distortion in multiple video frames that preceded the second frame; and generating, by the computer system, the stabilized version of the second frame by applying the second mathematical transformation to the second frame, where a stabilization effect of applying the second mathematical transformation to the second frame has been reduced based on the determined amount by which to reduce the stabilization effect where the second frame is a video frame immediately following the first video frame, where the mathematical transformation indicating camera movement includes a homograph transform matrix, and where the mathematical transformation modification includes applying a low-pass filter to the homography transformation matrix. 8. Computer-readable medium according to claim 7, CHARACTERIZED by the fact that the anticipated distortion is based on a difference between a horizontal zoom value in the second mathematical transformation and a vertical zoom value in the second mathematical transformation. 9. Computer-readable medium according to claim 7, CHARACTERIZED by the fact that modifying the mathematical transformation includes modifying the mathematical transformation so that the modified mathematical transformation is more representative of motion that has occurred over a long period of time than the mathematical transformation. 10. Computer-readable medium, according to claim 7, CHARACTERIZED by the fact that the determination of the amount by which to reduce the stabilization effect that results from the application of the second mathematical transformation to the second frame is still based on a certain camera movement speed from the first frame to the second frame exceeding an acceptable camera movement speed change that was calculated based on a camera movement speed between the multiple video frames that preceded the second frame. 11. Computer-readable medium according to claim 7, CHARACTERIZED by the fact that generating the stabilized version of the second frame includes zooming in on a version of the second frame that was generated by applying the second mathematical transformation to the second frame. 12. The computer-readable medium of claim 11, CHARACTERIZED in that the operations further comprise shifting an enlarged region of the second frame version horizontally or vertically to prevent the enlarged region of the second frame from displaying an invalid region.