WO2022214037A1 - Procédé et appareil de traitement anti-tremblement de vidéo, dispositif électronique et support de stockage - Google Patents

Procédé et appareil de traitement anti-tremblement de vidéo, dispositif électronique et support de stockage Download PDF

Info

Publication number
WO2022214037A1
WO2022214037A1 PCT/CN2022/085634 CN2022085634W WO2022214037A1 WO 2022214037 A1 WO2022214037 A1 WO 2022214037A1 CN 2022085634 W CN2022085634 W CN 2022085634W WO 2022214037 A1 WO2022214037 A1 WO 2022214037A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
image frames
different image
shooting position
target
Prior art date
Application number
PCT/CN2022/085634
Other languages
English (en)
Chinese (zh)
Inventor
杨松
刘宇龙
Original Assignee
北京字跳网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字跳网络技术有限公司 filed Critical 北京字跳网络技术有限公司
Publication of WO2022214037A1 publication Critical patent/WO2022214037A1/fr
Priority to US18/472,001 priority Critical patent/US20240013347A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2622—Signal amplitude transition in the zone between image portions, e.g. soft edges

Definitions

  • the present disclosure relates to the technical field of video processing, and in particular, to a video anti-shake processing method, device, electronic device and storage medium.
  • the embodiments of the present disclosure provide a video anti-shake processing method, apparatus, electronic device, and storage medium.
  • an embodiment of the present disclosure provides a video anti-shake processing method, including:
  • the initial change amount of the shooting position between different image frames in the video is determined based on the initial transformation method
  • the target variation of the shooting position between different image frames in the video is determined by adopting a target transformation method that matches the fitting error
  • the video is deformed to obtain an anti-shake processed video.
  • an embodiment of the present disclosure further provides a video anti-shake processing device, including:
  • an initial change amount determination module configured to determine the initial change amount of the shooting position between different image frames in the video based on the initial transformation method by tracking feature points between different image frames in the video;
  • a target variation determination module configured to determine, based on the fitting error corresponding to the initial variation, the target variation of the shooting position between different image frames in the video by adopting a target transformation method that matches the fitting error;
  • a movement trajectory generation module configured to form a movement trajectory of the shooting position of the video based on the target change amount of the shooting position between different image frames in the video, wherein the movement trajectory is used to indicate different images in the video the shooting position of the frame;
  • a smooth trajectory determination module configured to perform smooth processing on the shooting positions of different image frames in the moving trajectory respectively to obtain a smooth trajectory
  • a video anti-shake processing module configured to deform the video based on the difference between the smooth track and the moving track to obtain a video that has undergone anti-shake processing.
  • embodiments of the present disclosure further provide an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device is made to The device implements any of the video anti-shake processing methods provided in the embodiments of the present disclosure.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a computing device, the computing device enables the computing device to implement the embodiment of the present disclosure Any of the provided video anti-shake processing methods.
  • a computer program product comprising computer program instructions that cause a computer to perform a method as in the first aspect or implementations thereof.
  • a computer program causing a computer to perform the method as in the first aspect or implementations thereof.
  • the technical solutions provided by the embodiments of the present disclosure have at least the following advantages: in the embodiments of the present disclosure, the initial change amount of the shooting position between different image frames in the video is first determined based on the initial transformation method, and then based on the initial change
  • the fitting error corresponding to the fitting error is used to determine the target variation of the shooting position between different image frames in the video by using the target transformation method that matches the fitting error, that is, the fitting error can be used to evaluate whether the selection of the initial transformation method is reasonable.
  • the target change amount of the shooting position between different image frames in the video forms the movement trajectory of the shooting position of the video.
  • the video anti-shake processing effect is realized through the trajectory smoothing processing and video deformation processing.
  • the embodiment of the present disclosure realizes the effect of dynamically determining the target transformation mode between different image frames in the video based on the fitting error corresponding to the initial change of the shooting position between different image frames in the video, and ensures the processing effect of video anti-shake. , which avoids the introduction of excessive fitting errors and effectively improves the video quality.
  • FIG. 1 is a flowchart of a video anti-shake processing method provided by an embodiment of the present disclosure
  • FIG. 2 is a flowchart of another video anti-shake processing method provided by an embodiment of the present disclosure
  • FIG. 3 is a schematic structural diagram of a video anti-shake processing apparatus according to an embodiment of the present disclosure
  • FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 1 is a flowchart of a video anti-shake processing method provided by an embodiment of the present disclosure, which can be applied to a situation of performing anti-shake processing on a video.
  • the method can be performed by a video anti-shake processing apparatus, which can be implemented by software and/or hardware, and can be integrated on any electronic device with computing capability, such as a terminal or a server.
  • the video to be processed may be a video being shot or a video that has been shot, that is, the embodiment of the present disclosure may perform anti-shake processing on the shot video in real time during the video shooting process, or the video may be Anti-shake processing is performed on the video after shooting, which can improve the quality of the video.
  • the video anti-shake processing method may include:
  • matching feature points between different image frames in the video By tracking feature points between different image frames in the video, matching feature points between different image frames can be determined (referring to the feature points of the same object in different image frames, the number of matching feature points can be determined according to the situation) , and then based on the initial transformation method and the matching feature points, the initial variation of the shooting position between different image frames can be determined.
  • the feature point tracking may be implemented with reference to the prior art, which is not specifically limited in the embodiment of the present disclosure.
  • the different image frames in the video may be two adjacent frames of images in the video, or may be images separated by at least two frames, such as the current frame and the first frame of images in the video.
  • the initial transformation method is a calculation method that is used by default and is used to calculate the amount of change in the shooting position between different image frames.
  • the initial transformation method can be implemented by, for example, an initial transformation matrix used to characterize the change of the shooting position, for example, it can be Homography matrix, etc. It should be understood that, in the actual processing process, the initial transformation mode may be flexibly selected from multiple available transformation modes according to processing requirements, which is not specifically limited in the embodiment of the present disclosure.
  • the initial change amount of the shooting position between different image frames may be, for example, the change amount of the shooting position of the following frame image relative to the previous frame image.
  • the initial change amount of the shooting position between different image frames may be a transformation matrix from the previous frame image to the subsequent frame image.
  • the video anti-shake processing method provided by the embodiment of the present disclosure further includes: calculating a fitting corresponding to the initial variation of the shooting position between different image frames based on the initial transformation method and the feature points that are successfully matched between different image frames. error.
  • the fitting error can be used to evaluate whether the selection of the initial transformation method is reasonable, and then determine the influence of the initial transformation method on the anti-shake processing effect in the video anti-shake processing process.
  • the initial transformation method can be used to perform coordinate transformation on the feature points on the previous frame image, or the inverse transformation of the initial transformation method can be used to perform coordinate transformation on the feature points on the subsequent frame image, and then It is compared with the image coordinates of the feature points on the remaining image of another frame, so as to calculate the fitting error corresponding to the initial change of the shooting position between different image frames.
  • calculating the fitting error corresponding to the initial variation of the shooting position between different image frames may include:
  • the fitting error corresponding to the initial variation of the shooting position between different image frames is calculated.
  • the initial transformation method is implemented by using an initial transformation matrix.
  • Different image frames in the video refer to two adjacent frames of images in the video as an example.
  • the motion fitting effect of the image frame is poor, and the initial transformation matrix needs to be dynamically replaced, that is, the initial transformation method needs to be dynamically replaced.
  • the specific calculation method for obtaining the fitting error by using the image coordinates of the feature points on the subsequent frame image and the transformed coordinates of the feature points on the previous frame image can be flexibly determined in actual processing.
  • the image coordinates of the feature points on the subsequent frame image and the transformed coordinates of the feature points on the previous frame image can be calculated according to the corresponding relationship between the feature points on the two frames of images.
  • Each difference value or each quotient value is summed (including weighted summation) to obtain the fitting error corresponding to the initial change of the shooting position between different image frames, and the average value of each difference value or the average value of each quotient value can also be calculated. value, as the fitting error corresponding to the initial variation of the shooting position between different image frames.
  • a target transformation mode matching the fitting error can be determined, so as to improve the processing effect of video anti-shake.
  • the error threshold can be set to a value. At this time, if the fitting error is less than the error threshold, the initial transformation method can be determined as the target transformation method that matches the fitting error.
  • the matching The transformation methods with different degrees of freedom of the initial transformation method are used as the target transformation method to achieve the effect of reducing the fitting error of the shooting position between different image frames;
  • the error threshold can also be hierarchically set to multiple values, and each threshold corresponds to a
  • the degrees of freedom between the multiple optional transformation modes are different, and the fitting errors corresponding to the initial changes of the shooting positions between different image frames obtained by each transformation mode are also different.
  • each threshold mentioned in the implementation of the present disclosure can be flexibly set in the actual processing process, which is not specifically limited in the embodiment of the present disclosure.
  • the error threshold includes a first error threshold and a second error threshold, and the value of the first error threshold is less than the second error threshold, if the fitting error corresponding to the initial variation of the shooting position between different image frames in the video is less than If the first error threshold is set, the initial transformation mode is determined as the target transformation mode that matches the fitting error, and the target transformation mode is used to determine the target change amount of the shooting position between different image frames in the video; or
  • the first transformation with a degree of freedom smaller than the initial transformation is determined as the target transformation that matches the fitting error, and Determine the amount of target change in the shooting position between different image frames in the video by means of target transformation;
  • the second transformation method with a degree of freedom smaller than the first transformation method is determined as the target transformation method matching the fitting error, and the target transformation method is used to determine the video The amount of target change in the shooting position between different image frames in .
  • the initial transformation mode includes a homography transformation mode
  • the first transformation mode includes an affine transformation mode
  • the second transformation mode includes a similarity transformation mode
  • the homography transformation is the transformation relationship from one plane to another plane, with a total of eight degrees of freedom
  • the affine transformation is a linear transformation from two-dimensional coordinates to two-dimensional coordinates, which maintains the two-dimensional graphics.
  • "Flatness” and “parallelism” mainly include translation transformation, rotation transformation, scale transformation, tilt transformation (or called staggered transformation, shear transformation, offset transformation), flip transformation, a total of six degrees of freedom ;
  • staggered transformation, shear transformation, offset transformation or called staggered transformation, shear transformation, offset transformation
  • flip transformation a total of six degrees of freedom
  • affine transformation there is no oblique transformation and flip transformation in similarity transformation, and there are a total of four degrees of freedom.
  • a transformation relationship corresponds to a motion model.
  • a motion model with a higher degree of freedom ie, a transformation matrix with a higher degree of freedom
  • a motion model with a low degree of freedom replaces a motion model with a high degree of freedom, so as to avoid introducing large fitting errors while reducing the smoothing effect, to achieve a balance between motion smoothing and fitting errors between different image frames, and to ensure the final Video stabilization effect.
  • a movement track of the shooting position of the video is formed, wherein the movement track is used to indicate the shooting position of different image frames in the video.
  • one frame of image in the video can be selected as the reference frame image, and the reference frame image can be determined adaptively, and then the target variation of the shooting position between different image frames can be used to obtain the relative value of each frame of image in the video relative to the reference frame.
  • the target change amount of the shooting position of the image, and then the movement trajectory of the shooting position of the video (or the movement trajectory of the shooting device for shooting the video) is obtained based on the multiple target changes.
  • the initial change amount or the target change amount may be represented by a transformation matrix
  • the movement track includes multiple transformation matrices, that is, different transformation matrices in the movement track may respectively represent different image frames in the video.
  • the movement trajectory of the shooting position of the video is formed, including:
  • the movement track of the shooting position of the video is formed.
  • the target transformation matrix of the shooting position between the i-th frame image f i and the i+1-th frame image f i+1 in the video is represented as T i
  • the i-th frame image f i and the different images before the i-th frame can be expressed as T i .
  • the target transformation matrix of the shooting position between the frames (for example, each adjacent two frame images) is accumulated and processed, for example, the cumulative multiplication calculation (specifically can be determined according to the actual processing), and the difference between the i-th frame image f i relative to the first frame image is obtained.
  • the transformation matrix of the shooting position expressed as follows:
  • the movement trajectory of the shooting position of the video is obtained, that is, the shaking trend of the shooting position of the video is determined.
  • Any available smoothing algorithm in the prior art such as Gaussian smoothing algorithm, etc., can be used to shoot different image frames in the moving trajectory.
  • the position is smoothed to obtain a smooth trajectory.
  • the smooth trajectory can be expressed as, for example, Smoothing tracks are used to indicate where different image frames were taken in the smoothed video.
  • n represents the number of image frames included in the video; then, according to the corresponding relationship between each sub-value in the adjustment parameter and each frame of image in the video, the corresponding image frame can be deformed based on each sub-value in the adjustment parameter, so as to obtain anti-shake processed video.
  • the deformation processing process it involves the rotation, translation, scaling or cropping of a specific frame image, which can be performed according to actual processing requirements.
  • the initial change amount of the shooting position between different image frames in the video is first determined based on the initial transformation method, and then based on the fitting error corresponding to the initial change amount, the target transformation method that matches the fitting error is used to determine the video
  • the target change of the shooting position between different image frames in the video that is, the fitting error can be used to evaluate whether the selection of the initial transformation method is reasonable, and then the shooting position of the video is formed based on the target change of the shooting position between different image frames in the video.
  • the embodiment of the present disclosure realizes the effect of dynamically determining the target transformation mode between different image frames in the video based on the fitting error corresponding to the initial change of the shooting position between different image frames in the video, and ensures the processing effect of video anti-shake. , which avoids the introduction of excessive fitting errors and effectively improves the video quality.
  • FIG. 2 is a flowchart of another video anti-shake processing method provided by an embodiment of the present disclosure, which is further optimized and expanded based on the foregoing technical solution, and may be combined with the foregoing optional implementation manners.
  • the video anti-shake processing method may include:
  • the number of successfully matched feature points on these two frame images is represented as M i
  • the i-th frame image f i The j-th feature point on is represented as Then the transformed coordinates corresponding to the jth feature point can be expressed as T i represents the initial change of the shooting position between the i-th frame image f i and the i+1-th frame image f i +1
  • the j-th feature point on the i+1-th frame image f i+1 is expressed as Then the cumulative error corresponding to the initial change of the shooting position between the i-th frame image f i and the i+1-th frame image f i+1 can be expressed as
  • the mean value can be calculated based on the accumulated error and the number of feature points on the previous frame image (equal to the number of successfully matched feature points between different image frames) M i to obtain the shooting position between different image frames.
  • the fitting error E i corresponding to the initial variation of can be expressed as follows:
  • the fitting error corresponding to the initial variation of the shooting position between different image frames is determined, which ensures the accuracy of the fitting error calculation.
  • a movement track of the shooting position of the video is formed, wherein the movement track is used to indicate the shooting position of different image frames in the video.
  • smoothing processing may be performed on the shooting positions of different image frames in the moving track respectively to obtain a smooth track.
  • the value of the preset smoothing radius determines the number of image frames involved in the smoothing process, and its specific value can be flexibly determined in the actual processing process, which is not specifically limited in the embodiment of the present disclosure.
  • an image with a preset number of frames participating in the smoothing process may be determined based on a preset smoothing radius;
  • a weighted sum calculation can be performed on the corresponding shooting positions of the images with a preset number of frames in the moving trajectory to obtain the smoothed shooting position corresponding to each frame of image;
  • the smoothed shooting position of the image is obtained to obtain a smooth trajectory.
  • determining the images of the preset number of frames participating in the smoothing process may include: for each frame of the image in the video, based on the preset smoothing radius, determining the first preset before each frame of the image. Set the previous frame image of the frame number; determine each frame image and the previous frame image of the first preset frame number (the value of which is the value of the preset smoothing radius at this time) as the preset frame participating in the smoothing process or, based on the preset smoothing radius, determine the previous frame image of the second preset number of frames before each frame of image (the value of which is the value of the preset smoothing radius at this time), and determine each frame of image.
  • the subsequent frame images of the second preset frame number after the frame image; each frame image, the previous frame image of the second preset frame number, and the subsequent frame image of the second preset frame number are determined to participate in the smoothing process the preset number of frames of the image.
  • the smoothing of each frame of image f i is taken as an example.
  • the matrix can be represented as follows:
  • r is the preset smoothing radius
  • C t is the corresponding transformation matrix (ie shooting position) of each frame of image participating in the smoothing process in the moving track C
  • w i ⁇ t is the weight of each frame of the image participating in the smoothing process.
  • the value can be set adaptively, which is not specifically limited in the embodiment of the present disclosure.
  • the corresponding image frame can be deformed based on each sub-value in the adjustment parameter, so as to obtain the video after anti-shake processing.
  • the dynamic determination of the available transformation modes between different image frames according to the fitting error is realized, ensuring that The processing effect of video anti-shake avoids the introduction of excessive fitting errors and effectively improves the video quality.
  • FIG. 3 is a schematic structural diagram of a video anti-shake processing apparatus according to an embodiment of the present disclosure, which can be applied to a situation of performing anti-shake processing on a video.
  • the apparatus can be implemented by software and/or hardware, and can be integrated on any electronic device with computing capability, such as a terminal or a server.
  • the video anti-shake processing apparatus 300 may include an initial change amount determination module 301, a target change amount determination module 302, a movement trajectory generation module 303, a smooth trajectory determination module 304, and video anti-shake processing.
  • Module 305 where:
  • the initial change amount determination module 301 is used to determine the initial change amount of the shooting position between different image frames in the video based on the initial transformation method by performing feature point tracking between different image frames in the video;
  • the target change amount determination module 302 is used for determining the target change amount of the shooting position between different image frames in the video based on the fitting error corresponding to the initial change amount by adopting a target transformation method that matches the fitting error;
  • the movement trajectory generation module 303 is used to form the movement trajectory of the shooting position of the video based on the target variation of the shooting position between different image frames in the video, wherein the movement trajectory is used to indicate the shooting position of different image frames in the video;
  • the smooth trajectory determination module 304 is configured to perform smoothing processing on the shooting positions of different image frames in the moving trajectory respectively to obtain a smooth trajectory;
  • the video anti-shake processing module 305 is configured to deform the video based on the difference between the smooth track and the moving track, so as to obtain a video that has undergone anti-shake processing.
  • the video anti-shake processing apparatus 300 provided in this embodiment of the present disclosure further includes:
  • the transformation coordinate determination module is used to perform coordinate transformation on the feature points on the previous frame image in different image frames by using the initial transformation method, so as to obtain the transformed coordinates of the feature points on the previous frame image;
  • the fitting error calculation module is used to calculate the corresponding value of the initial change of the shooting position between different image frames based on the image coordinates of the feature points on the subsequent frame image and the transformed coordinates of the feature points on the previous frame image in different image frames. Fitting error.
  • the fitting error calculation module includes:
  • the cumulative error calculation unit is used to calculate the cumulative corresponding to the initial change of the shooting position between different image frames by using the image coordinates of the feature points on the subsequent frame image and the transformed coordinates of the feature points on the previous frame image in different image frames error;
  • the fitting error calculation unit is configured to calculate the fitting error corresponding to the initial variation of the shooting position between different image frames based on the accumulated error and the number of feature points on the previous frame image.
  • the target variation determination module 302 includes:
  • a first determining unit configured to determine the initial transformation mode as a target transformation mode matching the fitting error if the fitting error corresponding to the initial change amount is smaller than the first error threshold, and use the target transformation mode to determine different image frames in the video The amount of target change between shooting positions; or
  • a second determining unit configured to determine the first transformation mode with the degree of freedom smaller than the initial transformation mode as the fitting error if the fitting error corresponding to the initial change amount is greater than or equal to the first error threshold and less than the second error threshold
  • the matching target transformation method, and the target transformation method is used to determine the target change amount of the shooting position between different image frames in the video;
  • a second determining unit configured to determine a second transformation mode with a degree of freedom smaller than the first transformation mode as a target transformation mode matching the fitting error if the fitting error corresponding to the initial change amount is greater than or equal to the second error threshold, And adopt the target transformation method to determine the target change amount of the shooting position between different image frames in the video.
  • the initial transformation mode includes a homography transformation mode
  • the first transformation mode includes an affine transformation mode
  • the second transformation mode includes a similarity transformation mode
  • the initial change amount or the target change amount is represented by a transformation matrix
  • the movement trajectory generation module 303 includes:
  • a transformation matrix determining unit for determining the transformation matrix of each frame of image relative to the first frame of image in the video based on the target transformation matrix of the shooting position between different image frames in the video;
  • the movement trajectory generation unit is configured to form the movement trajectory of the shooting position of the video based on the transformation matrix of each frame of image in the video relative to the first frame of image.
  • the video anti-shake processing module 305 includes:
  • an adjustment parameter determination unit for determining an adjustment parameter based on the difference between the smooth trajectory and the moving trajectory
  • the video deformation unit is used to deform the video by using the adjustment parameters to obtain the video that has undergone anti-shake processing.
  • the video anti-shake processing apparatus provided by the embodiment of the present disclosure can execute any video anti-shake processing method provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • any video anti-shake processing method provided by the embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
  • FIG. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, which is used to exemplarily describe an electronic device that implements the video anti-shake processing method provided by the embodiment of the present disclosure.
  • the electronic devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, Mobile terminals such as car navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, smart home devices, wearable electronic devices, servers, and the like.
  • the electronic device shown in FIG. 4 is only an example, and should not impose any limitation on the functions and occupancy scope of the embodiments of the present disclosure.
  • electronic device 400 includes one or more processors 401 and memory 402 .
  • Processor 401 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 400 to perform desired functions.
  • CPU central processing unit
  • Processor 401 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in electronic device 400 to perform desired functions.
  • Memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Volatile memory may include, for example, random access memory (RAM) and/or cache memory, among others.
  • Non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the video anti-shake processing method provided by the embodiments of the present disclosure, and may also implement other desired functions.
  • Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
  • the video anti-shake processing method may include: by tracking feature points between different image frames in the video, determining the initial change amount of the shooting position between different image frames in the video based on an initial transformation method; The fitting error corresponding to the initial variation is determined by the target transformation method that matches the fitting error to determine the target variation of the shooting position between different image frames in the video; based on the target variation of the shooting position between different image frames in the video, the The moving track of the shooting position of the video, wherein the moving track is used to indicate the shooting position of different image frames in the video; the shooting positions of different image frames in the moving track are respectively smoothed to obtain a smooth track; based on the difference between the smooth track and the moving track The difference between the video and the video is deformed to obtain an anti-shake video.
  • the electronic device 400 may also perform other optional implementations provided by the method embodiments of the present disclosure.
  • the electronic device 400 may also include an input device 403 and an output device 404 interconnected by a bus system and/or other form of connection mechanism (not shown).
  • the input device 403 may also include, for example, a keyboard, a mouse, and the like.
  • the output device 404 can output various information to the outside, including the determined distance information, direction information, and the like.
  • the output devices 404 may include, for example, displays, speakers, printers, and communication networks and their connected remote output devices, among others.
  • the electronic device 400 may also include any other appropriate components according to the specific application.
  • the embodiments of the present disclosure may also be computer program products, which include computer programs or computer program instructions, which, when executed by a processor, cause a computing device to implement what is provided by the embodiments of the present disclosure. Any video anti-shake processing method.
  • the computer program product may write program code for performing operations of embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages, such as Java, C++, etc., as well as conventional procedural programming language, such as "C" language or similar programming language.
  • the program code may execute entirely on the user electronic device, partly on the user electronic device, as a stand-alone software package, partly on the user electronic device and partly on the remote electronic device, or entirely on the remote electronic device execute on.
  • embodiments of the present disclosure may further provide a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by the processor, the computer program instructions enable the computing device to implement any video anti-shake processing provided by the embodiments of the present disclosure method.
  • the video anti-shake processing method may include: by tracking feature points between different image frames in the video, determining the initial change amount of the shooting position between different image frames in the video based on an initial transformation method; The fitting error corresponding to the initial variation is determined by the target transformation method that matches the fitting error to determine the target variation of the shooting position between different image frames in the video; based on the target variation of the shooting position between different image frames in the video, the The moving track of the shooting position of the video, wherein the moving track is used to indicate the shooting position of different image frames in the video; the shooting positions of different image frames in the moving track are respectively smoothed to obtain a smooth track; based on the difference between the smooth track and the moving track
  • the video is deformed to obtain an anti-shake processed video.
  • a computer-readable storage medium can employ any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • a readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

La présente divulgation concerne, selon des modes de réalisation, un procédé et un appareil de traitement anti-tremblement de vidéo, un dispositif électronique et un support de stockage. Le procédé consiste : au moyen de la réalisation d'un suivi de point caractéristique entre différentes trames d'image dans une vidéo, et en fonction d'un mode de transformation initial, à déterminer une quantité initiale de variation dans des positions d'enregistrement vidéo par rapport à différentes trames d'image dans la vidéo; en fonction d'une erreur d'ajustement correspondant à la quantité initiale de variation, à utiliser un mode de transformation cible correspondant à l'erreur d'ajustement afin de déterminer la quantité cible de variation dans les positions d'enregistrement vidéo par rapport à différentes trames d'image dans la vidéo; et en fonction de la quantité cible de variation, à former une trajectoire de mouvement par rapport aux positions d'enregistrement vidéo de la vidéo; à effectuer un lissage sur chacune des positions d'enregistrement vidéo par rapport à différentes trames d'image dans la trajectoire de mouvement; et, en fonction de la différence entre une trajectoire lisse et la trajectoire de mouvement, à réaliser une mise en forme sur la vidéo pour obtenir une vidéo anti-tremblement traitée. Dans les modes de réalisation de la présente divulgation, le mode de transformation cible par rapport à différentes trames d'image dans la vidéo est déterminé de manière dynamique en fonction de l'erreur d'ajustement, assurant l'effet de traitement anti-tremblement vidéo, et évitant l'introduction d'erreurs d'ajustement excessivement importantes.
PCT/CN2022/085634 2021-04-08 2022-04-07 Procédé et appareil de traitement anti-tremblement de vidéo, dispositif électronique et support de stockage WO2022214037A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/472,001 US20240013347A1 (en) 2021-04-08 2023-09-21 Method for video anti-shake processing, electronic apparatus, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110379651.6 2021-04-08
CN202110379651.6A CN115209031B (zh) 2021-04-08 2021-04-08 视频防抖处理方法、装置、电子设备和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/085382 Continuation-In-Part WO2022214001A1 (fr) 2021-04-08 2022-04-06 Procédé et appareil de stabilisation d'image vidéo, dispositif électronique et support d'enregistrement

Publications (1)

Publication Number Publication Date
WO2022214037A1 true WO2022214037A1 (fr) 2022-10-13

Family

ID=83545102

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/085634 WO2022214037A1 (fr) 2021-04-08 2022-04-07 Procédé et appareil de traitement anti-tremblement de vidéo, dispositif électronique et support de stockage

Country Status (2)

Country Link
CN (1) CN115209031B (fr)
WO (1) WO2022214037A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988317A (zh) * 2022-12-12 2023-04-18 Oppo广东移动通信有限公司 图像处理方法、装置、芯片、通信设备、及存储介质
CN116506732A (zh) * 2023-06-26 2023-07-28 浙江华诺康科技有限公司 一种图像抓拍防抖的方法、装置、系统和计算机设备
CN116614706A (zh) * 2023-05-19 2023-08-18 北京源音文创科技股份有限公司 一种旋转拍摄中画面实时自适应保持水平的方法和装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116112761B (zh) * 2023-04-12 2023-06-27 海马云（天津）信息技术有限公司 生成虚拟形象视频的方法及装置、电子设备和存储介质
CN116797497B (zh) * 2023-08-24 2023-11-14 摩尔线程智能科技(北京)有限责任公司 一种图像处理方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050275727A1 (en) * 2004-06-15 2005-12-15 Shang-Hong Lai Video stabilization method
CN103139568A (zh) * 2013-02-05 2013-06-05 上海交通大学 基于稀疏度和保真度约束的视频稳像方法
CN103813099A (zh) * 2013-12-13 2014-05-21 中山大学深圳研究院 一种基于特征点匹配的视频防抖方法
US20140362240A1 (en) * 2013-06-07 2014-12-11 Apple Inc. Robust Image Feature Based Video Stabilization and Smoothing
CN105450907A (zh) * 2014-07-31 2016-03-30 北京展讯高科通信技术有限公司 智能终端及其视频稳像系统模型参数的标定方法及装置
CN110047091A (zh) * 2019-03-14 2019-07-23 河海大学 一种基于相机轨迹估计和特征块匹配的稳像方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014075022A1 (fr) * 2012-11-12 2014-05-15 Behavioral Recognition Systems, Inc. Techniques de stabilisation d'image destinées à des systèmes de surveillance vidéo
US9854168B2 (en) * 2014-03-07 2017-12-26 Futurewei Technologies, Inc. One-pass video stabilization
US10074015B1 (en) * 2015-04-13 2018-09-11 Google Llc Methods, systems, and media for generating a summarized video with video thumbnails
CN105049678B (zh) * 2015-08-17 2018-08-03 成都鹰眼视觉科技有限公司 一种基于绕环自适应相机路径优化的视频防抖方法
CN105791705B (zh) * 2016-05-26 2019-06-11 厦门美图之家科技有限公司 适用于移动式延时摄影的视频防抖方法、系统及拍摄终端
CN106878612B (zh) * 2017-01-05 2019-05-31 中国电子科技集团公司第五十四研究所 一种基于在线全变差优化的视频稳定方法
CN109089015B (zh) * 2018-09-19 2020-12-22 厦门美图之家科技有限公司 视频防抖显示方法及装置
CN110415186B (zh) * 2019-07-05 2021-07-20 浙江大华技术股份有限公司 一种图像去抖动的方法和设备
CN110753181A (zh) * 2019-09-29 2020-02-04 湖北工业大学 一种基于特征跟踪和网格路径运动的视频稳像方法
CN111225155B (zh) * 2020-02-21 2021-09-28 Oppo广东移动通信有限公司 视频防抖方法、装置、电子设备、计算机设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050275727A1 (en) * 2004-06-15 2005-12-15 Shang-Hong Lai Video stabilization method
CN103139568A (zh) * 2013-02-05 2013-06-05 上海交通大学 基于稀疏度和保真度约束的视频稳像方法
US20140362240A1 (en) * 2013-06-07 2014-12-11 Apple Inc. Robust Image Feature Based Video Stabilization and Smoothing
CN103813099A (zh) * 2013-12-13 2014-05-21 中山大学深圳研究院 一种基于特征点匹配的视频防抖方法
CN105450907A (zh) * 2014-07-31 2016-03-30 北京展讯高科通信技术有限公司 智能终端及其视频稳像系统模型参数的标定方法及装置
CN110047091A (zh) * 2019-03-14 2019-07-23 河海大学 一种基于相机轨迹估计和特征块匹配的稳像方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988317A (zh) * 2022-12-12 2023-04-18 Oppo广东移动通信有限公司 图像处理方法、装置、芯片、通信设备、及存储介质
CN116614706A (zh) * 2023-05-19 2023-08-18 北京源音文创科技股份有限公司 一种旋转拍摄中画面实时自适应保持水平的方法和装置
CN116506732A (zh) * 2023-06-26 2023-07-28 浙江华诺康科技有限公司 一种图像抓拍防抖的方法、装置、系统和计算机设备
CN116506732B (zh) * 2023-06-26 2023-12-05 浙江华诺康科技有限公司 一种图像抓拍防抖的方法、装置、系统和计算机设备

Also Published As

Publication number Publication date
CN115209031B (zh) 2024-03-29
CN115209031A (zh) 2022-10-18

Similar Documents

Publication Publication Date Title
WO2022214037A1 (fr) Procédé et appareil de traitement anti-tremblement de vidéo, dispositif électronique et support de stockage
CN108694700B (zh) 用于深度学习图像超分辨率的系统和方法
CN109344755B (zh) 视频动作的识别方法、装置、设备及存储介质
US9697587B2 (en) Adaptive path smoothing for video stabilization
WO2021114990A1 (fr) Procédé et appareil permettant de corriger une distorsion de visage, dispositif électronique et support de stockage
US9953400B2 (en) Adaptive path smoothing for video stabilization
WO2020140976A1 (fr) Procédé d'acquisition d'image, dispositif, dispositif de lecture en mode point, dispositif électronique, et support de stockage
WO2019242271A1 (fr) Procédé et appareil de déformation d'image, et dispositif électronique
WO2018050128A1 (fr) Procédé de suivi de cible, dispositif électronique et support d'informations
WO2023035531A1 (fr) Procédé de reconstruction à super-résolution pour image de texte et dispositif associé
WO2013087003A1 (fr) Procédé et dispositif de reconstruction de super-résolution basée sur la non-localité
CN110956131A (zh) 单目标追踪方法、装置及系统
US8873811B2 (en) Method and apparatus for face tracking utilizing integral gradient projections
WO2018058476A1 (fr) Procédé et dispositif de correction d'image
WO2022214001A1 (fr) Procédé et appareil de stabilisation d'image vidéo, dispositif électronique et support d'enregistrement
CN115705651A (zh) 视频运动估计方法、装置、设备和计算机可读存储介质
CN111966473B (zh) 一种线性回归任务的运行方法及装置、电子设备
KR102697687B1 (ko) 이미지 병합 방법 및 이를 수행하는 데이터 처리 장치
CN116129228B (zh) 图像匹配模型的训练方法、图像匹配方法及其装置
WO2024021504A1 (fr) Procédé et appareil d'entraînement de modèle de reconnaissance faciale, procédé de reconnaissance et dispositif et support
CN116089788B (zh) 在线缺失数据处理方法、装置、计算机设备及存储介质
CN110223220B (zh) 一种处理图像的方法和装置
US8872832B2 (en) System and method for mesh stabilization of facial motion capture data
CN116012418A (zh) 多目标跟踪方法及装置
WO2022222922A1 (fr) Procédé et appareil de traitement de signal vocal

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22784103

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22784103

Country of ref document: EP

Kind code of ref document: A1