WO2021103653A1 - Procédé de synthèse d'audio et de vidéo, terminal, et support de stockage - Google Patents

Procédé de synthèse d'audio et de vidéo, terminal, et support de stockage Download PDF

Info

Publication number
WO2021103653A1
WO2021103653A1 PCT/CN2020/108270 CN2020108270W WO2021103653A1 WO 2021103653 A1 WO2021103653 A1 WO 2021103653A1 CN 2020108270 W CN2020108270 W CN 2020108270W WO 2021103653 A1 WO2021103653 A1 WO 2021103653A1
Authority
WO
WIPO (PCT)
Prior art keywords
action
audio
feature
video
target audio
Prior art date
Application number
PCT/CN2020/108270
Other languages
English (en)
Chinese (zh)
Inventor
杜昊燃
Original Assignee
北京达佳互联信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京达佳互联信息技术有限公司 filed Critical 北京达佳互联信息技术有限公司
Publication of WO2021103653A1 publication Critical patent/WO2021103653A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81—Monomedia components thereof

Definitions

  • the inventor realizes that it is necessary to design a new method for producing MV based on video and audio synthesis technology to overcome the above-mentioned shortcomings.
  • the corresponding image material is obtained.
  • parsing the target audio specifically includes:
  • the processor is configured to read and execute executable instructions stored in the memory to implement the video and audio synthesis method described in any one of the above.
  • FIG. 1 is a schematic flowchart of a video and audio synthesis method in an embodiment of the disclosure
  • the video and audio synthesis method provided by the embodiments of the present disclosure can be applied to any APP for making creative MV works, such as MV Master, etc., or it can be used as an MV function component and added to an APP in the existing technology. .
  • the video and audio synthesis method proposed in the embodiments of the present disclosure can also be used to synthesize other types of short videos except MV.
  • the material types may include, but are not limited to, pictures, stickers, watermarks, and filters.
  • pictures may include, but are not limited to, pictures, stickers, watermarks, and filters.
  • only the acquired image materials are pictures and stickers as examples for description.
  • the acquired image materials are picture 1 and sticker 1.
  • sticker 1 and picture 1 are selected as the controlled objects.
  • the action type includes various executable actions. For example, for pictures, it can be jitter, flicker, zoom-in and zoom-in, zoom-out and zoom-out, parallel sliding out of the screen and resetting, tilting and resetting, recovery after the blinds disappeared, rotating out of the screen, and many other types of actions, generally ,
  • the action frequency is consistent or corresponding to the rhythm feature in the audio feature. Or you can divide the picture into multiple parts and perform the corresponding actions in sequence.
  • sticker 2 For another example, for the rhythm characteristics of song 2, for sticker 2, set sticker 2 to set the sticker to be a large wave of water ripples in each strong shot, and set sticker 2 to be a small water ripple in each down shot. The amplitude is jittered once. For picture 2, the picture 2 is divided into multiple small pictures, and each small picture sequentially performs the action of zooming in and restoring at each beat.
  • the total duration of Song 2 For another example, set the total duration of Song 2 to 4 minutes and 50 seconds.
  • Song 2 When Song 2 is played to the first minute and 30 seconds, a picture flashes for each beat in the first 1-3 seconds. In 4-10 seconds, the picture 2 Divide into multiple small pictures, and each small picture performs the action of zooming in and restoring in each beat.
  • the user has determined multiple character pictures and a water ripple effect sticker as the image material.
  • the analyzed rhythm feature is 2/4 beats
  • the loudness feature is 100 -150 decibels
  • set the sticker to jitter once in a water ripple state at each upbeat and set the sticker to jitter once in a small water ripple state at each downbeat, and produce
  • the water ripples present colorful color changes; for pictures, according to the extracted rhythm characteristics, a picture flashes for each beat in the first 1-3 seconds, and in 4-10 seconds, a picture is divided into multiple Small pictures, each small picture performs the action of zooming in and restoring in each beat, for example, refer to Figure 5a and Figure 5b, Figure 5a and Figure 5b are one of the small pictures after being segmented, and there are three people on the small picture , Where the m point is a fixed position of the small picture on
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Circuits (AREA)

Abstract

La présente invention se rapporte au domaine du traitement de vidéo, et en particulier, elle concerne un procédé de synthèse de vidéo et d'audio, un terminal et un support de stockage, et résout le problème technique d'un mauvais effet de correspondance entre la vidéo et l'audio. Le procédé comprend les étapes suivantes : analyser de l'audio cible pour obtenir une caractéristique d'audio correspondant à l'audio cible, et définir un paramètre d'action correspondant en fonction de la caractéristique d'audio ; commander le matériau d'image pour exécuter une action correspondant au paramètre d'action, et produire une image dynamique ; et synthétiser l'image dynamique et l'audio cible pour obtenir une vidéo cible. L'effet de rythme de MV produit par le procédé est bon, et l'expérience de l'utilisateur est améliorée.
PCT/CN2020/108270 2019-11-29 2020-08-10 Procédé de synthèse d'audio et de vidéo, terminal, et support de stockage WO2021103653A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911207438.6A CN110798737A (zh) 2019-11-29 2019-11-29 视音频合成方法、终端和存储介质
CN201911207438.6 2019-11-29

Publications (1)

Publication Number Publication Date
WO2021103653A1 true WO2021103653A1 (fr) 2021-06-03

Family

ID=69447097

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/108270 WO2021103653A1 (fr) 2019-11-29 2020-08-10 Procédé de synthèse d'audio et de vidéo, terminal, et support de stockage

Country Status (2)

Country Link
CN (1) CN110798737A (fr)
WO (1) WO2021103653A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110798737A (zh) * 2019-11-29 2020-02-14 北京达佳互联信息技术有限公司 视音频合成方法、终端和存储介质
CN111782858B (zh) * 2020-03-31 2024-04-05 北京沃东天骏信息技术有限公司 音乐匹配的方法和装置
CN113592986B (zh) * 2021-01-14 2023-05-23 腾讯科技（深圳）有限公司 基于神经网络的动作生成方法、装置及计算设备
CN113784196B (zh) * 2021-11-11 2022-02-08 深圳市速点网络科技有限公司 一种视频效果元素自动律动展示方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103165152A (zh) * 2011-12-14 2013-06-19 联想(北京)有限公司 播放多媒体文件的方法及装置
CN104144280A (zh) * 2013-05-08 2014-11-12 上海恺达广告有限公司 电子贺卡的语音动作动画同步控制及装置
CN104683781A (zh) * 2013-11-26 2015-06-03 深圳市快播科技有限公司 视频播放处理方法及装置
CN104732593A (zh) * 2015-03-27 2015-06-24 厦门幻世网络科技有限公司 一种基于移动终端的3d动画编辑方法
US20190287553A1 (en) * 2018-03-18 2019-09-19 Christopher Griffin Byerly Automatic phonographic record playing and archiving device, system and method
CN110798737A (zh) * 2019-11-29 2020-02-14 北京达佳互联信息技术有限公司 视音频合成方法、终端和存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107329980B (zh) * 2017-05-31 2022-04-12 福建星网视易信息系统有限公司 一种基于音频的实时联动显示方法及存储设备
CN108989706A (zh) * 2017-06-02 2018-12-11 北京字节跳动网络技术有限公司 基于音乐节奏生成特效的方法及装置
CN107360383B (zh) * 2017-07-26 2019-07-30 北京百思科技有限公司 一种自动生成视频的方法及系统
CN108322802A (zh) * 2017-12-29 2018-07-24 广州市百果园信息技术有限公司 视频图像的贴图处理方法、计算机可读存储介质及终端
CN109168026B (zh) * 2018-10-25 2020-09-29 北京字节跳动网络技术有限公司 即时视频显示方法、装置、终端设备及存储介质
CN109495767A (zh) * 2018-11-29 2019-03-19 百度在线网络技术（北京）有限公司 用于输出信息的方法和装置
CN110233976B (zh) * 2019-06-21 2022-09-09 广州酷狗计算机科技有限公司 视频合成的方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103165152A (zh) * 2011-12-14 2013-06-19 联想(北京)有限公司 播放多媒体文件的方法及装置
CN104144280A (zh) * 2013-05-08 2014-11-12 上海恺达广告有限公司 电子贺卡的语音动作动画同步控制及装置
CN104683781A (zh) * 2013-11-26 2015-06-03 深圳市快播科技有限公司 视频播放处理方法及装置
CN104732593A (zh) * 2015-03-27 2015-06-24 厦门幻世网络科技有限公司 一种基于移动终端的3d动画编辑方法
US20190287553A1 (en) * 2018-03-18 2019-09-19 Christopher Griffin Byerly Automatic phonographic record playing and archiving device, system and method
CN110798737A (zh) * 2019-11-29 2020-02-14 北京达佳互联信息技术有限公司 视音频合成方法、终端和存储介质

Also Published As

Publication number Publication date
CN110798737A (zh) 2020-02-14

Similar Documents

Publication Publication Date Title
WO2021103653A1 (fr) Procédé de synthèse d'audio et de vidéo, terminal, et support de stockage
JP7134248B2 (ja) ビデオ作成方法並びにその装置、コンピュータ機器、記憶媒体、及びコンピュータプログラム
US11392642B2 (en) Image processing method, storage medium, and computer device
CN107613357B (zh) 声画同步优化方法、装置及可读存储介质
US7952535B2 (en) Electronic visual jockey file
WO2020107297A1 (fr) Procédé de commande de découpage vidéo, dispositif terminal, système
JP6942300B2 (ja) コンピュータグラフィックスのプログラム、表示装置、送信装置、受信装置、動画生成装置、データ変換装置、データ生成装置、情報処理方法及び情報処理システム
WO2020029523A1 (fr) Procédé et appareil de génération de vidéos, dispositif électronique et support de stockage
JP6936298B2 (ja) 三次元仮想ポートレートの口形の変化を制御する方法および装置
US20060268121A1 (en) In-camera cinema director
KR20220103110A (ko) 비디오 생성 장치 및 방법, 전자 장치, 및 컴퓨터 판독가능 매체
KR20210082232A (ko) 실시간 비디오 특수 효과 시스템 및 방법
US20200120400A1 (en) Method and system for generating interactive media content
WO2018085982A1 (fr) Procédé et appareil d'enregistrement vidéo, et dispositif de prise de vues
JP2023554470A (ja) ビデオ処理方法、装置、機器、記憶媒体、及びコンピュータプログラム製品
WO2018014518A1 (fr) Procédé et appareil de traitement de photographie
CN113704390A (zh) 虚拟对象的交互方法、装置、计算机可读介质及电子设备
JP2010049406A (ja) 絵本画像再生装置、絵本画像再生方法、絵本画像再生プログラム及び記録媒体
WO2023182937A2 (fr) Procédé et appareil de détermination de vidéo à effet spécial, dispositif électronique et support de stockage
JP6217221B2 (ja) コンテンツ再生方法、装置及びプログラム
CN114025103A (zh) 视频制作方法及装置
CN112565873A (zh) 屏幕录制方法和装置、设备及存储介质
JP5551403B2 (ja) 動画作成装置、コンピュータプログラム及び記憶媒体
JP2009272846A (ja) 画像処理装置、画像処理方法およびプログラム
JP2009296504A (ja) 画像処理装置、画像処理方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20893284

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20893284

Country of ref document: EP

Kind code of ref document: A1