JP7450623B2 - 機械学習のための動的視聴覚セグメント・パディング - Google Patents

機械学習のための動的視聴覚セグメント・パディング Download PDF

Info

Publication number
JP7450623B2
JP7450623B2 JP2021535241A JP2021535241A JP7450623B2 JP 7450623 B2 JP7450623 B2 JP 7450623B2 JP 2021535241 A JP2021535241 A JP 2021535241A JP 2021535241 A JP2021535241 A JP 2021535241A JP 7450623 B2 JP7450623 B2 JP 7450623B2
Authority
JP
Japan
Prior art keywords
segment
padding
audiovisual
audiovisual segment
unpadded
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2021535241A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022521120A5 (https=
JP2022521120A (ja
Inventor
ボウマン、アーロン
ハンマー、ステファン
キャノン、グレイ
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of JP2022521120A publication Critical patent/JP2022521120A/ja
Publication of JP2022521120A5 publication Critical patent/JP2022521120A5/ja
Application granted granted Critical
Publication of JP7450623B2 publication Critical patent/JP7450623B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/088—Non-supervised learning, e.g. competitive learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0475—Generative networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G06V20/47—Detecting features for summarising video content
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G11B27/036—Insert-editing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/251—Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8549—Creating video summaries, e.g. movie trailer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Signal Processing (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computer Security & Cryptography (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Television Signal Processing For Recording (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Electrically Operated Instructional Devices (AREA)
JP2021535241A 2019-02-25 2020-02-25 機械学習のための動的視聴覚セグメント・パディング Active JP7450623B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/283,912 US10832734B2 (en) 2019-02-25 2019-02-25 Dynamic audiovisual segment padding for machine learning
US16/283,912 2019-02-25
PCT/IB2020/051586 WO2020174383A1 (en) 2019-02-25 2020-02-25 Dynamic audiovisual segment padding for machine learning

Publications (3)

Publication Number Publication Date
JP2022521120A JP2022521120A (ja) 2022-04-06
JP2022521120A5 JP2022521120A5 (https=) 2022-06-06
JP7450623B2 true JP7450623B2 (ja) 2024-03-15

Family

ID=72143024

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021535241A Active JP7450623B2 (ja) 2019-02-25 2020-02-25 機械学習のための動的視聴覚セグメント・パディング

Country Status (5)

Country Link
US (2) US10832734B2 (https=)
JP (1) JP7450623B2 (https=)
CN (1) CN113439277B (https=)
GB (1) GB2596463B (https=)
WO (1) WO2020174383A1 (https=)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10824487B2 (en) 2018-07-17 2020-11-03 Xandr Inc. Real-time data processing pipeline and pacing control systems and methods
US10997464B2 (en) * 2018-11-09 2021-05-04 Adobe Inc. Digital image layout training using wireframe rendering within a generative adversarial network (GAN) system
US10832734B2 (en) * 2019-02-25 2020-11-10 International Business Machines Corporation Dynamic audiovisual segment padding for machine learning
WO2021162935A1 (en) 2020-02-13 2021-08-19 Stats Llc Dynamically predicting shot type using a personalized deep neural network
GB2616012A (en) * 2022-02-23 2023-08-30 Sony Group Corp A method, apparatus and computer program for generating sports game highlight video based on excitement of gameplay
US12093883B2 (en) * 2022-03-10 2024-09-17 International Business Machines Corporation Automated delivery coordination and meeting scheduling for multiple-recipient orders received at a computer system
CN118055199A (zh) 2022-11-17 2024-05-17 北京字跳网络技术有限公司 视频剪辑方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050185918A1 (en) 2002-10-04 2005-08-25 Frederick Lowe System and method for generating and distributing personalized media
JP2006058874A (ja) 2004-08-20 2006-03-02 Mitsubishi Electric Research Laboratories Inc マルチメディア中の事象を検出する方法
JP2008529338A (ja) 2005-01-20 2008-07-31 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ プロダクトプレイスメントを含むトレイラの自動生成
US20140321831A1 (en) 2013-04-26 2014-10-30 Microsoft Corporation Video service with automated video timeline curation
JP2015527668A (ja) 2012-09-25 2015-09-17 インテル コーポレイション 閲覧者の反応推定及びビジュアル・キュー検出によるビデオ・インデクシング

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5327518A (en) 1991-08-22 1994-07-05 Georgia Tech Research Corporation Audio analysis/synthesis system
US5793888A (en) 1994-11-14 1998-08-11 Massachusetts Institute Of Technology Machine learning apparatus and method for image searching
CA2167748A1 (en) 1995-02-09 1996-08-10 Yoav Freund Apparatus and methods for machine learning hypotheses
US5596159A (en) 1995-11-22 1997-01-21 Invision Interactive, Inc. Software sound synthesis system
US6266068B1 (en) 1998-03-13 2001-07-24 Compaq Computer Corporation Multi-layer image-based rendering for video synthesis
US6513025B1 (en) 1999-12-09 2003-01-28 Teradyne, Inc. Multistage machine learning process
US7024033B2 (en) 2001-12-08 2006-04-04 Microsoft Corp. Method for boosting the performance of machine-learning classifiers
US20030131362A1 (en) 2002-01-09 2003-07-10 Koninklijke Philips Electronics N.V. Method and apparatus for multimodal story segmentation for linking multimedia content
US8326775B2 (en) 2005-10-26 2012-12-04 Cortica Ltd. Signature generation for multimedia deep-content-classification by a large-scale matching system and method thereof
US9218606B2 (en) 2005-10-26 2015-12-22 Cortica, Ltd. System and method for brand monitoring and trend analysis based on deep-content-classification
US9047374B2 (en) * 2007-06-08 2015-06-02 Apple Inc. Assembling video content
US8207989B2 (en) 2008-12-12 2012-06-26 Microsoft Corporation Multi-video synthesis
US10269390B2 (en) 2015-06-11 2019-04-23 David M. DeCaprio Game video processing systems and methods
EP3475920A4 (en) 2016-06-23 2020-01-15 Loomai, Inc. SYSTEMS AND METHODS FOR GENERATING COMPUTER-READY ANIMATION MODELS OF A HUMAN HEAD FROM IMAGES OF DETECTED DATA
CN107707931B (zh) 2016-08-08 2021-09-10 阿里巴巴集团控股有限公司 根据视频数据生成解释数据、数据合成方法及装置、电子设备
US11024009B2 (en) 2016-09-15 2021-06-01 Twitter, Inc. Super resolution using a generative adversarial network
US10074038B2 (en) 2016-11-23 2018-09-11 General Electric Company Deep learning medical systems and methods for image reconstruction and quality evaluation
US10043109B1 (en) 2017-01-23 2018-08-07 A9.Com, Inc. Attribute similarity-based search
US10474881B2 (en) 2017-03-15 2019-11-12 Nec Corporation Video retrieval system based on larger pose face frontalization
CN107464210B (zh) 2017-07-06 2020-02-21 浙江工业大学 一种基于生成式对抗网络的图像风格迁移方法
CN107483843B (zh) * 2017-08-16 2019-11-15 成都品果科技有限公司 音视频匹配剪辑方法及装置
CN108256627A (zh) 2017-12-29 2018-07-06 中国科学院自动化研究所 视听信息互生装置及其基于循环对抗生成网络的训练系统
US10635939B2 (en) * 2018-07-06 2020-04-28 Capital One Services, Llc System, method, and computer-accessible medium for evaluating multi-dimensional synthetic data using integrated variants analysis
US10832734B2 (en) 2019-02-25 2020-11-10 International Business Machines Corporation Dynamic audiovisual segment padding for machine learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050185918A1 (en) 2002-10-04 2005-08-25 Frederick Lowe System and method for generating and distributing personalized media
JP2006058874A (ja) 2004-08-20 2006-03-02 Mitsubishi Electric Research Laboratories Inc マルチメディア中の事象を検出する方法
JP2008529338A (ja) 2005-01-20 2008-07-31 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ プロダクトプレイスメントを含むトレイラの自動生成
JP2015527668A (ja) 2012-09-25 2015-09-17 インテル コーポレイション 閲覧者の反応推定及びビジュアル・キュー検出によるビデオ・インデクシング
US20140321831A1 (en) 2013-04-26 2014-10-30 Microsoft Corporation Video service with automated video timeline curation

Also Published As

Publication number Publication date
WO2020174383A1 (en) 2020-09-03
CN113439277A (zh) 2021-09-24
US10832734B2 (en) 2020-11-10
US20210012809A1 (en) 2021-01-14
JP2022521120A (ja) 2022-04-06
US11521655B2 (en) 2022-12-06
CN113439277B (zh) 2025-08-22
GB2596463B (en) 2022-05-11
GB2596463A (en) 2021-12-29
US20200273495A1 (en) 2020-08-27

Similar Documents

Publication Publication Date Title
JP7450623B2 (ja) 機械学習のための動的視聴覚セグメント・パディング
US11663827B2 (en) Generating a video segment of an action from a video
US20230128721A1 (en) Custom video workout programs for execution on an exercise machine
JP7033587B2 (ja) 映像ハイライトを自動的に製作する方法及びシステム
CN108712661B (zh) 一种直播视频处理方法、装置、设备及存储介质
Sun et al. Ranking domain-specific highlights by analyzing edited videos
US11308148B2 (en) Identifying a thumbnail image to represent a video
US11910060B2 (en) System and method for automatic detection of periods of heightened audience interest in broadcast electronic media
CN107463698B (zh) 基于人工智能推送信息的方法和装置
US12368909B2 (en) Image analysis system
JP2018534700A (ja) メディアコンテンツのパフォーマンスの即時予測のための方法及び装置
US11206453B2 (en) Cognitive broadcasting of an event
US20250111674A1 (en) Summarizing Events Over a Time Period
US20250254402A1 (en) Systems and methods for generating sports media content for an interactive display
US11450351B1 (en) Dynamic and intelligent video editor
CN120568143A (zh) 一种基于人工智能的视频剪辑处理方法和系统
Godi et al. Indirect match highlights detection with deep convolutional neural networks
US20240078806A1 (en) Automated tolerance-based matching of video streaming events with replays in a video
Gu et al. SumBot: Summarize videos like a human
US20250252632A1 (en) Systems and methods for generating sports media content for an interactive display
KR102829091B1 (ko) 하이라이트 영상 제공 장치 및 방법
US20250272330A1 (en) Methods and systems for generating automated sports summaries
US20250292154A1 (en) Machine learning techniques for stoppage time prediction in soccer
US20250316084A1 (en) Systems and methods for agentic operations using multimodal generative models for cricket
US11475668B2 (en) System and method for automatic video categorization

Legal Events

Date Code Title Description
RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20220512

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20220527

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20220725

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20230831

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20230912

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231010

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20231219

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20240130

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240220

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240305

R150 Certificate of patent or registration of utility model

Ref document number: 7450623

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150