WO2021212207A1 - Systèmes et procédés pour traiter des données d'image afin qu'elles coïncident à un instant donné avec des données audio - Google Patents

Systèmes et procédés pour traiter des données d'image afin qu'elles coïncident à un instant donné avec des données audio Download PDF

Info

Publication number
WO2021212207A1
WO2021212207A1 PCT/CA2021/050187 CA2021050187W WO2021212207A1 WO 2021212207 A1 WO2021212207 A1 WO 2021212207A1 CA 2021050187 W CA2021050187 W CA 2021050187W WO 2021212207 A1 WO2021212207 A1 WO 2021212207A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
timeline
image data
data
synchronized multimedia
Prior art date
Application number
PCT/CA2021/050187
Other languages
English (en)
Inventor
Wilter MONTEIRO SIQUEIRA FRANCESCHI
Vanessa DA SILVA FIGUEIREDO
Original Assignee
Monteiro Siqueira Franceschi Wilter
Da Silva Figueiredo Vanessa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Monteiro Siqueira Franceschi Wilter, Da Silva Figueiredo Vanessa filed Critical Monteiro Siqueira Franceschi Wilter
Publication of WO2021212207A1 publication Critical patent/WO2021212207A1/fr

Links

Classifications

    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/34—Indicating arrangements 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234336—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4722—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
    • H04N21/4725—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content using interactive regions of the image, e.g. hot spots
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8547—Content authoring involving timestamps for synchronizing content
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/858—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586—Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/10—Multimedia information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/52—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services

Definitions

  • This application relates to the field of software engineering. More particularly, the present disclosure relates to implementing methods and systems to synchronize unconnected image data and audio data.
  • Existing technology supports the inclusion of presentation slides or other kinds of graphical data to coincide in a point of time with audio data and video data to generate synchronized presentations. Nevertheless, the methods and systems employed to process the inclusion of such data produce video data by merging audio and graphical data. As a result, the audio and graphical data become a single file in a video format (e.g. .mp4). If an individual need to edit parts of that single file, it will be necessary to upload the data that generated such a file again. Furthermore, the existing technology does not support the synchronization of audio data and image data during live transmissions occurring on the Internet. For example, an individual hosting a podcast wants to include an image to illustrate what it is being presented during the podcast. Such an image cannot be added to coincide in a point of time with the audio playback using the existing technology unless the individual uses a video camera to record a video.
  • a video format e.g. .mp4
  • the present disclosure concerns implementing systems and methods for processing image data to coincide in a point of time with audio data, maintaining the integrity of both image and audio data.
  • a tutor teaching online lessons wants to use an image to support the learning of their pupils while narrating the characteristics of such an image.
  • the tutor uploads the supporting image and manipulates the image to be displayed during the online lessons.
  • the tutor chooses to zoom in the image and display the details, thus supporting the narration describing the characteristics of the image.
  • the tutor generates a timestamp and assigns this timestamp to the image. Consequently, the image will be displayed in a point of time coinciding with the audio timeline playback in a synchronized format.
  • the present disclosure results in a synchronized format establishing a timed connection between the image and audio data.
  • a social media influencer expert on reviews and tutorials concerning makeup takes pictures of her face wearing makeup and adds links and descriptions overlaying those pictures by using a number of manipulations existing in the present disclosure.
  • the social media influencer’s audience will be able to save in the form of bookmarks the content existing on the images.
  • the present disclosure concerns implementing systems and methods for processing image data to coincide in a point of time with audio data to create a master timeline coordinating audio timeline playback and image data display to generate a synchronized multimedia presentation using content creator and content player modules.
  • the system comprises a network architecture, a synchronization system and a file system database.
  • the network architecture comprises one or more computing devices and computer networks connected to the synchronization system.
  • the synchronization system comprises the content creator module, the content player module and the file system database.
  • the file system database is configured to store data pertaining to a master timeline, a digital control file and one or more content player user events.
  • the method comprising: accessing a content creator module of a synchronization system; manipulating one or more audio data to assemble an audio timeline containing a sequential order of one or more audio data; manipulating one or more image data to display transformations made as a result of editing; creating one or more timestamps on the audio timeline; assigning each image data to a timestamp corresponding to a time value of a play duration of the audio timeline; generating a master timeline containing manipulated image data and the audio timeline; generating a digital control file containing records and information pertaining to manipulations of image data and the audio timeline in the master timeline; including meta-elements describing characteristics of a synchronized multimedia presentation; storing the master timeline in the form of a digital control file in a file system database; making a request to the file system database; processing the digital control file containing records and information of a synchronization of image data with an audio timeline; loading a master timeline containing a synchronized multimedia presentation; activating the master timeline
  • FIG. 1 is an illustration of a computer network, one or more computing devices and a synchronization system in accordance with an embodiment of the invention.
  • FIG. 2 is an illustration of a synchronization system in accordance with an embodiment of the invention.
  • FIG. 3 is an illustration of a content creator module system in accordance with an embodiment of the invention.
  • FIG. 4 is an illustration of a content player module system in accordance with an embodiment of the invention.
  • FIG. 5 is an illustration of a user interface for a content creator module system in accordance with an embodiment of the invention.
  • FIG. 6 is an illustration of a user interface for a content player module system in accordance with an embodiment of the invention.
  • FIGS. 7A-7B (collectively referred to as “FIG.7”) provide a flow diagram of an illustrative method for processing image data to coincide in a point of time with audio data in accordance with an embodiment of the invention.
  • FIG.8 provides a flow diagram of an illustrating method for reproducing a master timeline synchronizing image data with audio data in accordance with an embodiment of the invention.
  • FIG. 9 is an illustration of a master timeline playback illustrating image data processed to coincide in a point of time with an audio timeline in accordance with an embodiment of the invention.
  • Systems for processing image data to coincide in a point of time with audio data to create a master timeline coordinating audio timeline playback and image data display to generate a synchronized multimedia presentation using content creator and content player modules comprises a network architecture, a synchronization system and a file system database.
  • FIG.1 A network architecture 100 for producing, transmitting, receiving and reproducing synchronized multimedia presentations is illustrated in FIG.1.
  • synchronized multimedia presentations refer to one or more image data arranged to coincide in a point of time with parts of audio data containing play duration times.
  • the one or more image data may be photographic image files and computer graphic images in accordance with an embodiment of the invention.
  • the one or more audio data may be relating to files reproducing recorded sounds in accordance with an embodiment of the invention.
  • the network architecture 100 comprises one or more computing devices 101 and computer networks 102 connected to a synchronization system 103 as illustrated in FIG. 1.
  • the computing devices 101 may be any electronic device for storing and processing data comprising at least a screen and enabled to connect to computer networks 102.
  • the computer network 102 may be the Internet and other networks that connect to the Internet.
  • the synchronization system 103 comprises a content creator module 201, a content player module 202 and a file system database 203 in accordance with an embodiment of the invention.
  • the synchronization system 103 is configured to permit the creation, transmission, reception, storage, preview and playback of synchronized multimedia presentations.
  • Each of the content modules 202- 203 comprise a processor 204 to execute commands and operations generated in each module 202-203 in accordance with an embodiment of the invention.
  • Users accessing the synchronization system 103 may be content creator users and content player users in accordance with an embodiment of the invention.
  • the content creator module 201 operates to permit content creator users using a computing device 101 to create, transmit, receive, save, preview and reproduce synchronized multimedia presentations as illustrated in FIG. 3 in accordance with an embodiment of the invention.
  • a content creator user may create, list and manipulate synchronized multimedia presentations using one or more image data and audio data in accordance with an embodiment of the invention.
  • the content creator module 201 comprises an audio data section 301, an image data section 306, a master timeline section 310, a synchronization section 313 and a file system database 203 in accordance with an embodiment of the invention.
  • the content creator module may comprise a live transmission section 317 and an audio to text transcription section 323.
  • the audio data section 301 comprises an audio data creation 302, an audio data list 303, an audio data manipulation 304 and an audio data timeline creation 305.
  • the audio data section 301 is configured to provide an interface to create, list and manipulate one or more audio data.
  • the image data section 306 comprises an image data creation 307, an image data list 308 and an image data manipulation 309.
  • the image data section 306 is configured to provide an interface to create, list and manipulate one or more image data.
  • the master timeline section 310 comprises a timestamp creator 311 and an audio timeline and image data time relationship generator 312.
  • the master timeline section 310 is configured to provide an interface to create timestamps in an audio timeline and process a synchronization of one or more image data with the audio timeline.
  • timestamps refer to digital records of time to indicate a play duration time of audio data wherein image data is displayed to coincide in time with a time assigned in an audio timeline
  • the synchronization section 313 comprises a master timeline information compilation 314, an audio timeline and image data time relationship execution 315, and a digital control file generator 316.
  • the synchronization section 313 is configured to process and execute a digital control file containing commands to execute a synchronization of one or more image data with the audio timeline.
  • a content creator user may activate a live transmission 317 of one or more synchronized multimedia presentations as shown in FIG.3. If the content creator user chooses to activate a live transmission 317, a live transmission section 318 starts to operate.
  • the live transmission section 318 comprises a live transmission activation 319, a transmission delay time 320, a live transmission operation 312.
  • the live transmission activation 319 is configured to start live transmissions at the same time that the content creator user creates the one or more synchronized multimedia presentations.
  • the transmission delay time 320 is configured to delay the time wherein one or more synchronized multimedia presentations are transmitted to one or more content player users.
  • the live transmission operation 312 is configured to permit live transmissions and reception of one or more synchronized multimedia presentations using a computing device 101 over a computer network 102, allowing playback to start while the rest of the data is still being received.
  • a content creator user may activate an audio to text transcription 322 based on the content of an audio timeline as shown in FIG. 3. If the content creator user chooses to activate an audio to text transcription 322, an audio to text transcription section 323 starts to operate.
  • the audio to text transcription section 323 comprises a manual audio to text transcription upload 324, an automated audio to text transcription activation 325, and an audio to text transcription display 325.
  • the manual audio to text transcription upload 324 is configured to permit transferring audio to text transcription files from a computing device 101 to the content creator module 201.
  • the automated audio to text transcription activation 325 is configured to generate automatic audio to text transcriptions of an audio timeline using machine learning techniques generated by an application programming interface.
  • the audio to text transcription display 326 is configured to enable the display of audio to text transcriptions at the same time that a synchronized multimedia presentation is reproduced.
  • the content player module 202 comprises a synchronization section 323, a synchronization section processing section 401, a playback section 404 and a user interface section 406 in accordance with one embodiment of the invention.
  • the synchronization processing section 401 comprises a digital control file processor 402 and a master timeline loader 403.
  • the synchronization processing section 401 is configured to retrieve the digital control file from the synchronization section 323 and provide an interface to process the digital control file 402 and load the master timeline 403.
  • the playback section 404 is configured to reproduce a master timeline playback 405 guiding a synchronized multimedia presentation wherein one or more image data are configured to coincide in a point of time with parts of an audio timeline.
  • the user interface section 406 comprises a synchronized master timeline playback viewer 407, a content player user events 408 and a log content player user events 409.
  • the synchronized master timeline playback viewer 407 is configured to play, forward, backward, pause, stop, adjust sound volume, vary playback speed of the master timeline and enable content player user events in accordance with an embodiment of the invention.
  • the content player user events 408 is configured to create, store and distribute one or more content player user events.
  • Content player user events may be to store and distribute preferred parts of a master timeline, create marks on the master timeline, add one or more image data to the master timeline and enable audio to text transcriptions in accordance with an embodiment of the invention.
  • the log content player user events 408 is configured to load and store one or more content player user events in the file system database 203.
  • An exemplary content creator interface 501 for creating a synchronized multimedia presentation according to an embodiment of the invention is illustrated in FIG 5.
  • the content creator interface 501 is provided as a computer program to computing devices. Nevertheless, in other embodiments, the content creator interface 501 may be provided as a webpage by a webpage provider to computing devices accessing a webpage processing the synchronization system 103.
  • the content creator interface 501 comprises a live transmission module 502, a list of images section 503, an add image data section 505, a manipulation tools section 506, a current image section 507, an add audio data section 508, an audio manipulation tools section 509, an audio timeline assembling section 510, a timestamp section 514, an image timestamp assignment section 517, and a master timeline section 518 and an audio to text transcription module 521 in accordance with an embodiment of the invention.
  • the current image section 507 displays an image data being manipulated by the content creator user.
  • the manipulation tools section 506 comprises one or more manipulations a content creator user generates for an image data.
  • the one or more manipulations may be resize, crop, color manipulation, rotate, include layers with graphical elements, tridimensional manipulations, animations, zoom, sharpen, enhance, remove blemish, add tone effect, reverse image, reverse exposure, add, delete, and modify one or more graphical image elements in accordance with an embodiment of the invention.
  • the assign timestamp to image data section 508 enables the assignment of timestamps containing the play duration time of audio data wherein image data is displayed to coincide in time with a time assigned in an audio timeline.
  • the add audio data section 508 comprises creating and/or listing one or more audio data, wherein a content creator user captures, transfers from a computing device and/or manipulates the one or more audio data in accordance with an embodiment of the invention.
  • the audio manipulation tools section 509 comprise tools to equalize sounds of audio data, add and/or remove parts of audio data, add and/or remove sound effects of audio data in accordance with an embodiment of the invention.
  • the audio timeline assembling section 510 is configured to list one or more audio data 511 in a sequential order and assemble 512 the one or more data 511 into an audio timeline 513.
  • the timestamp section is configured to add timestamps 515 corresponding to time durations in a playback of the audio timeline 513.
  • the image timestamp assignment section 517 is configured to assign one or more timestamps 515 to each of the image data 507 corresponding to a time duration in a playback of the audio timeline 513.
  • the master timeline section 518 is configured to generate 519 and reproduce a master timeline 520 comprising the time synchronization of image data with audio data in accordance with an embodiment of the invention.
  • the live transmission area 502 comprises a transmission and reception of the synchronized multimedia presentation using a computing device over a computer network, allowing playback to start while the rest of the data is still being received while in a live transmission in accordance with one embodiment of the invention.
  • the audio to text transcription area 521 comprises an area to transfer an audio to text transcription generated manually and an area to enable an audio to text transcription generated automatically.
  • the automatic audio to text transcription is generated by an application programming interface.
  • the content creator interface 501 illustrated in FIG. 5 represents one of many possibilities to arrange the content creator interface. Nevertheless, in other embodiments, the sections represented in the content creator interface 501 may be arranged in different configurations.
  • a synchronized multimedia presentation may be used by a content creator user generating audio data to be available over computer networks to one or more individuals (e.g., podcasts).
  • the synchronized multimedia presentations may be used by a content creator teaching electronic lessons to one or more individuals in person and/or over computer networks.
  • the synchronized multimedia presentation may be used by a content creator generating content to be distributed over one or more web applications.
  • a content player module 202 may operate to permit content player users operating a computing device 101 to search synchronized multimedia presentations, navigate synchronized multimedia presentations, add content player user events and reproduce synchronized multimedia presentations in accordance with an embodiment of the invention.
  • the content player module 202 is configured to permit content player users to receive live transmissions of synchronized multimedia presentations created by content creator users.
  • FIG. 6 An exemplary content player interface 601 for reproducing a synchronized multimedia presentation according to one embodiment of the invention is illustrated in FIG. 6.
  • the content player interface 601 is provided as a computer program to computing devices.
  • the content creator interface may be provided as a webpage by a webpage provider to computing devices accessing the webpage.
  • the content player interface comprises a content player user events area 602, a synchronized multimedia presentation viewer 608, and a synchronized presentation sections area 610.
  • the content player user events area 602 comprises one or more digital records 603 generated by content player users to mark a master timeline, web applications to distribute content over computer networks 604 to one or more content player users, an audio to text transcription module 605, an add content player user events section 606 and a log content player user events section 607 in accordance with one embodiment of the invention.
  • the one or more digital records 603 comprise sections in the master timeline wherein the content player user creates marks on one or more preferred sections of a master timeline (e.g., bookmarks).
  • the web applications distributing content over computer networks 604 comprise one or more applications to distribute preferred master timelines and sections in master timelines to other users over a computer network.
  • the audio to text transcription module 605 comprises a section to enable the audio to text transcript to be displayed in a master timeline.
  • the add content player user events section 606 is configured to include and record information created by a content player user for a synchronized multimedia presentation. The information may be annotations, links, meta-elements, image data and audio data in accordance with one embodiment of the invention.
  • the log content player user events section 607 is configured to store records and information pertaining to content player user events and generated by a content player user for a synchronized multimedia presentation in accordance with one embodiment of the invention.
  • the synchronized presentation viewer 608 comprises an image data viewer area 609 and a master timeline playback area 610 in accordance with an embodiment of the invention.
  • the image data viewer area 609 displays one or more synchronized multimedia presentations.
  • the one or more synchronized multimedia presentations are reproduced in a live transmission generated by a content creator user.
  • the master timeline playback area 610 reproduces the synchronized multimedia presentation containing image data and the audio timeline in accordance with an embodiment of the invention.
  • the master timeline playback area 610 comprises play, forward, backward, pause, stop, adjust sound volume, and vary playback speed of the synchronized multimedia presentation.
  • the synchronized multimedia presentation sections area 611 comprises one or more sections 612 wherein a content player user selects a preferred point in the master timeline to view the synchronized multimedia presentation.
  • the one or more sections 612 may be generated automatically or manually by a content creator user in accordance with one embodiment of the invention.
  • the one or more sections 612 generated automatically are retrieved from the timestamps 515 generated by the content creator user.
  • the content player interface 601 illustrated in FIG. 6 represents one of many possibilities to arrange the content creator interface. Nevertheless, in other embodiments, the sections represented in the content creator interface 601 may be arranged in different configurations.
  • the method 700 begins with a content creator user accessing a synchronization system using a computing device 101 as described in the process 702.
  • accessing the synchronization system 103 may occur over a computer network 102.
  • the method 700 continues with enabling the live transmission 704 in accordance with an embodiment of the invention.
  • the live transmission 703 is configured to transmit and receive one or more synchronized multimedia presentations in a computing device 101 over a computer network 102 allowing playback to start while the rest of the data is still being received while in a live transmission.
  • the live transmission may be configured to delay the transmission and reception of synchronized multimedia presentations to one or more content player users in accordance with an embodiment of the invention.
  • the one or more audio data is manipulated to assemble an audio timeline containing a sequential order of one or more audio data.
  • the one or more audio data are created by using an apparatus to capture sounds and transfer to the content creator module 201.
  • the one or more audio data are listed by transferring the one or more audio data from a computing device 101 to the content creator module 201.
  • the one or more audio data is manipulated by equalizing sounds of audio files, adding and/or removing parts of audio files, adding and/or removing sound effects.
  • the one or more audio data is reproduced by playing, forwarding, backwarding, pausing, stopping, adjusting sound volume, varying playback speed of audio data.
  • the one or more audio data are assembled to generate an audio timeline 513 comprising the sum of time durations of the one or more audio data in accordance with an embodiment of the invention.
  • the audio timeline 513 comprises a sequential order of one or more audio data wherein the sequential order may be determined by a content creator user in accordance with an embodiment of the invention.
  • one or more image data are manipulated to display transformations made as a result of editing.
  • the one or more image data are created using an apparatus to capture images and transfer to the content creator module 201 in accordance with one embodiment of the invention.
  • the one or more image data are listed by transferring the one or more image data from a computing device 101 to the content creator module 201.
  • the one or more image data are manipulated using one or more manipulation tools 506.
  • the one or more manipulation tools comprise resizing, cropping, color manipulating, rotating, including layers with graphical elements, tridimensional manipulations, animations, zooming, sharpening, enhancing, removing blemish, adding tone effect, reversing image, reversing exposure, adding, deleting, and modifying one or more graphical image elements in accordance with one embodiment of the invention.
  • the one or more image data transferred to the content creator module 201 may be previewed by a content creator user.
  • one or more timestamps are generated for an audio timeline to be assigned to each of the image data listed in the content creator module.
  • the audio timestamps 515 represent the current play time for each of the image data listed in a list of image data section 503.
  • each of the image data listed in the content creator module is assigned a timestamp 513 recording a play time wherein each image is displayed to coincide in time with the audio timeline 513.
  • each image timestamp is connected to the master timeline, thus generating an audio data and image data time relationship.
  • the master timeline containing image data and the audio timeline generates a digital control file.
  • the digital control file contains records or information concerning one or more manipulations processed in audio data and image data wherein the characteristic integrity of audio data and image data are maintained.
  • the digital control file may use any text file standard, wherein some embodiments may use HTML, XML and Json.
  • the digital control file comprises a text file standard recording a time relationship between the audio timeline and image data.
  • one or more meta-elements describing the characteristics of a synchronized multimedia presentation may be added permitting that synchronized multimedia presentations are discoverable by search engines in accordance with one embodiment of the invention.
  • the one or more metaelements comprising records and information stored in the digital control file and audio to text transcription may be text files, meta-tags, titles, author and date to generate meta-elements.
  • the method 700 continues with enabling the audio to text transcriptions 713 in accordance with an embodiment of the invention.
  • Automatic and manual audio to text transcripts describing the content in an audio timeline may be enabled in accordance with one embodiment of the invention.
  • the manual audio to text transcription comprises a transcription generated by a human operator (e.g., content creator user) transferred from a computing device 101 to the content creator module 201.
  • the automated audio to text transcription comprises a transcription generated using machine learning techniques generated by an application programming interface in accordance with an embodiment of the invention.
  • the manual and automated audio to text transcriptions comprising a text-type file may be recorded in a different file than the digital control file storing the master timeline synchronizing one or more image data with an audio timeline 513.
  • the text-type file generated for audio to text transcriptions is stored in the file system database 203.
  • the digital control file containing the commands to execute the time relationship between image data and the audio timeline, thus comprising a master timeline is stored in a file system database 203.
  • a method 800 for reproducing a master timeline 520 to reproduce a synchronized multimedia presentation containing image data coinciding in time with an audio timeline is described in accordance with an embodiment of the invention.
  • the content player module is accessed by a content player user.
  • a request to the file system database to retrieve the digital control file is generated in 802.
  • the content player module 202 processing the digital control file in 803 to execute the time relationship between image data and the audio timeline. Then, the master timeline is loaded to proceed with reproducing a synchronized multimedia presentation in 804. In the process 805, the master timeline playback is activated.
  • the method 800 continues with displaying the audio to text transcriptions.
  • the content player module 202 may enable the exhibition of meta-elements, records content player user events and distributes content player user events to one or more content player users in accordance with an embodiment of the invention.
  • the master timeline is reproduced, and the synchronized multimedia presentation is viewed by the content player user.
  • the audio timeline reproducing a sequential order of audio data during the time that listed, created and/or manipulated image data are displayed in 908.
  • an image not having the use of manipulations 902 may be displayed as the first image in the point of time synchronized with the timestamps 517 generated for the audio timeline 513.
  • a manipulated version of the first image may be displayed in the point of time synchronized with the timestamps generated for the audio timeline, wherein a section of the first image displaying a close-up with one or more characteristics of the first image are viewed in detail 903.
  • a second image not having the use of manipulations 904 may be displayed in the point of time synchronized with timestamps 517 generated for the audio timeline 513.
  • a third image not having the use of manipulations 906 may be displayed in the point of time synchronized with the timestamps generated for the audio timeline.
  • a manipulated version of the third image may be displayed in the point of time synchronized with the timestamps generated for the audio timeline, wherein one or more hyperlinks 907 overlaying the third image allowing a content player user pressing a button or touching a computing device screen to access the content of the one or more hyperlinks 907.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

La présente invention concerne des systèmes et des procédés pour traiter des données d'image afin qu'elles coïncident à un instant donné avec des données audio pour créer une ligne de temps maître coordonnant la lecture de ligne de temps audio et l'affichage de données d'image pour générer une présentation multimédia synchronisée en utilisant des modules de créateur de contenu et de lecteur de contenu. Le système comprend une architecture de réseau, un système de synchronisation et une base de données de système de fichier. Le procédé consiste à : accéder à un module de créateur de contenu d'un système de synchronisation; manipuler une ou plusieurs données audio pour assembler une ligne de temps audio; manipuler une ou plusieurs données d'image pour afficher les transformations effectuées suite au montage; créer une ou plusieurs estampilles temporelles sur la ligne de temps audio; affecter chaque donnée d'image à une estampille temporelle correspondant à une valeur de temps d'une durée de lecture de la ligne de temps audio; générer une ligne de temps maître; générer un fichier de commande numérique; et reproduire la présentation multimédia synchronisée dans un module de lecteur de contenu.
PCT/CA2021/050187 2020-04-24 2021-02-19 Systèmes et procédés pour traiter des données d'image afin qu'elles coïncident à un instant donné avec des données audio WO2021212207A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CA3079444A CA3079444C (fr) 2020-04-24 2020-04-24 Systemes et methodes de traitement de donnees d`image pour coincider avec les donnees sonores a un point dans le temps
CA3079444 2020-04-24

Publications (1)

Publication Number Publication Date
WO2021212207A1 true WO2021212207A1 (fr) 2021-10-28

Family

ID=78270781

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2021/050187 WO2021212207A1 (fr) 2020-04-24 2021-02-19 Systèmes et procédés pour traiter des données d'image afin qu'elles coïncident à un instant donné avec des données audio

Country Status (2)

Country Link
CA (1) CA3079444C (fr)
WO (1) WO2021212207A1 (fr)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7512886B1 (en) * 2004-04-15 2009-03-31 Magix Ag System and method of automatically aligning video scenes with an audio track
US20100281381A1 (en) * 2009-04-30 2010-11-04 Brian Meaney Graphical User Interface for a Media-Editing Application With a Segmented Timeline
US20110123972A1 (en) * 2008-08-04 2011-05-26 Lior Friedman System for automatic production of lectures and presentations for live or on-demand publishing and sharing
US20160328105A1 (en) * 2015-05-06 2016-11-10 Microsoft Technology Licensing, Llc Techniques to manage bookmarks for media files
US20170229152A1 (en) * 2016-02-10 2017-08-10 Linkedin Corporation Adding content to a media timeline
US20180270446A1 (en) * 2017-03-19 2018-09-20 Apple Inc. Media message creation with automatic titling
US10560502B2 (en) * 2017-08-24 2020-02-11 OpenExchange, Inc. Method to re-synchronize live media streams, commands, and on-screen events transmitted through different internet pathways
US10582277B2 (en) * 2017-03-27 2020-03-03 Snap Inc. Generating a stitched data stream

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7512886B1 (en) * 2004-04-15 2009-03-31 Magix Ag System and method of automatically aligning video scenes with an audio track
US20110123972A1 (en) * 2008-08-04 2011-05-26 Lior Friedman System for automatic production of lectures and presentations for live or on-demand publishing and sharing
US20100281381A1 (en) * 2009-04-30 2010-11-04 Brian Meaney Graphical User Interface for a Media-Editing Application With a Segmented Timeline
US20160328105A1 (en) * 2015-05-06 2016-11-10 Microsoft Technology Licensing, Llc Techniques to manage bookmarks for media files
US20170229152A1 (en) * 2016-02-10 2017-08-10 Linkedin Corporation Adding content to a media timeline
US20180270446A1 (en) * 2017-03-19 2018-09-20 Apple Inc. Media message creation with automatic titling
US10582277B2 (en) * 2017-03-27 2020-03-03 Snap Inc. Generating a stitched data stream
US10560502B2 (en) * 2017-08-24 2020-02-11 OpenExchange, Inc. Method to re-synchronize live media streams, commands, and on-screen events transmitted through different internet pathways

Also Published As

Publication number Publication date
CA3079444A1 (fr) 2021-10-24
CA3079444C (fr) 2023-06-27

Similar Documents

Publication Publication Date Title
US10600445B2 (en) Methods and apparatus for remote motion graphics authoring
US10728354B2 (en) Slice-and-stitch approach to editing media (video or audio) for multimedia online presentations
US9837077B2 (en) Enhanced capture, management and distribution of live presentations
JP3657206B2 (ja) 個人用映画コレクションの作成を可能にするシステム
US8353406B2 (en) System, method, and computer readable medium for creating a video clip
US9251852B2 (en) Systems and methods for generation of composite video
US20020091658A1 (en) Multimedia electronic education system and method
EP2171717B1 (fr) Production automatisée non séquentielle, par un kit d'auto-interview ou scenario avec prompteur, d'une vidéo de contenu multimédia généré par l'utilisateur.
US20110072037A1 (en) Intelligent media capture, organization, search and workflow
EP0469850A2 (fr) Méthode et appareil pour le préenregistrement, l'édition et la reproduction des présentations sur un système d'ordinateur
US11989808B2 (en) Systems and methods for template image edits
WO2014186052A2 (fr) Procédé et système pour fournir des informations de repérage d'emplacement
KR20060035729A (ko) 가상교실에서의 수업내용을 제공 및 기록하는 방법 및시스템
US20140282087A1 (en) System and Methods for Facilitating the Development and Management of Creative Assets
Braun Listen up!: podcasting for schools and libraries
JP4129162B2 (ja) コンテンツ作成実演システム及びコンテンツ作成実演方法
CA3079444C (fr) Systemes et methodes de traitement de donnees d`image pour coincider avec les donnees sonores a un point dans le temps
JP4686990B2 (ja) コンテンツ処理システム及びコンテンツ処理方法、並びにコンピュータ・プログラム
JP2008066794A (ja) テレビ会議予約・実施制御・記録制御装置、テレビ会議予約・実施制御・記録制御方法およびテレビ会議予約・実施制御・記録制御プログラム
JP2004266578A (ja) 動画像編集方法および装置
Clark et al. Captivate and Camtasia
JP2011199360A (ja) コンテンツ配信装置、コンテンツ配信プログラム、コンテンツ再生プログラム、コンテンツ配信方法およびコンテンツ再生方法
Jago Adobe Premiere Pro Classroom in a Book (2022 Release)
JP2016152545A (ja) 情報配信システムおよびサーバ装置
Buehner et al. A Music Librarian’s Guide to Creating Videos and Podcasts

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21793068

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21793068

Country of ref document: EP

Kind code of ref document: A1