US20210185415A1 - Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods - Google Patents
Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods Download PDFInfo
- Publication number
- US20210185415A1 US20210185415A1 US17/119,784 US202017119784A US2021185415A1 US 20210185415 A1 US20210185415 A1 US 20210185415A1 US 202017119784 A US202017119784 A US 202017119784A US 2021185415 A1 US2021185415 A1 US 2021185415A1
- Authority
- US
- United States
- Prior art keywords
- audio
- media file
- file
- video
- media
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000010606 normalization Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 description 23
- 238000004891 communication Methods 0.000 description 11
- 239000000872 buffer Substances 0.000 description 10
- 230000006835 compression Effects 0.000 description 7
- 238000007906 compression Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- RZVAJINKPMORJF-UHFFFAOYSA-N Acetaminophen Chemical compound CC(=O)NC1=CC=C(O)C=C1 RZVAJINKPMORJF-UHFFFAOYSA-N 0.000 description 2
- 238000012952 Resampling Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000282693 Cercopithecidae Species 0.000 description 1
- 241000586891 Poliaspis media Species 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- NHDHVHZZCFYRSB-UHFFFAOYSA-N pyriproxyfen Chemical compound C=1C=CC=NC=1OC(C)COC(C=C1)=CC=C1OC1=CC=CC=C1 NHDHVHZZCFYRSB-UHFFFAOYSA-N 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000005236 sound signal Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/116—Details of conversion of file system types or formats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
- G06F16/164—File meta data generation
- G06F16/166—File name conversion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
- G06F16/61—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
-
- H04L65/607—
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
- H04N21/2335—Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/8106—Monomedia components thereof involving special audio data, e.g. different tracks for different languages
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/85—Assembly of content; Generation of multimedia applications
- H04N21/854—Content authoring
- H04N21/85406—Content authoring involving a specific file format, e.g. MP4 format
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Definitions
- the invention relates to simultaneous recording and uploading systems and methods, and, more particularly to a simultaneous recording and uploading of multiple files from the same conversation. Further, the present invention relates to audio drift normalization, and more particularly to audio drift normalization systems and methods that can normalize audio drift of a plurality of recordings from a source.
- podcasters, audio book producers, and content creators find themselves spending time, effort, and extra costs to produce quality video and/or audio recordings. These additional logistical steps increase operating costs and increase the time needed to publish video and/or audio works. These previous methods needed to record an audio and/or video file in its entirety before being uploaded to the cloud.
- the recording assembly system must take all the chunks and write them into one file. In doing so they need to ensure that the data is captured, is continuous, and mixed together. This assembly can result in audio drift.
- the invention in one form is directed to a method that ensures that recorded audio and video files are always accessible in the event of a catastrophe, removes the need for the audio and video file to exist in its entirety before uploading and requires less system resources be present within the recording device.
- the invention in another form improves the reliable access to complete audio files and reduces time waiting for large files to upload after a recording is finished.
- the invention in yet another form allows for the use of low-end devices due to the efficiency of the recording system.
- the invention improves over existing chunk style recording methods.
- system can also include algorithms and/or software which eliminate manual processes to perform system processes and utilize methods.
- system can include methods to perform chunk style recording to produce synchronized recordings.
- the system can include processes to normalize audio drift that arises between two or more audio recordings of an original source that is captured on one or a plurality of devices.
- the system can also include components that are configured to minimize human interaction and/or executed steps to produce enhanced audio and video quality recordings. These recordings are designed to produce top grade audio recordings, the recordings configured to be combined in other audio and/or video recordings without the need for back end engineering to improve quality of the recordings.
- system can include systems configured to provide near real-time interview capability with video and record everyone's audio in separate media files without drift.
- the system can include a platform which provides a medium to connect podcast hosts, co-hosts, & guests to record studio quality audio and video from anywhere.
- system can also include allowing system users to download, listen, and share one or all of user's recordings organized by any parameter, including but not limited to session, date, geography and/or recording medium.
- the system can also include a video feature so a podcaster and a guest can see each other while recording.
- the system can include mobile applications and can be utilized on a mobile device.
- system can also include laptops, desktop computers, and computing mediums.
- the system can include utilizing a standardized 48 kHz sampling rate.
- the system can include processes which do not require echo cancellation (Acoustic Echo Cancellation (AEC) Software).
- AEC Acoustic Echo Cancellation
- system can also include remote audio signals and can exist on the cloud/web within a network.
- the system can include saving recordings in buffers of any length of time, preferably between 2 and 8 seconds.
- the system can also utilize resampling to 48 kHz.
- the system can include longform recordings, exceeding one hour in length.
- the system can reduce the need for local storage and is unbounded in the duration of recording.
- system can also include components and/or methods which are non-destructive wherein no data is lost.
- system can include systems and/or methods that do not utilize a master clock or drift threshold.
- the system can include methods and/or components which normalize audio drift from different sources.
- the system can include quality, reliable, repeatable, systems efficient enough to utilize low end devices.
- system can be used alongside aviation flight data recording systems to reduce the need to search for the “black box” because the recordings are not stored locally.
- an audio drift normalization system is presented.
- the system can ensure that recorded audio files are automatically modified to correct audio drift arising from a plurality of recording devices and/or buffer size differences.
- system can also include algorithms and/or software which eliminate manual processes to correct audio drift.
- system can include methods to perform chunk style recording to produce synchronized recordings.
- the system can include processes to normalize audio drift that arises between two or more audio recordings of an original source that is captured on one or a plurality of devices.
- FIG. 1 is a method flowchart for simultaneous recording audio files in accordance with an embodiment of the present invention
- FIG. 2 is a method flowchart for simultaneous recording audio files in accordance with an embodiment of the present invention.
- FIG. 3 is a method flow chart for audio drift normalization in accordance with an embodiment of the present invention.
- chunks when used by the instant invention may refer to segments of audio and/or video recordings.
- media recordings audio recordings, video recordings may be used interchangeably and mean a recording of the using a media capture device such as a microphone or video camera.
- media data media files audio data, audio files, video data, video files may be used interchangeably and mean data stored on a computer representing the data of the captured media.
- WAV Wideband Audio
- MP3 Motion Picture Expert Group 4
- audio Video file
- AVI Audio Video Interleave
- FLV Flash Video Format
- WMV Windows Media Video
- MOV Apple QuickTime Movie
- MP4 Motion Pictures Expert Group 4
- cloud when used in context by the instant invention can mean Amazon cloud, Google cloud, Verizon cloud or any suitable cloud storage system.
- the term computing device capable may include any type of device for computing, such as a tablet, mobile device, smart speaker, flight data recorder, or computer that may be in communication with a media recording device such as a microphone or video camera.
- the term public URL is a URL that can be asked by anyone on the Internet as it is indexed by search engines.
- video file format is a type of file format for storing digital video data on a storage device such as a computer system. Video is almost always stored using compression to reduce the file size.
- a video file normally consists of a container containing video data in a video coding format alongside audio data in an audio coding format.
- stereo commonly refers to Stereophonic sound and is the reproduction of sound, using two or more independent audio channels, through a symmetrical configuration of loudspeakers, in such a way as to create a pleasant and natural impression of sound heard from various directions, as in natural hearing.
- monophonic sound commonly called mono sound, mono, or non-stereo sound
- this early sound system used a single channel of audio for sound output.
- the signal sent to the sound system encodes one single stream of sound and it usually uses just one speaker or the same exact sound in both speakers.
- database can alternatively mean any suitable storage medium can be substituted such as a data structures, files, stacks utilizing Direct access storage device (DASD) or non-volatile computer storage an example of which is solid state storage medium.
- DASD Direct access storage device
- audio file format is a file format for storing digital audio data on a computer system.
- the bit layout of the audio data (excluding metadata) is called the audio coding format and can be uncompressed, or compressed to reduce the file size, often using compression.
- the data can be a raw bitstream in an audio coding format, but it is usually embedded in a container format or an audio data format with defined storage layer.
- Typical audio recording devices tend to sample audio at one of two different rates, 44.1 kHz or 48 kHz.
- Uncompressed audio formats such as WAV, AIFF, AU or raw header-less PCM;
- Opus is a lossy audio coding format and standardized by the Internet Engineering Task Force, designed to efficiently code speech and general audio in a single format, while remaining low-latency enough for real-time interactive communication and low-complexity enough for low-end embedded processors.
- the invention can utilize any lossy compression method.
- Embodiments of the invention can be divided into three separate applications.
- the first is within a Client Server software architecture, i.e., a web application
- the second is within a cloud-based Server which sends and or receives data from the client server software
- the third can be the audio drift normalization where the audio is synchronized.
- the audio drift normalization can be implemented with a different architecture which can provide required inputs and is capable of utilizing the specific output of the audio drift normalization application.
- Typical audio drift algorithms can be selected from the group consisting of drift correction modulation, clock drift compensation techniques and multimedia decoding device.
- European patent specification EP 2038886 B1 inventor Abrol publication date Nov. 4, 2012 discloses various techniques to achieve clock drift compensation for audio decoding and the disclosure of which is incorporated by reference herein for all purposes.
- Client Server software architecture there is shown a Web Application Client running in a Web Browser.
- the Media Stream is provided to a Media Recorder (https://www.w3.org/TR/mediastream-recording)
- a POST request is an arbitrary amount of data of any type that can be sent to the server in the body of the request message.
- a header field in the POST request usually indicates the message body's Internet media type.
- Audio recording devices tend to sample audio at one of two different rates, 44.1 kHz or 48 kHz. Audio files encoded with the Opus codec/encoder and stored within the WebM media file container format is processed by the FFMPEG open source audio/video utility in the following ways to standardize the output files.
- Base64 Clarifying binary-to-text encoding schemes, Base64 is designed to carry data stored in binary formats across channels that only reliably support text content. Base64 is particularly prevalent on the World Wide Web where its uses include the ability to embed image files or other binary assets inside textual assets such as Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) files. However, any similar application/product could be used to achieve similar results.
- HTML Hypertext Markup Language
- CSS Cascading Style Sheets
- WebM is an open, royalty-free, media file format designed for the web.
- WebM defines the file container structure, video and audio formats.
- WebM files consist of video streams compressed with the VP8 or VP9 video codecs and audio streams compressed with the Vorbis audio compression or Opus audio coder-decoder which is a real-time interactive audio coder-decoder.
- Vorbis audio compression or Opus audio coder-decoder which is a real-time interactive audio coder-decoder.
- any similar application/product could be used to achieve similar results.
- a WAV file is a raw audio format that are uncompressed lossless audio and as such can take up quite a bit of space, coming in around 10 MB per minute with a maximum file size of 4 GB.
- any similar application/product could be used to achieve similar results.
- An MP3 file is an audio file saved in a compressed audio format developed by the Moving Picture Experts Group (MPEG) that uses “Layer 3” audio compression.
- MPEG Moving Picture Experts Group
- any similar application/product could be used to achieve similar results.
- WAV and MP3 are protocols for converting the audio or video data to the associated file formats.
- WAV Wide Video Interleave
- FLV Flash Video Format
- WMV Windows Media Video
- MOV Apple QuickTime Movie
- MP4 Moving Pictures Expert Group 4
- codec is a device or computer program for encoding or decoding a digital data stream or signal.
- Codec is a portmanteau of coder-decoder.
- a coder encodes a data stream or a signal for transmission or storage, possibly in encrypted form, and the decoder function reverses the encoding for playback or editing.
- FFMPEG is a free and open-source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams.
- FIGS. 1, 2 and 3 there is shown a Client Server software architecture of the invention web application 1000 .
- the server side is shown in FIG. 2 server application.
- the invention increases the efficiency of the transfer.
- the application of the invention starts at step 10 which resides on the client side of the application on a web application running on a computing device.
- the client computing device being in communication with a microphone.
- the client application requests permission to access the microphone step 20 .
- the media stream is initialized by the web application 1000 as shown in step 30 .
- the media recorder 40 is enabled so as to be capable of receiving the microphone and/or video camera media stream.
- the microphone and/or video camera sends a start recoding signal.
- the start recording signal step 50 is received by the web application 1000
- the client side web application opens a file step 60 and starts media recorder that records data from the microphone and/or camera media stream and stores it in the file.
- the web application 1000 then creates a chunk and appends a name to the file for identification purposes.
- the segments are based on a preset time period.
- the recorded data is then segmented into chunks of data step 70 . Chunks of data can be saved in any reasonable time length of the recording which is preferably from 0.5 to 60 seconds. However, the most preferable length is 8 seconds and is not to exceed 1 Megabyte (MB) in size.
- MB Megabyte
- the recorded data is recorded until said recording device sends stop recording signal to receive the web application 1000 .
- the files are then converted to Base64 encoded raw data chunk and placed in an array buffer in step 80 using the name assigned to the chunk in step 70 .
- the Base64 encoded raw data chunk is then encoded with a date and time stamp in step 90 which is the transfer chunk and that transfer chunk is uploaded to the cloud and stored in a Firestore database.
- any similar database application/product could be used to achieve similar results such as Oracle, Zoho Creator, Quick Base, Amazon Aurora, Kintone, Amazon RDS, Rubrik, Airtable, Knack, TeamDesk, Firebolt or Firestore.
- the media recorder step 40 has a second feature which is continually sensing the device for the stop recording signal step 120 . Once the stop recording signal is received, the web application 1000 processes and sends HTTP Post request step 110 to the cloud server where the chunks are stored in the Firestore database via link A.
- Step 210 receives the HTTP post request sent from the client application step 110 on FIG. 1 and retrieves the data from the Firestore database step 220 .
- the Firestore database can be replaced with any similar application capable of storing text, binary, ASCII, audio or video files.
- the files are organized by the name created in step 60 of FIG. 1 in step 230 .
- Step 240 the application 2000 decodes the Base64 and stores the results in an array buffer. The application 2000 does this for all the chunks of step 230 until the complete recording is decoded and each file is identified by the name created in step 60 of FIG.
- Step 250 takes decoded data from step 240 and concatenates the files into a file buffer where the media, audio or video file is recreated.
- the concatenated file is created it is stored to the server storage medium step 260 in an audiovisual media file format such as webm.
- the server storage medium can be any suitable technology including random and serial data storage media such as a disk drive or electronic media accessible by the server running application 2000 .
- the storage medium does not have to be contained in the same server but can be a networked device.
- step 270 the file stored in an audiovisual media file format in step 260 which is identified by the name created in step 60 of FIG. 1 plus a sequence number or identifier assigned in step 230 .
- Step 280 of application 2000 then operates on the file from step 270 and creates a WAV and or MP3 file in step 290 .
- WAV, MPS, audio, video file, AVI (Audio Video Interleave), FLV (Flash Video Format), WMV (Windows Media Video), MOV (Apple QuickTime Movie), MP4 (Moving Pictures Expert Group 4) are used interchangeably within the specification and are all forms of media files.
- Step 300 then stores the newly created WAV or MP3 file in a cloud storage bucket.
- cloud storage buckets are the basic containers on the Google or Amazon cloud that contains the data. However, any suitable storage medium could be used.
- Step 310 then stores the public URL of the cloud storage bucket to the Firebase real time database however any suitable database application can be employed. Once the WAV or MP3 file and associated public URL are stored the application 2000 ends step 320 .
- the process begins in step 400 where the process receives from step 270 of application 2000 the audiovisual media file format stored in step 260 as shown in FIG. 2 .
- the audiovisual media file format can be any file format which is applicable such as WebM, AVI, MPEG transport stream, Matroska, F4V, Vob, Ogg Video, 3GPP2, SVI, AMV video format, MPEG-1, MPEG-4, MPEG transport stream, QuickTime file format and Windows media video.
- the process places the file into queue 410 within the server storage system comprising of a direct access storage device (DASD) or non-volatile computer storage, an example of which is solid state storage medium for further processing.
- DASD direct access storage device
- non-volatile computer storage an example of which is solid state storage medium for further processing.
- Step 420 utilizes the WebM container media file format files and then processes them with the FFMPEG open source audio and video utility in the following ways to standardize the output files.
- the process determines the sample rate of the file. If the sample rate is not 48 kHz then the file is resampled to 48 kHz in step 440 . Resampling, in terms of audio files, is also known as Sample Rate Conversion. This is done when you need to convert a digital audio file from a given sample rate into a different sample rate. Sample rate is the number of samples of audio carried per second, measured in Hz or kHz (one kHz being 1,000 Hz). If the file is 48 kHz, the file is then passed to step 450 .
- the audio drift normalization process has now synchronized the wave forms and determines if the input file is in stereo or mono 450 . If the input file is stereo, the process sums the file to mono 460 else the file is passed to step 470 .
- Summing a stereo media file to mono means taking both channels of your stereo media file (left and right) and creating a composite single mono media file.
- the file is converted to media file such as a WAV and MP3 container formats. However, any applicable media file such as a WAV, MPS, AVI, FLV, WMV, MOV, MP3 and MP4 is acceptable.
- the process then saves the media file to disk step 480 .
- any suitable storage medium can be substituted such as a database, files or stacks all of which can reside on a Direct Access Storage Device (DASD) or non-volatile computer storage such as a solid state storage medium.
- DASD Direct Access Storage Device
- the WAV and MP3 files are stored in separate files step 490 for further processing.
- the process then returns control of the file to application 2000 step 280 as shown in FIG. 2 .
- the audio drift normalization can be implemented separately from the recording applications 1000 and 2000 shown in FIG. 1 and FIG. 2 .
- the utilizing application must provide the correct file format and be able to receive the file format being returned from application 3000 .
- the system, method or methods described above may be executed or carried out by a computing system including a tangible computer-readable storage medium, also described herein as a storage machine, that holds machine-readable instructions executable by a logic machine (i.e. a processor or programmable control device) to provide, implement, perform, and/or enact the above described methods, processes and/or tasks.
- a logic machine i.e. a processor or programmable control device
- the state of the storage machine may be changed to hold different data.
- the storage machine may include memory devices such as various hard disk drives, CD, flash drives, cloud storage, or DVD devices.
- the logic machine may execute machine-readable instructions via one or more physical information and/or logic processing devices.
- the logic machine may be configured to execute instructions to perform tasks for a computer program.
- the logic machine may include one or more processors to execute the machine-readable instructions.
- the computing system may include a display subsystem to display a graphical user interface (GUI) or any visual element of the methods or processes described above.
- GUI graphical user interface
- the display subsystem, storage machine, and logic machine may be integrated such that the above method may be executed while visual elements of the disclosed system and/or method are displayed on a display screen for user consumption.
- the computing system may include an input subsystem that receives user input.
- the input subsystem may be configured to connect to and receive input from devices such as a mouse, keyboard, gaming controller microphone or camera.
- a user input may indicate a request that a certain task is to be executed by the computing system, such as requesting the computing system to display any of the above described information, or requesting that the user inputs updates or modifies existing stored information for processing.
- a communication subsystem may allow the methods described above to be executed or provided over a computer network.
- the communication subsystem may be configured to enable the computing system to communicate with a plurality of personal computing devices.
- the communication subsystem may include wired and/or wireless communication devices to facilitate networked communication.
- the described methods or processes may be executed, provided, or implemented for a user or one or more computing devices via a computer-program product such as via an application programming interface (API).
- API application programming interface
Abstract
Description
- This application claims priority from U.S. Provisional Patent Application No. 62/947,490, filed Dec. 12, 2019, titled SIMULTANEOUS RECORDING AND UPLOADING OF MULTIPLE AUDIO FILES OF THE SAME CONVERSATION AND AUDIO DRIFT NORMALIZATION SYSTEMSAND METHODS the disclosure of which is incorporated by reference herein for all purposes.
- The invention relates to simultaneous recording and uploading systems and methods, and, more particularly to a simultaneous recording and uploading of multiple files from the same conversation. Further, the present invention relates to audio drift normalization, and more particularly to audio drift normalization systems and methods that can normalize audio drift of a plurality of recordings from a source.
- A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyrights whatsoever.
- As advances in internet communications including video and audio mediums continue to increase, the need to effectively and efficiently communicate and create media across a broad spectrum of platforms has also increased. Many of the communications platforms configured to communicate recordings cannot ensure that the data is accessible in the event of a problem with the recording process and minimize or eliminate the need to record the complete session prior to uploading or produce quality media recordings.
- Previous attempts to create systems and methods to ensure that the data is accessible in the event of a problem with the recording process and minimize or eliminate the need to record the complete session prior to uploading or produce quality media recordings, have been unsuccessful, time consuming and costly.
- These previous attempts have necessitated additional hardware, systems, time expenditure, and costs. Previous attempts to address these problems have not been efficient or successful.
- In many instances, podcasters, audio book producers, and content creators find themselves spending time, effort, and extra costs to produce quality video and/or audio recordings. These additional logistical steps increase operating costs and increase the time needed to publish video and/or audio works. These previous methods needed to record an audio and/or video file in its entirety before being uploaded to the cloud.
- Further, these methods require additional system resources to be present within a recording device thereby increasing overhead of systems and methods.
- Quite often recording professionals find themselves spending additional time waiting for large files to upload after a recording is finished.
- Many of the communications platforms configured to communicate recordings require systems, methods, and logistical intervention to correct or adjust recordings for irregularities. Some of these irregularities include drift between multiple collaborators, lack of synchronization, and data inconsistency.
- Previous attempts to create systems and methods to correct or minimize drift in recordings have necessitated additional hardware, systems, time expenditure, and costs. Previous attempts to address these problems have not been successful.
- Further, imperfections in recordings can cause drift. Specifically, audio drift requires efforts to realign each audio file to the actual timing for it to be usable in products, adding time and work for producers and audio engineers in post-production.
- Quite often recording professionals find themselves realigning audio recordings in post-production to correct recordings with drift or that are out of sync.
- When multiple systems are recording, and two or more audio recordings of the same conversation are captured on multiple devices, reconstruction and integration of the recordings require a process to normalize the drift that arises between the two or more audio recordings. Therefore, audio drift results in the need to realign each audio file to the actual timing for a usable product. This adds time and work for producers and audio engineers in post-production.
- Another problem can occur when the recording process utilizes segments or chunks of data based on a preset time period and the recording being transferred at different times. The recording assembly system must take all the chunks and write them into one file. In doing so they need to ensure that the data is captured, is continuous, and mixed together. This assembly can result in audio drift.
- Additionally, combining recordings from a plurality of sources often requires alignment of the waveforms to eliminate the over writing of each other when they are out of phase. This condition requires additional steps, costs, and time to publish works.
- What is needed in the art is a solution to address issues with recording and uploading of audio and video files. Additionally, there is a need for a solution to address audio drift issues.
- The invention in one form is directed to a method that ensures that recorded audio and video files are always accessible in the event of a catastrophe, removes the need for the audio and video file to exist in its entirety before uploading and requires less system resources be present within the recording device.
- The invention in another form improves the reliable access to complete audio files and reduces time waiting for large files to upload after a recording is finished.
- The invention in yet another form allows for the use of low-end devices due to the efficiency of the recording system.
- In yet another form, the invention improves over existing chunk style recording methods.
- In another embodiment, the system can also include algorithms and/or software which eliminate manual processes to perform system processes and utilize methods.
- In yet another embodiment, the system can include methods to perform chunk style recording to produce synchronized recordings.
- In an aspect, the system can include processes to normalize audio drift that arises between two or more audio recordings of an original source that is captured on one or a plurality of devices.
- In another aspect, the system can also include components that are configured to minimize human interaction and/or executed steps to produce enhanced audio and video quality recordings. These recordings are designed to produce top grade audio recordings, the recordings configured to be combined in other audio and/or video recordings without the need for back end engineering to improve quality of the recordings.
- In yet another aspect, the system can include systems configured to provide near real-time interview capability with video and record everyone's audio in separate media files without drift.
- In an embodiment, the system can include a platform which provides a medium to connect podcast hosts, co-hosts, & guests to record studio quality audio and video from anywhere.
- In another embodiment, the system can also include allowing system users to download, listen, and share one or all of user's recordings organized by any parameter, including but not limited to session, date, geography and/or recording medium.
- In yet another embodiment, the system can also include a video feature so a podcaster and a guest can see each other while recording.
- In an aspect, the system can include mobile applications and can be utilized on a mobile device.
- In another aspect, the system can also include laptops, desktop computers, and computing mediums.
- In yet another aspect, the system can include utilizing a standardized 48 kHz sampling rate.
- In an embodiment, the system can include processes which do not require echo cancellation (Acoustic Echo Cancellation (AEC) Software).
- In another embodiment, the system can also include remote audio signals and can exist on the cloud/web within a network.
- In yet another embodiment, the system can include saving recordings in buffers of any length of time, preferably between 2 and 8 seconds. The system can also utilize resampling to 48 kHz.
- In an aspect, the system can include longform recordings, exceeding one hour in length.
- In an aspect, the system can reduce the need for local storage and is unbounded in the duration of recording.
- In another aspect, the system can also include components and/or methods which are non-destructive wherein no data is lost.
- In yet another aspect, the system can include systems and/or methods that do not utilize a master clock or drift threshold.
- In an embodiment, the system can include methods and/or components which normalize audio drift from different sources.
- In another embodiment, the system can include quality, reliable, repeatable, systems efficient enough to utilize low end devices.
- In another embodiment, the system can be used alongside aviation flight data recording systems to reduce the need to search for the “black box” because the recordings are not stored locally.
- According to an aspect of the present invention, an audio drift normalization system is presented. The system can ensure that recorded audio files are automatically modified to correct audio drift arising from a plurality of recording devices and/or buffer size differences.
- In another embodiment, the system can also include algorithms and/or software which eliminate manual processes to correct audio drift.
- In yet another embodiment, the system can include methods to perform chunk style recording to produce synchronized recordings.
- In an aspect, the system can include processes to normalize audio drift that arises between two or more audio recordings of an original source that is captured on one or a plurality of devices.
- These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
- The features and advantages of the present invention will be better understood when the Detailed Description of the Preferred Embodiments given below is considered in conjunction with the figures provided.
-
FIG. 1 is a method flowchart for simultaneous recording audio files in accordance with an embodiment of the present invention; -
FIG. 2 is a method flowchart for simultaneous recording audio files in accordance with an embodiment of the present invention; and -
FIG. 3 is a method flow chart for audio drift normalization in accordance with an embodiment of the present invention. - The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout.
- With respect to the term chunks when used by the instant invention may refer to segments of audio and/or video recordings.
- The terms media recordings, audio recordings, video recordings may be used interchangeably and mean a recording of the using a media capture device such as a microphone or video camera.
- The terms media data media files, audio data, audio files, video data, video files may be used interchangeably and mean data stored on a computer representing the data of the captured media.
- The term WAV, MP3, MPS, audio, video file, AVI (Audio Video Interleave), FLV (Flash Video Format), WMV (Windows Media Video), MOV (Apple QuickTime Movie), MP4 (Moving Pictures Expert Group 4) may be used interchangeably within the specification and are all forms of media files.
- The term cloud when used in context by the instant invention can mean Amazon cloud, Google cloud, Verizon cloud or any suitable cloud storage system.
- The term computing device capable may include any type of device for computing, such as a tablet, mobile device, smart speaker, flight data recorder, or computer that may be in communication with a media recording device such as a microphone or video camera.
- The term public URL is a URL that can be asked by anyone on the Internet as it is indexed by search engines.
- The term video file format is a type of file format for storing digital video data on a storage device such as a computer system. Video is almost always stored using compression to reduce the file size. A video file normally consists of a container containing video data in a video coding format alongside audio data in an audio coding format.
- The term stereo commonly refers to Stereophonic sound and is the reproduction of sound, using two or more independent audio channels, through a symmetrical configuration of loudspeakers, in such a way as to create a pleasant and natural impression of sound heard from various directions, as in natural hearing.
- The term monophonic sound commonly called mono sound, mono, or non-stereo sound, this early sound system used a single channel of audio for sound output. In monophonic sound systems, the signal sent to the sound system encodes one single stream of sound and it usually uses just one speaker or the same exact sound in both speakers.
- When the term database is used within the specification it can alternatively mean any suitable storage medium can be substituted such as a data structures, files, stacks utilizing Direct access storage device (DASD) or non-volatile computer storage an example of which is solid state storage medium.
- The term audio file format is a file format for storing digital audio data on a computer system. The bit layout of the audio data (excluding metadata) is called the audio coding format and can be uncompressed, or compressed to reduce the file size, often using compression. The data can be a raw bitstream in an audio coding format, but it is usually embedded in a container format or an audio data format with defined storage layer.
- Typical audio recording devices tend to sample audio at one of two different rates, 44.1 kHz or 48 kHz.
- There are three major groups of audio file formats:
- Uncompressed audio formats, such as WAV, AIFF, AU or raw header-less PCM;
- Formats with lossless compression, such as FLAC, Monkey's Audio (filename extension .ape), WavPack (filename extension .wv), TTA, ATRAC Advanced Lossless, ALAC (filename extension .m4a), MPEG-4 SLS, MPEG-4 ALS, MPEG-4 DST, Windows Media Audio Lossless (WMA Lossless), and Shorten (SHN).
- Formats with lossy compression, such as Opus, MP3, Vorbis, Musepack, AAC, ATRAC and Windows Media Audio Lossy (WMA lossy).
- Specifically, Opus is a lossy audio coding format and standardized by the Internet Engineering Task Force, designed to efficiently code speech and general audio in a single format, while remaining low-latency enough for real-time interactive communication and low-complexity enough for low-end embedded processors. However, when used in the specification the invention can utilize any lossy compression method.
- Any embodiment of the invention disclosed can be used with any media, audio or video file format and all terms are used interchangeably.
- Embodiments of the invention can be divided into three separate applications. The first is within a Client Server software architecture, i.e., a web application, and the second is within a cloud-based Server which sends and or receives data from the client server software, and the third can be the audio drift normalization where the audio is synchronized. However, the audio drift normalization can be implemented with a different architecture which can provide required inputs and is capable of utilizing the specific output of the audio drift normalization application.
- Typical audio drift algorithms can be selected from the group consisting of drift correction modulation, clock drift compensation techniques and multimedia decoding device. European patent specification EP 2038886 B1 inventor Abrol publication date Nov. 4, 2012 discloses various techniques to achieve clock drift compensation for audio decoding and the disclosure of which is incorporated by reference herein for all purposes.
- Referring now to the Client Server software architecture there is shown a Web Application Client running in a Web Browser.
- A) Permission is requested for access to a Microphone and/or Video Camera Media Device
- B) Upon permission being granted, a Media Stream (https://w3c.github.io/mediacapture-main/#stream-api) is initialized
- C) The Media Stream is provided to a Media Recorder (https://www.w3.org/TR/mediastream-recording)
- D) Upon Start recording:
-
- a. the Media Recorder is configured to slice “Chunks” of the recording into Chunks of data which can be saved in any reasonable time length of the recording which is preferably from 0.5 to 60 seconds and is not to exceed 1 Megabyte (MB) in size. However, the most preferable length is 8 seconds,
- b. the File_name is created.
- Every 8 seconds thereafter each “Chunk” is:
-
- a. Encoded in Base64 format to convert it from a raw Array Buffer to a String,
- b. Given a Date Time Stamp,
- c. Saved to a Collection in a cloud database such as but not limited to Google Cloud Platform's Firestore Database.
- Upon Stop recording an HTTP Post Request is sent to a server running in the Cloud containing:
-
- a. The creator's identification/authorization token.
- b. The reference path to the Collection on “Chunks” within a database
- c. The File_name.
- A POST request is an arbitrary amount of data of any type that can be sent to the server in the body of the request message. A header field in the POST request usually indicates the message body's Internet media type.
- Therefore, referring to the Server running in the Cloud:
-
- a. The reference path to the Collection on “Chunks” within Google Cloud Platform's Firestore Database is used to Get the “Chunks” ordered by the Date Time Stamp.
- b. Each “Chunk” is Decoded from Base64 back to Array Buffer.
- c. The Array Buffers are concatenated into a File Buffer.
- d. The File Buffer is saved to disk using a multi-media container format with the provided File_name.
- e. Audio Drift Normalization—the saved file from the multi-media container can be converted to an audio and/or video file format and then is normalized without reduction of quality using an Audio drift normalization application.
- f. Media files are uploaded to a Cloud Storage Bucket within Google Cloud Platform with Public URLs being provided in return. The media formats can alternatively be any audio or video file format which provides an efficient data encoding format.
- g. The Public URLs of the media files are saved to the Application Data within a database such as Google Cloud Platform's Firebase Real-Time Database using the Reference Path.
- Referring to the audio drift normalization phase:
- Audio recording devices tend to sample audio at one of two different rates, 44.1 kHz or 48 kHz. Audio files encoded with the Opus codec/encoder and stored within the WebM media file container format is processed by the FFMPEG open source audio/video utility in the following ways to standardize the output files.
-
- a. If the Sample Rate is 44.1 kHz then resampled it to 48 kHz;
- b. Synchronizing the wave forms if the input is stereo then sum it to mono;
- c. Converts to WAV and MP3 container formats;
- d. Saves WAV and MP3 files to disk.
- Clarifying binary-to-text encoding schemes, Base64 is designed to carry data stored in binary formats across channels that only reliably support text content. Base64 is particularly prevalent on the World Wide Web where its uses include the ability to embed image files or other binary assets inside textual assets such as Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) files. However, any similar application/product could be used to achieve similar results.
- It should be noted that WebM is an open, royalty-free, media file format designed for the web. WebM defines the file container structure, video and audio formats. WebM files consist of video streams compressed with the VP8 or VP9 video codecs and audio streams compressed with the Vorbis audio compression or Opus audio coder-decoder which is a real-time interactive audio coder-decoder. However, any similar application/product could be used to achieve similar results.
- A WAV file is a raw audio format that are uncompressed lossless audio and as such can take up quite a bit of space, coming in around 10 MB per minute with a maximum file size of 4 GB. However, any similar application/product could be used to achieve similar results.
- An MP3 file is an audio file saved in a compressed audio format developed by the Moving Picture Experts Group (MPEG) that uses “Layer 3” audio compression. However, any similar application/product could be used to achieve similar results.
- WAV and MP3 are protocols for converting the audio or video data to the associated file formats.
- The term WAV, MPS, audio, video file, AVI (Audio Video Interleave), FLV (Flash Video Format), WMV (Windows Media Video), MOV (Apple QuickTime Movie), MP4 (Moving Pictures Expert Group 4) are used interchangeably within the specification and are all forms of media files.
- The term codec is a device or computer program for encoding or decoding a digital data stream or signal. Codec is a portmanteau of coder-decoder. A coder encodes a data stream or a signal for transmission or storage, possibly in encrypted form, and the decoder function reverses the encoding for playback or editing.
- The term FFMPEG is a free and open-source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams.
- Referring now to the drawings as shown in
FIGS. 1, 2 and 3 , and more particularly toFIG. 1 , there is shown a Client Server software architecture of theinvention web application 1000. The server side is shown inFIG. 2 server application. By splitting the invention into two separate components, client side and web side, the invention increases the efficiency of the transfer. - The application of the invention starts at
step 10 which resides on the client side of the application on a web application running on a computing device. The client computing device being in communication with a microphone. The client application then requests permission to access themicrophone step 20. Once permission has been received to access the microphone or video capture device, the media stream is initialized by theweb application 1000 as shown instep 30. Themedia recorder 40 is enabled so as to be capable of receiving the microphone and/or video camera media stream. The microphone and/or video camera sends a start recoding signal. When the startrecording signal step 50 is received by theweb application 1000, then the client side web application opens afile step 60 and starts media recorder that records data from the microphone and/or camera media stream and stores it in the file. Theweb application 1000 then creates a chunk and appends a name to the file for identification purposes. The segments are based on a preset time period. The recorded data is then segmented into chunks of data step 70. Chunks of data can be saved in any reasonable time length of the recording which is preferably from 0.5 to 60 seconds. However, the most preferable length is 8 seconds and is not to exceed 1 Megabyte (MB) in size. - The recorded data is recorded until said recording device sends stop recording signal to receive the
web application 1000. Then the chunks of data represented by the files with the name created instep 60 and an appended sequence number which starts a 1 and the sequence number stop with an integer value equal to the number of files, therefore sequence number two would equal 1+1 which equals 2. The files are then converted to Base64 encoded raw data chunk and placed in an array buffer instep 80 using the name assigned to the chunk instep 70. The Base64 encoded raw data chunk is then encoded with a date and time stamp instep 90 which is the transfer chunk and that transfer chunk is uploaded to the cloud and stored in a Firestore database. However, any similar database application/product could be used to achieve similar results such as Oracle, Zoho Creator, Quick Base, Amazon Aurora, Kintone, Amazon RDS, Rubrik, Airtable, Knack, TeamDesk, Firebolt or Firestore. - The
media recorder step 40 has a second feature which is continually sensing the device for the stoprecording signal step 120. Once the stop recording signal is received, theweb application 1000 processes and sends HTTPPost request step 110 to the cloud server where the chunks are stored in the Firestore database via link A. - Referring now to
FIG. 2 , there is shown the cloud server application where the chunks of data are stored. Step 210 receives the HTTP post request sent from theclient application step 110 onFIG. 1 and retrieves the data from theFirestore database step 220. The Firestore database can be replaced with any similar application capable of storing text, binary, ASCII, audio or video files. Once all the chunks of data have been received, the files are organized by the name created instep 60 ofFIG. 1 instep 230. Step 240 theapplication 2000 decodes the Base64 and stores the results in an array buffer. Theapplication 2000 does this for all the chunks ofstep 230 until the complete recording is decoded and each file is identified by the name created instep 60 ofFIG. 1 instep 230 plus a sequence number or identifier. Step 250 takes decoded data fromstep 240 and concatenates the files into a file buffer where the media, audio or video file is recreated. Once the concatenated file is created it is stored to the serverstorage medium step 260 in an audiovisual media file format such as webm. The server storage medium can be any suitable technology including random and serial data storage media such as a disk drive or electronic media accessible by theserver running application 2000. The storage medium does not have to be contained in the same server but can be a networked device. Instep 270 the file stored in an audiovisual media file format instep 260 which is identified by the name created instep 60 ofFIG. 1 plus a sequence number or identifier assigned instep 230. Step 280 ofapplication 2000 then operates on the file fromstep 270 and creates a WAV and or MP3 file instep 290. The term WAV, MPS, audio, video file, AVI (Audio Video Interleave), FLV (Flash Video Format), WMV (Windows Media Video), MOV (Apple QuickTime Movie), MP4 (Moving Pictures Expert Group 4) are used interchangeably within the specification and are all forms of media files. Step 300 then stores the newly created WAV or MP3 file in a cloud storage bucket. The term cloud storage buckets are the basic containers on the Google or Amazon cloud that contains the data. However, any suitable storage medium could be used. Step 310 then stores the public URL of the cloud storage bucket to the Firebase real time database however any suitable database application can be employed. Once the WAV or MP3 file and associated public URL are stored theapplication 2000 endsstep 320. - Referring to
FIG. 3 , there is shown the audiodrift normalization process 280 fromFIG. 2 . The process begins instep 400 where the process receives fromstep 270 ofapplication 2000 the audiovisual media file format stored instep 260 as shown inFIG. 2 . The audiovisual media file format can be any file format which is applicable such as WebM, AVI, MPEG transport stream, Matroska, F4V, Vob, Ogg Video, 3GPP2, SVI, AMV video format, MPEG-1, MPEG-4, MPEG transport stream, QuickTime file format and Windows media video. The process then places the file intoqueue 410 within the server storage system comprising of a direct access storage device (DASD) or non-volatile computer storage, an example of which is solid state storage medium for further processing. Step 420 utilizes the WebM container media file format files and then processes them with the FFMPEG open source audio and video utility in the following ways to standardize the output files. Instep 430 the process determines the sample rate of the file. If the sample rate is not 48 kHz then the file is resampled to 48 kHz instep 440. Resampling, in terms of audio files, is also known as Sample Rate Conversion. This is done when you need to convert a digital audio file from a given sample rate into a different sample rate. Sample rate is the number of samples of audio carried per second, measured in Hz or kHz (one kHz being 1,000 Hz). If the file is 48 kHz, the file is then passed to step 450. The audio drift normalization process has now synchronized the wave forms and determines if the input file is in stereo ormono 450. If the input file is stereo, the process sums the file tomono 460 else the file is passed to step 470. Summing a stereo media file to mono means taking both channels of your stereo media file (left and right) and creating a composite single mono media file. Instep 470, the file is converted to media file such as a WAV and MP3 container formats. However, any applicable media file such as a WAV, MPS, AVI, FLV, WMV, MOV, MP3 and MP4 is acceptable. The process then saves the media file todisk step 480. However, any suitable storage medium can be substituted such as a database, files or stacks all of which can reside on a Direct Access Storage Device (DASD) or non-volatile computer storage such as a solid state storage medium. The WAV and MP3 files are stored in separate files step 490 for further processing. The process then returns control of the file toapplication 2000step 280 as shown inFIG. 2 . - The audio drift normalization can be implemented separately from the
recording applications FIG. 1 andFIG. 2 . The utilizing application must provide the correct file format and be able to receive the file format being returned fromapplication 3000. - In some embodiments, the system, method or methods described above may be executed or carried out by a computing system including a tangible computer-readable storage medium, also described herein as a storage machine, that holds machine-readable instructions executable by a logic machine (i.e. a processor or programmable control device) to provide, implement, perform, and/or enact the above described methods, processes and/or tasks. When such methods and processes are implemented, the state of the storage machine may be changed to hold different data. For example, the storage machine may include memory devices such as various hard disk drives, CD, flash drives, cloud storage, or DVD devices. The logic machine may execute machine-readable instructions via one or more physical information and/or logic processing devices. For example, the logic machine may be configured to execute instructions to perform tasks for a computer program. The logic machine may include one or more processors to execute the machine-readable instructions. The computing system may include a display subsystem to display a graphical user interface (GUI) or any visual element of the methods or processes described above. For example, the display subsystem, storage machine, and logic machine may be integrated such that the above method may be executed while visual elements of the disclosed system and/or method are displayed on a display screen for user consumption. The computing system may include an input subsystem that receives user input. The input subsystem may be configured to connect to and receive input from devices such as a mouse, keyboard, gaming controller microphone or camera. For example, a user input may indicate a request that a certain task is to be executed by the computing system, such as requesting the computing system to display any of the above described information, or requesting that the user inputs updates or modifies existing stored information for processing. A communication subsystem may allow the methods described above to be executed or provided over a computer network. For example, the communication subsystem may be configured to enable the computing system to communicate with a plurality of personal computing devices. The communication subsystem may include wired and/or wireless communication devices to facilitate networked communication. The described methods or processes may be executed, provided, or implemented for a user or one or more computing devices via a computer-program product such as via an application programming interface (API).
- Since many modifications, variations, and changes in detail can be made to the described embodiments of the invention, it is intended that all matters in the foregoing description and shown in the accompanying drawings be interpreted as illustrative and not in a limiting sense. Furthermore, it is understood that any of the features presented in the embodiments may be integrated into any of the other embodiments unless explicitly stated otherwise. The scope of the invention should be determined by the appended claims and their legal equivalents.
- The present invention has been described with reference to embodiments, it should be noted and understood that various modifications and variations can be crafted by those skilled in the art without departing from the scope and spirit of the invention. Accordingly, the foregoing disclosure should be interpreted as illustrative only and is not to be interpreted in a limiting sense. Further it is intended that any other embodiments of the present invention that result from any changes in application or method of use or operation, method of manufacture, shape, size, or materials which are not specified within the detailed written description or illustrations contained herein are considered within the scope of the present invention.
- Insofar as the description above and the accompanying drawings, disclose any additional subject matter that is not within the scope of the claims below, the inventions are not dedicated to the public and the right to file one or more applications to claim such additional inventions is reserved.
- Although very narrow claims are presented herein, it should be recognized that the scope of this invention is much broader than presented by the claim. It is intended that broader claims will be submitted in an application that claims the benefit of priority from this application.
Claims (4)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/119,784 US11388489B2 (en) | 2019-12-12 | 2020-12-11 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
US17/858,363 US11876850B2 (en) | 2019-12-12 | 2022-07-06 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962947490P | 2019-12-12 | 2019-12-12 | |
US17/119,784 US11388489B2 (en) | 2019-12-12 | 2020-12-11 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/858,363 Continuation-In-Part US11876850B2 (en) | 2019-12-12 | 2022-07-06 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210185415A1 true US20210185415A1 (en) | 2021-06-17 |
US11388489B2 US11388489B2 (en) | 2022-07-12 |
Family
ID=76317625
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/119,764 Active US11540030B2 (en) | 2019-12-12 | 2020-12-11 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
US17/119,784 Active US11388489B2 (en) | 2019-12-12 | 2020-12-11 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
US18/061,609 Pending US20230156298A1 (en) | 2019-12-12 | 2022-12-05 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/119,764 Active US11540030B2 (en) | 2019-12-12 | 2020-12-11 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US18/061,609 Pending US20230156298A1 (en) | 2019-12-12 | 2022-12-05 | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods |
Country Status (2)
Country | Link |
---|---|
US (3) | US11540030B2 (en) |
WO (1) | WO2021119488A1 (en) |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5996022A (en) * | 1996-06-03 | 1999-11-30 | Webtv Networks, Inc. | Transcoding data in a proxy computer prior to transmitting the audio data to a client |
US6625656B2 (en) * | 1999-05-04 | 2003-09-23 | Enounce, Incorporated | Method and apparatus for continuous playback or distribution of information including audio-visual streamed multimedia |
US20100077022A1 (en) * | 2001-06-27 | 2010-03-25 | John Mikkelsen | Media delivery platform |
US8009697B2 (en) * | 2003-07-07 | 2011-08-30 | Broadcom Corporation | Method and apparatus for segmentation of messages in a communications system |
US8285403B2 (en) * | 2004-03-04 | 2012-10-09 | Sony Corporation | Mobile transcoding architecture |
US20050276572A1 (en) * | 2004-06-14 | 2005-12-15 | Rok Productions Limited | Providing audio-visual content |
US10394757B2 (en) * | 2010-11-18 | 2019-08-27 | Microsoft Technology Licensing, Llc | Scalable chunk store for data deduplication |
WO2013033242A1 (en) * | 2011-08-29 | 2013-03-07 | Latakoo, Inc. | Compressing, transcoding, sending, and retrieving video and audio files in a server-based system |
US9648295B2 (en) * | 2014-07-18 | 2017-05-09 | Pankaj Sharma | System and methods for simultaneously capturing audio and image data for digital playback |
JP6728154B2 (en) * | 2014-10-24 | 2020-07-22 | ドルビー・インターナショナル・アーベー | Audio signal encoding and decoding |
US20180301169A1 (en) * | 2015-02-24 | 2018-10-18 | Plaay, Llc | System and method for generating a highlight reel of a sporting event |
US20170105039A1 (en) * | 2015-05-05 | 2017-04-13 | David B. Rivkin | System and method of synchronizing a video signal and an audio stream in a cellular smartphone |
US10742733B2 (en) * | 2015-10-12 | 2020-08-11 | Timecode Systems Limited | Synchronizing data between media devices |
US10623801B2 (en) * | 2015-12-17 | 2020-04-14 | James R. Jeffries | Multiple independent video recording integration |
WO2017176657A1 (en) * | 2016-04-06 | 2017-10-12 | Tudor Charles R | Video broadcasting through selected video hosts |
US10326817B2 (en) * | 2016-12-20 | 2019-06-18 | Cisco Technology, Inc. | System and method for quality-aware recording in large scale collaborate clouds |
US10423382B2 (en) * | 2017-12-12 | 2019-09-24 | International Business Machines Corporation | Teleconference recording management system |
US10803865B2 (en) * | 2018-06-05 | 2020-10-13 | Voicify, LLC | Voice application platform |
US10880434B2 (en) * | 2018-11-05 | 2020-12-29 | Nice Ltd | Method and system for creating a fragmented video recording of events on a screen using serverless computing |
-
2020
- 2020-12-11 US US17/119,764 patent/US11540030B2/en active Active
- 2020-12-11 WO PCT/US2020/064603 patent/WO2021119488A1/en active Application Filing
- 2020-12-11 US US17/119,784 patent/US11388489B2/en active Active
-
2022
- 2022-12-05 US US18/061,609 patent/US20230156298A1/en active Pending
Also Published As
Publication number | Publication date |
---|---|
US20210185112A1 (en) | 2021-06-17 |
US11388489B2 (en) | 2022-07-12 |
US20230156298A1 (en) | 2023-05-18 |
WO2021119488A1 (en) | 2021-06-17 |
US11540030B2 (en) | 2022-12-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9478256B1 (en) | Video editing processor for video cloud server | |
US20190069006A1 (en) | Seeking in live-transcoded videos | |
US20050044499A1 (en) | Method for capturing, encoding, packaging, and distributing multimedia presentations | |
US20140192200A1 (en) | Media streams synchronization | |
TW201545551A (en) | Audio and video media files composition method and server system | |
US20150371426A1 (en) | Motion covers | |
WO2023116254A1 (en) | Live video recording method, apparatus and system, and terminal device | |
US20220030286A1 (en) | Data storage server with on-demand media subtitles | |
US20100104267A1 (en) | System and method for playing media file | |
CA3159656A1 (en) | Distributed network recording system with synchronous multi-actor recording | |
US11388489B2 (en) | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods | |
US20130060925A1 (en) | Content distribution system, content distribution server, content distribution method, software program, and storage medium | |
WO2012092901A2 (en) | Media storage system and method | |
US11876850B2 (en) | Simultaneous recording and uploading of multiple audio files of the same conversation and audio drift normalization systems and methods | |
KR102248097B1 (en) | Method for transmiting contents and terminal apparatus using the same | |
US9113150B2 (en) | System and method for recording collaborative information | |
US11611609B2 (en) | Distributed network recording system with multi-user audio manipulation and editing | |
KR20010070867A (en) | Web streaming mechanism and data structure of multimedia contents for on-line presentation, and a system structure for transfer-management in computer networks | |
RU2690163C2 (en) | Information processing device and information processing method | |
US20180316926A1 (en) | Transcoding streamed film | |
KR20150086749A (en) | Apparatus and method for media production based on cloud | |
US20230377606A1 (en) | Video editing projects using single bundled video files | |
Xin et al. | Live Signal Recording and Segmenting Solution Based on Cloud Architecture | |
Chernyshev | Library for Remote Copying of Video File Fragments | |
CN116055765A (en) | Media playing method, system and related equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SQUADCAST, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MORENO, ZACHARIAH STEVEN;REEL/FRAME:054622/0852 Effective date: 20201211 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: SQUADCAST STUDIOS, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SQUADCAST, INC.;REEL/FRAME:063927/0023 Effective date: 20230612 |