WO2019227751A1 - 媒体文件转换方法、装置及存储介质 - Google Patents

媒体文件转换方法、装置及存储介质 Download PDF

Info

Publication number
WO2019227751A1
WO2019227751A1 PCT/CN2018/103581 CN2018103581W WO2019227751A1 WO 2019227751 A1 WO2019227751 A1 WO 2019227751A1 CN 2018103581 W CN2018103581 W CN 2018103581W WO 2019227751 A1 WO2019227751 A1 WO 2019227751A1
Authority
WO
WIPO (PCT)
Prior art keywords
media
container
media data
media file
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/103581
Other languages
English (en)
French (fr)
Inventor
银国徽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing ByteDance Network Technology Co Ltd
Original Assignee
Beijing ByteDance Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing ByteDance Network Technology Co Ltd filed Critical Beijing ByteDance Network Technology Co Ltd
Priority to JP2020552076A priority Critical patent/JP7068489B2/ja
Publication of WO2019227751A1 publication Critical patent/WO2019227751A1/zh
Priority to US16/913,821 priority patent/US11064269B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233—Processing of audio elementary streams
    • H04N21/2335—Processing of audio elementary streams involving reformatting operations of audio signals, e.g. by converting from one coding standard to another
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234336—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4341—Demultiplexing of audio and video streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440236—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments

Definitions

  • the present disclosure relates to multimedia technology, and in particular, to a method, device, and storage medium for media file conversion.
  • the Moving Picture Experts Group (MPEG, Moving Picture Experts Group) -4 package format (MP4 format for short in this article) is a widely used container format that can store video data and audio data in multiple encoding methods.
  • the MP4 package format is a non-streaming media format.
  • Files that use the MP4 package format (herein referred to as MP4 files) do not support playback in the form of streaming media (that is, while downloading and playing). For this reason, related technologies have to convert MP4 files are converted to streaming media formats, such as FLV (Flash Video), Hypertext Transfer Protocol Live Streaming (HLS, HTTP Live Streaming), etc., so as to achieve playback in the form of streaming media.
  • FLV Flash Video
  • HLS Hypertext Transfer Protocol Live Streaming
  • HTTP Live Streaming etc.
  • embodiments of the present disclosure provide a method, device and storage medium for media file conversion, capable of converting a non-streaming media file into a segmented media file of smaller size that can be independently decoded and adapted for network transmission. .
  • An embodiment of the present disclosure provides a method for converting a media file, including: parsing metadata encapsulated in a metadata container of the media file to obtain media information for describing the media data encapsulated in the media data container of the media file;
  • the media file uses a non-streaming media format; finds media data of a given time period according to the time of the media data indicated by the media information; and positions of the media data in the media file according to the media information To find the position of the media data of the given time period in the media file; extract the media data corresponding to the position from the media data container of the media file; describe the extracted media data and describe the media
  • the metadata of the data is packaged according to the packaging structure of the segmented media file to obtain a segmented media file that can be used for independent decoding.
  • An embodiment of the present disclosure further provides a media file conversion device, including: a parsing module configured to parse metadata encapsulated in a metadata container of a media file to obtain a media encapsulated in a media data container for describing the media file Media information of the data, the media file adopts a non-streaming media format; a search module that searches for media data of a given time period according to the time of the media data indicated by the media information; the search module is configured to The position of the media data in the media file indicated by the media information, looking for the position of the media data in the given time period in the media file; an extraction module configured to extract the media data from the media data container of the media file Extracting media data corresponding to the location; a synthesis module configured to encapsulate the extracted media data and metadata describing the media data according to the packaging structure of the segmented media file to obtain segments that can be used for independent decoding Media files.
  • a parsing module configured to parse metadata encapsulated in a metadata container of
  • an embodiment of the present disclosure further provides a media file conversion device, including:
  • Memory for storing executable instructions
  • a processor configured to execute executable instructions stored in the memory to implement the media file conversion method provided by the embodiment of the present disclosure.
  • an embodiment of the present disclosure further provides a storage medium that stores executable instructions. When the executable instructions are executed, the method is used to implement the media file conversion method provided by the embodiments of the present disclosure.
  • an embodiment of the present disclosure further provides a storage medium that stores executable instructions. When the executable instructions are executed, the method is used to implement the media file conversion method provided by the embodiments of the present disclosure.
  • FIG. 1 is a schematic diagram of an optional structure of a container for multimedia file packaging according to an embodiment of the present disclosure
  • FIG. 2 is a schematic diagram of an optional packaging structure of an MP4 file in an embodiment of the present disclosure
  • FIG. 3 is an optional structure diagram of a media data container storing media data in a media file according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of an optional packaging structure of an FMP4 file in an embodiment of the present disclosure
  • FIG. 5 is an optional schematic structural diagram of a media file conversion device according to an embodiment of the present disclosure.
  • FIG. 6 is an optional structural diagram of a media file conversion device provided by an embodiment of the present disclosure.
  • FIG. 7 is an optional flowchart of a method for converting a media file according to an embodiment of the present disclosure
  • FIG. 8 is a schematic flowchart of an optional process for obtaining media information from a metadata container according to an embodiment of the present disclosure
  • FIG. 9 is an optional flowchart for searching media data of a given period from a media file according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic flowchart of an optional process for finding a position of media data of a given time period from a media file according to an embodiment of the present disclosure
  • FIG. 11 is an optional flowchart of an encapsulated segmented media file according to an embodiment of the present disclosure
  • FIG. 12 is a schematic diagram of an optional usage scenario of a media file conversion method provided by an embodiment of the present disclosure.
  • FIG. 13 is a schematic diagram of converting an MP4 file to an FMP4 file and playing it through a media source extension interface according to an embodiment of the present disclosure.
  • Media file a file that stores encoded media data (such as at least one of audio data and video data) in the form of a container (Box), which also includes media information to ensure media The metadata is correctly decoded.
  • MP4 files For example, a media file formed by using the Moving Picture Experts Group (MPEG, Moving Picture Experts Group) -4 encapsulation format to encapsulate media data is called an MP4 file.
  • MP4 files store video data encoded with Advanced Video Coding (AVC, Advanced Video Coding, H.264) or MPEG-4 (Part 2) specifications, and audio encoded with Advanced Audio Coding (AAC, Advanced Audio Coding) specifications. Data, of course, does not exclude other encodings of video and audio.
  • AVC Advanced Video Coding
  • H.264 Advanced Video Coding
  • AAC Advanced Audio Coding
  • a container also called a box
  • a container is an object-oriented component defined by a unique type identifier and a length. See FIG. 1 for an optional structural diagram of a container provided by an embodiment of the present disclosure, including a container header. Department (Box Header) and Container Data (Box Data), which are filled with binary data used to express various information.
  • the container header includes a size (size) and a type (type).
  • the capacity indicates the length occupied by the container in the media file, and the type indicates the type of the container. See FIG. 2, which is an example of an MP4 file provided by the embodiment of the present disclosure. Schematic diagram of the selected package structure.
  • the basic container types involved in MP4 files include file type containers (ftyp boxes), metadata containers (moov boxes), and media data containers (mdat boxes).
  • the container data section can store specific data. At this time, the container is called a "data container”, and the container data section can further encapsulate other types of containers. At this time, the container is called a "container container”.
  • Track also called stream, is a time-related related sample in a media data container.
  • a track represents a video frame sequence or an audio frame sequence. It consists of a subtitle track synchronized with a sequence of video frames. A set of consecutive samples in the same track is called a chunk.
  • a file type container which is used to store the capacity (ie, the length of the occupied bytes) and type of container in the media file. As shown in Figure 2, the file type container is marked as "ftyp box", where the binary data is stored. The file type and compatibility are described according to the specified byte length.
  • Metadata container a container used to store metadata (that is, data describing multimedia data stored in the media data container) in the media file.
  • metadata that is, data describing multimedia data stored in the media data container
  • Information expressed by binary data stored in the metadata container in the MP4 file is called media information.
  • the header of the metadata container uses binary data to indicate that the type of the container is "moov box".
  • the container data part encapsulates the mvhd container used to store the overall information of the MP4 file. It is independent of the MP4 file and is related to MP4 file playback is related, including duration, creation time and modification time.
  • the media data container of the media file may include sub-containers corresponding to multiple tracks, such as an audio track container (video track box) and a video track container (video track box), which are included in the sub-containers of the audio track container and the video track container.
  • the necessary sub-containers include: a container (denoted as a tkhd box) for describing the characteristics and general information of the track (such as duration, width and height), and recording the track's media information (such as media Type and sampled information) container (denoted as mdia box).
  • the sub-containers encapsulated in the mdia box may include: a container that records related attributes and content of the track (denoted as mdhd box), a container that records information about the playback process of the media (denoted as hdlr box), and describes media data in the track Container of media information (recorded as minf box); minf box also encapsulates a sub-container (recorded as dinf box) for explaining how to locate media information, and used to record all time information (decode time / Display time), location information and codec sub-containers (recorded as stbl box).
  • FIG. 3 it is a schematic diagram of a structure for storing media data in a media data container in a media file provided by an embodiment of the present disclosure.
  • the sampling time, type, capacity, and The position in the media data container is described below for each child container in the stbl box.
  • each media file can have one or more description tables.
  • the description table can find the description information of each sample.
  • the description information can ensure the correct decoding of the samples.
  • Different media types store different description information. For example, for video media, the description information is the structure of the image.
  • the stts box stores the sampling time information, and provides a table to map the time (decoding time) and the sequence number of the sample. Through the stx box, you can locate the sample at any time in the media file; the stts box also uses other tables to map the sample Capacity and pointer. Each entry in the table provides the serial number of consecutive samples in the same time offset, and the offset of the sample. Increasing these offsets can build a complete time-sample mapping table. Calculated as follows:
  • the sum of all offsets is the duration of the media data in the track.
  • stssbox records the sequence number of the key frames in the media file.
  • the stsbox records the mapping relationship between the sample and the block where the sample is stored.
  • the relationship between the number of the sample and the number of the block is mapped through the table.
  • the block containing the specified sample can be found by looking up the table.
  • the stco box defines the position of each block in the track, and the position is expressed by an offset from the start byte of the media data container and a length (ie, capacity) from the start byte.
  • stszbox records the capacity (ie size) of each sample in the media file.
  • Media data container a container for storing multimedia data in a media file, for example, a media data container in an MP4 file, as shown in FIG. 3, sampling is a unit stored in the media data container and stored in a block of the media file The length of the blocks and samples can be different.
  • Segmented media files, sub-files formed by segmenting media files, and each segmented media file can be decoded independently.
  • media data in MP4 files is divided according to key frames.
  • the divided media data and corresponding metadata are encapsulated to form segmented MP4 (FMP4, Fragmented MP4) files.
  • the metadata in each FMP4 file can guarantee The media data is decoded correctly.
  • each FMP4 file shown in FIG. 2 is converted into multiple FMP4 files, referring to FIG. 4, it is a schematic diagram of an optional package structure of the FMP4 file provided by the embodiment of the present disclosure.
  • One MP4 file can be converted into multiple FMP4 files, each FMP4 file includes three basic containers: moov container, moof container, and mdat container.
  • the moov container includes MP4 file-level metadata, which is used to describe all media data in the MP4 file from which the FMP4 file is derived, such as the length, creation time, and modification time of the MP4 file.
  • the moof container stores segment-level metadata, which is used to describe the media data encapsulated in the FMP4 file in which it is located, ensuring that the media data in FMP4 can be decoded.
  • One moof container and one mdat container constitute one segment of a segmented MP4 file.
  • One segmented MP4 file can include one or more such segments.
  • the metadata encapsulated in each segment guarantees the segmentation.
  • Media data encapsulated in a segment can be independently decoded.
  • MSE Media Resource Extensions
  • MSE Media Source Extensions
  • the player is provided with a function of playing a media stream by invoking a Hypertext Markup Language (HTML) Media Element, for example, using a video element ⁇ video> and an audio element ⁇ audio> to implement a video / audio playback function.
  • HTML Hypertext Markup Language
  • Streaming media format which encapsulates media data into streaming media files.
  • Media files can be decoded and played without having to be completely downloaded or transcoded, that is, natively supporting the packaging technology while downloading and playing at the same time.
  • Files in a typical streaming media format include: TS media file fragments based on HTTP Live Streaming (HLS, HTTP Live Streaming) technology, FLV (Flash Video) files, and the like.
  • Non-streaming media format a packaging technology that encapsulates media data into media files and can be decoded and played after the media file is completely downloaded.
  • Typical non-streaming format files include: MP4 files, Windows Media Video (WMV, Windows Media (Video) file, Advanced Streaming Format (ASF) file, etc.
  • MP4 files do not natively support streaming media playback, but invalid binary data is filled with missing binary data (for example, in the ftyp container) through the media stream after transcoding to the player or the missing part of the downloaded MP4 file.
  • missing binary data for example, in the ftyp container
  • filling the missing part of the mdat container with invalid binary data can also be used to achieve the technical effect of downloading and playing. Called non-streaming format.
  • the media file conversion device may be provided in a manner of hardware, software, or a combination of software and hardware.
  • FIG. 5 is an optional composition diagram of the media file conversion device according to the embodiment of the present disclosure.
  • the media file conversion device 500 may include a video playback function. Mobile phones, computers, digital broadcast terminals, messaging equipment, game consoles, tablet devices, medical equipment, fitness equipment, personal digital assistants, etc.
  • the media file conversion apparatus 500 shown in FIG. 5 includes at least one processor 501, a memory 502, at least one network interface 504, and a user interface 503.
  • the various components in the media file conversion apparatus 500 are coupled together via a bus system 505. It can be understood that the bus system 505 is configured to implement connection and communication between these components.
  • the bus system 505 includes a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are marked as the bus system 505 in FIG. 6.
  • the user interface 503 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch panel, or a touch screen.
  • the memory 502 may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories.
  • the non-volatile memory may be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), or an erasable programmable read-only memory (EPROM, Erasable Programmable Read- Only Memory), electrically erasable and programmable read-only memory (EEPROM, Electrically Programmable Read-Only Memory); magnetic surface memory can be magnetic disk storage or magnetic tape storage.
  • the volatile memory may be a random access memory (RAM, Random Access Memory), which is used as an external cache.
  • RAM Static Random Access Memory
  • SSRAM Synchronous Static Random Access Memory
  • DRAM Dynamic Random Access Memory
  • SDRAM Synchronous Dynamic Random Access Memory
  • the memory 502 in the embodiment of the present disclosure includes, but is not limited to, a tri-state content addressable memory and a static random access memory capable of storing media file data obtained from a server to support the operation of the media file conversion apparatus 500.
  • Examples of these data include: any computer program for operating on the media file conversion apparatus 500, such as an operating system 5021 and an application program 5022; various types of media file data information, and the like.
  • the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.
  • the application 5022 may include various applications, such as a client with a media file conversion function, or an application.
  • a program that implements a method for converting a media file according to an embodiment of the present disclosure may be included in an application program 5022.
  • the method disclosed in the embodiment of the present disclosure is implemented by the processor 501.
  • the processor 501 may be an integrated circuit chip and has a signal processing capability.
  • the media file conversion method provided by the embodiment of the present disclosure may be completed by an operation of a hardware integrated logic circuit or software in the processor 501.
  • the above-mentioned processor 501 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.
  • DSP Digital Signal Processor
  • the processor 501 may implement or execute the methods, steps, and logic block diagrams provided in the embodiments of the present disclosure.
  • a general-purpose processor may be a microprocessor or any conventional processor.
  • the steps of the method provided by the embodiments of the present disclosure may be directly implemented by a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in a storage medium.
  • the storage medium is located in the memory 502.
  • the processor 501 reads the information in the memory 502 and completes the media file conversion method in combination with its hardware.
  • an embodiment of the present disclosure further provides a computer-readable storage medium, such as a memory 502 including a computer program, and the computer program may be executed by the processor 501 of the media file conversion apparatus 500 to complete a media file. Conversion method.
  • the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; it may also be a variety of devices including one or any combination of the above memories, such as Mobile phones, computers, tablet devices, personal digital assistants, etc.
  • the pure hardware implementation of the media file conversion device is described below.
  • the media file conversion device according to the embodiment of the present disclosure can be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, and Programmable Logic Devices (PLDs). Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), Field Programmable Gate Array (FPGA, Field-Programmable Gate Array), or other electronic components to implement the media file conversion provided by the embodiments of the present disclosure method.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processing
  • PLDs Programmable Logic Device
  • CPLD complex programmable logic device
  • FPGA Field Programmable Gate Array
  • Field-Programmable Gate Array Field-Programmable Gate Array
  • the pure software implementation of the media file conversion device is described below.
  • the media file conversion device according to the embodiment of the present disclosure may be implemented by an application program or a plug-in, or a combination of the two.
  • the application program may be a client dedicated to media file conversion, or a client that uses the media file conversion function as an optional function by installing a corresponding plug-in.
  • the plug-in can be implemented as an application's function upgrade installation package, and the function of media file conversion is superimposed in the application for media playback; it can also be an element in a web page for media playback, implemented in a front-end language, and viewed by The device directly interprets and executes the functions of media file conversion and media playback in the web page.
  • the media file conversion device may be provided as a series of modules that have a coupling relationship at the signal / information / data level, which will be described below with reference to FIG. 6, see FIG. 6, FIG. 6
  • the modules are further split or merged according to the different functions implemented. The functions implemented by the modules shown in FIG. 6 are described below.
  • the parsing module 601 is configured to parse the metadata encapsulated in the metadata container of the media file to obtain media information for describing the media data encapsulated in the media data container of the media file, and the media file is in a non-streaming media format.
  • the parsing module 601 is configured to parse the nested structure of the sub-containers in the metadata container, and read out the binary data in each sub-container according to the nested structure; the parsing module 601 is configured to The read binary data parses out the media information of the media data represented by each sub-container.
  • the container can be a nested structure
  • the nested structure of the child container in the metadata container is first parsed, and the corresponding binary data is parsed from the mdat container.
  • the analysis of the metadata container of the nested structure and the acquisition of related data can be realized.
  • the searching module 602 searches for media data of a given period according to the time of the media data indicated by the media information.
  • the search module 602 is configured to search for two key frames in a media file according to the time of the media data, and the two key frames correspond to the start time and the end time of a given time period; the search module 602, It is configured to find the audio frame aligned with the video frame between the two key frames according to the time of the media data; the finding module 602 is configured to find the video frame between the two key frames in the media file according to the position of the media data Position and find the position of the audio frame in the media file.
  • the video frames in the nested structure can be determined, and the positions of the corresponding audio frames in the audio track container can be realized.
  • the search module 602 is configured to search for the position of the media data in a media file at a given time period according to the position of the media data indicated in the media information in the media file.
  • the search module 602 is further configured to search for a first key frame whose decoding time is aligned with the start time of a given time period according to the time of the media data; the search module 602 is further configured to search for an earlier decoding time.
  • the first key frame that is closest to the start time among the key frames at the start time of the given period; the search module 602 is further configured to find the second key aligned with the end time of the given period according to the time of the media data.
  • a frame; the searching module 602 is further configured to find a second key frame closest to the end time among key frames whose decoding time is later than the end time of the given time period.
  • the search module 602 is further configured to search for audio frames whose decoding time is in a given period according to the time of the media data;
  • the decoding time of the last audio frame is aligned with the end time of the given period, or is later than the end time of the given period and is closest to the end time.
  • the extraction module 603 is configured to extract media data at a corresponding location from a media data container of a media file.
  • the extraction module 603 is configured to be based on an offset and a capacity corresponding to a position of a video frame between two key frames, and an offset corresponding to a position of an audio frame aligned with the video frame. And a capacity, to determine an interval composed of a minimum offset and a maximum capacity; an extraction module 603 is configured to extract media data from a corresponding interval of a media data container of a media file.
  • the offset characterizes the storage start position of the media data in the media data container of the media file
  • the capacity characterizes the length occupied by the media data from the storage start position
  • the extraction module 603 is configured to extract a video frame according to the storage position of the video frame, and after the video frame extraction is completed, extract the audio frame according to the storage position of the audio frame.
  • the extraction module 603 is configured to extract video frames and audio frames from the media data according to a preset break value. With the technical solution shown in this embodiment, it is possible to obtain corresponding audio frames and video frames through one extraction, so as to improve the processing speed of media files.
  • the synthesizing module 604 is configured to encapsulate the extracted media data and metadata describing the media data according to the packaging structure of the segmented media file to obtain a segmented media file that can be used for independent decoding.
  • the composition module 604 is configured to perform filling operations to fill data representing the type and compatibility of the media file into the file type container of the segment media file; the composition module 604 is configured to display the The file-level metadata of the media file is filled into the metadata container of the segmented media file; the synthesis module 604 is configured to fill the extracted media data and the metadata describing the media data into the segmented media file correspondingly. Media data container in the segment container, and metadata container at the segment level. With reference to the structure shown in FIG. 4, the synthesis module 604 is configured to fill the moov container with data indicating the type and compatibility of the media file, fill the moof container with the file-level metadata representing the media file, and extract the extracted data. Media data, and metadata describing the media data is populated into the mdat container.
  • composition module 604 is further configured to call a class write operation function in a device corresponding to the sub-container of the segmented media file to complete the merging of the sub-container and the sub-container with a nested relationship.
  • composition module 604 is further configured to call the write operation function of the class to complete the writing and merging of binary data in the memory buffer of the subcontainer, and return an instance of the class. The returned instance is used For merging child containers with child containers with nested relationships.
  • the synthesis module 604 is further configured to provide a memory buffer through a class Stream to store the written binary data; and to convert the multi-byte decimal data to be filled through a static method provided by the Stream To binary data; through the write operation function provided by each type of Stream instance, the binary data to be filled into the child container is merged in the memory buffer; the method returns a new Stream instance, and through the technical solution shown in this embodiment, Merging of the current sub-container with other sub-containers with a nested relationship can be implemented to flexibly implement adjustment of the sub-container.
  • the metadata of the requested media file is further included.
  • the metadata of the original resolution media file or the target resolution media file is requested, first, Check whether the requested cache has been requested. If so, continue to request media data, and do not repeatedly request metadata.
  • the requested media data can be used to save data acquisition time and improve operations. speed.
  • the synthesis module 604 is further configured to obtain binary data corresponding to the metadata container from the binary data of the media file according to the position of the metadata container in the media file; and sequentially parse the obtained binary data
  • the binary data corresponding to the length of the container data in the parsed data is used to obtain the media information represented by the container data.
  • a process for the player to acquire media data in a given time period is described.
  • the player When a video file or a track media file is played by a player embedded in a webpage, the player must be able to correctly parse the data stream, obtain corresponding media data for a certain time, and ensure that this piece of media data can Decoded independently.
  • the method for implementing the media file conversion according to the embodiment of the present disclosure will be described in conjunction with the media file conversion device provided by the embodiment of the present disclosure.
  • the following description uses the media file as an MP4 file as an example.
  • the solution can be easily applied to other non-streaming media formats, such as MKV, WMV, and ASF, and other non-streaming media files.
  • FIG. 7 is an optional flowchart of a media file conversion method provided by an embodiment of the present disclosure, and will be described in conjunction with the steps shown in FIG. 7.
  • Step 701 Parse metadata encapsulated in a metadata container of a media file to obtain media information of media data encapsulated in a media data container for describing the media file.
  • the media file is an MP4 file.
  • the nested structure of the sub-containers in the metadata container of the media file is parsed, and the binary data in each sub-container is read according to the nested structure.
  • the binary data parses out the media information of the media data represented by each sub-container.
  • the moov container of the MP4 file is a nested structure.
  • the nested structure of the sub-containers in the metadata container is analyzed to determine the sub-containers nested in the moov container, such as mvhd containers, audio track containers, Video track containers, etc. If the sub-containers are still nested, continue parsing, until the sub-containers are no longer nested, read the binary data encapsulated in the corresponding sub-containers, and parse to obtain the binary data.
  • Media messages such as the key frame sequence number in the media file recorded by stss box, the capacity (ie, size) of each sample in the media file recorded by stsz box, and so on.
  • a method for setting a parser according to a container type and parsing a child container in a metadata container according to the container type to obtain media information will be described with reference to FIG. 8.
  • FIG. 8 is a schematic diagram of an optional process for obtaining media information from a metadata container according to an embodiment of the present disclosure, and will be described in conjunction with the steps shown in FIG. 8.
  • Step 801 Locate the position of the metadata container in the media file.
  • the binary data conforming to the container header specification is read from the binary data of the media file, according to the type and length of the container identified in the read binary data, until the bias of the metadata container in the media file is located. Volume and capacity.
  • the binary data starting from zero bytes corresponds to the file type container.
  • the canonical length of the container header reads the starting position of the binary data of the media file and reads the container header.
  • the length of the binary data of the standard length Parsing the read binary data can determine the type and length of the container in the media file that is located after the file type container.
  • the parsed type is a file type container
  • the length (that is, capacity) of the metadata container can be parsed.
  • the offset of the metadata container is the length of the file type container.
  • the parsed type is a media data container
  • the length of the media data container and the length of the subtype container as an offset continue to read binary data that conforms to the canonical length of the container header, so that the metadata container can be parsed.
  • Length that is, capacity
  • the length of the metadata container is the sum of the length of the file type container and the length of the media data container.
  • the packaging order of subsequent containers is not standardized.
  • the packaging order of the containers in the media file is a file type container, metadata container, media data container, or file type Containers, media data containers, and metadata containers can accurately and efficiently locate the location of file type containers in media files.
  • Step 802 Obtain binary data corresponding to the metadata container from the binary data of the media file according to the position of the metadata container in the media file.
  • the position of the metadata container in the media file is expressed by the offset and capacity.
  • the binary data is read from the media file at the position corresponding to the offset, until the length of the read binary data matches the capacity of the metadata container. To read out the binary data corresponding to the metadata container.
  • step 803 the binary data corresponding to the canonical length of the header of the metadata container binary data is sequentially parsed to obtain the container type of the child container in the metadata container and the length of the container data of the child container.
  • the offset of each read of the binary data is the sum of the lengths of the identified sub-containers, and the length of the read binary data conforms to The canonical length of the container header, so that the type and length of the child container currently being processed can be parsed.
  • the binary data when reading for the first time, the binary data is read from the zero byte of the binary data of the metadata container, and the length of the read binary data conforms to the canonical length of the container header, so that the first child container can be parsed
  • the type and length of the read when reading for the second time, the length of the first read sub-container is used as an offset to start reading binary data, and the length of the read binary data conforms to the canonical length of the container header, so You can parse out the type and length of the second child container.
  • Reading the binary data in the above manner does not cause the situation that multiple reads cause rollback, nor does it occur that there are fewer reads and secondary reads, and the efficiency and accuracy of parsing are guaranteed.
  • step 804 a parser corresponding to the container type of the child container is called, and the binary data corresponding to the length of the container data in the unparsed data is sequentially parsed to obtain the media information represented by the container data.
  • a typical container type nested in the metadata container is pre-labeled to indicate whether the container is directly used to encapsulate binary data or further encapsulates the container. For example, for mvhd box, audio track shown in FIG. 2 Boxes and video boxes are further encapsulated with containers. For the stts box, stsd box, etc. shown in FIG. 2, they are directly encapsulated with binary data.
  • a parser corresponding to the container type is set, and the parser is used to parse out the represented media information according to the binary data; in step 804, the sub-container parsed in step 803
  • the parser is used to parse out the represented media information according to the binary data; in step 804, the sub-container parsed in step 803
  • Case 1 When the container type of the sub-container is determined by comparison and is pre-marked to directly encapsulate binary data, a parser corresponding to the container type of the sub-container is called, and the sub-container is parsed by the parser. Container data to get the media information represented by the container data.
  • Case 2 When the container type of the sub-container is pre-marked by comparison and is pre-marked to continue to encapsulate the container, according to the canonical length of the container header in the media file, recursively parse the binary data corresponding to the sub-container, Until the container type resolved to the container enclosed in the child container is pre-marked and pre-marked for direct packaging of binary data, a parser corresponding to the container type of the container enclosed in the child container is invoked to parse the binary data byte by byte , The length of the parsed binary data corresponds to the length of the container data of the container encapsulated in the child container to obtain the media information represented by the container data of the container encapsulated in the child container.
  • a method for recording media information in the process of parsing the metadata container is described.
  • the binary data corresponding to the canonical length of the container header in the binary data of the metadata container is sequentially parsed to obtain the subcontainers in the metadata container.
  • an object is established based on the nested relationship between the child container and the container to which it belongs, and the nested relationship between the child container and the encapsulated container.
  • the container type of the child container is pre-marked to directly encapsulate binary data
  • An array including media information is stored in the object created by the corresponding sub-container, and the stored media information is represented by the container data of the sub-container.
  • a description is given of a manner of recording a nested relationship between sub-containers in the process of parsing a metadata container.
  • the binary data corresponding to the canonical length of the container header in the metadata container binary data is sequentially parsed to obtain metadata
  • the parsed sub-container is recorded in the called parser; the recorded sub-container instance is set to the sub-container property
  • the child container attribute is included in the container to which the child container belongs, and is used to describe the nesting relationship between the child container and the container to which the child container belongs.
  • the container type of the sub-container is not pre-marked, or is pre-marked to directly encapsulate the binary data, but the corresponding type of parser is not called, the binary data corresponding to the parsed sub-container is ignored.
  • the length of the subcontainer jumps to the part of the binary data corresponding to the next subcontainer and continues to parse.
  • Step 702 Search for media data of a given period according to the time of the media data indicated by the media information.
  • FIG. 9 is an optional flowchart for finding media data of a given period from a media file according to an embodiment of the present disclosure, and will be performed in conjunction with the steps shown in FIG. 9. Instructions.
  • Step 901 Find two key frames in a media file according to the time of the media data, and the two key frames correspond to a start time and an end time of a given time period.
  • the length of a media file is 120 seconds, which contains several key frames.
  • the given time period is 20 to 40 seconds, according to the mapping relationship between the decoding time of the frame and the sequence number of the frame, first find whether there is a decoding time that is just right The keyframe at the 20th second, if it exists, the decoding time is exactly the keyframe at the 20th second as the first keyframe; if it does not exist, the decoding time is earlier than the 20th second, and it is closest to the start time at the 20th second A key frame is used as the first key frame.
  • Second find out if there is a key frame with a decoding time that is exactly 40th second. If it exists, the key frame with a decoding time that is exactly 40th second is used as the second key frame; if it does not exist, look for a decoding time that is later than 40th, And the closest key frame from the 40th second is used as the second key frame.
  • Step 902 Find the audio frame aligned with the video frame between the two key frames according to the time of the media data.
  • the player can find the audio frame aligned with the given time period from the time of the media data by: finding the audio frame with the decoding time distributed from the time of the media data according to the given time period, and using the video frame as The benchmark is to locate audio frames that are synchronized in time at the video frame; where, when the time of the playback point corresponds to the presence of an audio frame, the decoding time of the first audio frame is aligned with the start time of a given period; when the time of the playback point does not have a corresponding audio frame
  • the decoding time of the first audio frame is earlier than the start time of the given time period and is closest to the start time, to ensure that the decoding time of the first audio frame is not later than the decoding start time of the first video frame (the first key frame mentioned above); when When the end time of a fixed period corresponds to the presence of audio frames, the decoding time of the last audio frame is aligned with the end time of the given time period; when there is no corresponding audio frame at the end time of the
  • the segmented media file including video frames and corresponding audio frames overcomes the problem of inconsistent video and audio time lengths, and ensures that there is synchronized audio playback during each frame of video playback. Silence will occur.
  • Step 903 Find the position of the video frame between the two key frames in the media file according to the position of the media data, and find the position of the audio frame in the media file.
  • the position of the media data in the media file may be expressed as two intervals: the first interval, that is, the position of the video frame between the two key frames in the media file, is included in the media The offset and capacity in the media data container of the file; the second interval, the position of the audio frame between the two audio frames in the media file, including the offset and capacity in the media data container of the media file .
  • the offset indicates the storage start position of the media data in the media data container of the media file
  • the capacity indicates the length occupied by the media data from the storage start position.
  • the position of the media data in the media file is represented as an interval, according to the offset and capacity corresponding to the position of the video frame between the two key frames, and the position of the audio frame aligned with the video frame.
  • the corresponding offset and capacity determine the interval consisting of the minimum offset and the maximum capacity.
  • the offset of the position of the video frame between the first key frame and the second key frame at the target resolution media file, the corresponding interval is [a, b] (the address is in ascending order), and the audio frame is at the target resolution media.
  • the offset of the position of the file, the corresponding interval is [c, d] (the address is in ascending order), then the interval formed by taking the upper and lower limits of the position is [min (a, c), max (b, d)] .
  • media data of a given time can be extracted from the media file at one time, reducing the number of reading times of the player, and improving processing efficiency.
  • the player sends a network request carrying the offset and capacity of the target interval to the server to request the media data of the target interval.
  • the server extracts the media data in the media file based on the offset and capacity of the target interval and returns it once.
  • the media data in the target interval does not need to be obtained twice, which reduces the number of requests from the player and improves the processing efficiency.
  • the video frame in the nested structure can be determined, and the position of the corresponding audio frame in the audio track container can be realized.
  • Step 703 Find the position of the media data of the given time period in the media file according to the position of the media data in the media file indicated by the media information.
  • FIG. 10 uses the package structure shown in FIG. 4 as an example for a media file to describe the position of the media data in a media file for a given period of time from the media file.
  • FIG. 10 An optional flow chart for finding the location of media data in a given time period in a media file will be described with reference to the steps shown in FIG. 10.
  • Step 1001 Determine a period corresponding to the media data to be acquired.
  • the time period is a period of time after the playback point.
  • the time corresponding to the playback point is a time measurement relative to the media time coordinate system (with the media file playback start time as the time origin).
  • the predetermined proportion is 5%, or a set length such as 10 minutes.
  • Step 1002 Check the stts box to determine the sequence number of samples (including video frames and audio frames) whose decoding time is within a given period.
  • Step 1003 Query the STSC box according to the sampled sequence number to determine the sequence number of the block including the sample.
  • Step 1004 Find the offset of the block from the stco box according to the sequence number of the block.
  • step 1005 the stsz box is searched according to the sequence number of the sample, and the offset and the capacity of the sample within the block are found.
  • the serial number and capacity of the sample are recorded in the stsz box, and the offset of the block and the offset of the sample within the block are accumulated to obtain the offset of the sample in the mdat container (box).
  • step 1006 the offset corresponding to the block with the smallest offset and the offset sampled in the corresponding block are accumulated to be used as the offset of the media data in a given period.
  • Step 1007 Accumulate the offset corresponding to the block with the largest offset, the offset of the corresponding sample within the block, and the capacity of the corresponding sample as the length that can cover the media data and the offset from the media data. Do the difference to get the capacity of the media data.
  • steps 1006 and 1007 calculate the position (including offset and capacity) of the audio frame in a given period and the position (including offset and capacity) of the video frame in a given period, respectively, to obtain video. Intervals when frames and audio frames are stored in mdat boxes.
  • Step 704 Extract the media data at the corresponding location from the media data container of the media file.
  • step 703 when the position of the media file data in the media data container is represented by the two intervals described in step 703, that is, when the interval corresponding to the video frame and the audio frame is used to represent, respectively, according to the position of the video frame
  • media data is extracted from the corresponding interval of the media data container of the media file at a time, which effectively improves The processing efficiency of media data, especially when network transmission is required, can significantly save the occupation of the connection and the consumption of traffic.
  • Step 705 Encapsulate the extracted media data and metadata describing the media data according to the packaging structure of the segmented media file to obtain a segmented media file that can be used for independent decoding.
  • FIG. 11 is a schematic diagram of an optional process of packaging segmented media files according to an embodiment of the present disclosure, and will be described in conjunction with the steps shown in FIG.
  • Step 1101 Fill data indicating the type and compatibility of the segmented media file into the file type container of the segmented media file.
  • the file type container of the FMP4 file is the head of the ftyp box and the type and length of the container (representing the overall length of the ftyp box).
  • the data part of the box is filled to generate data (binary data) indicating that the file type is FMP4 and compatible protocols.
  • step 1102 the metadata representing the file level of the segmented media file is filled into the metadata container of the segmented media file.
  • the metadata describing the media data required to fill the nested structure is calculated .
  • the file-level metadata representing the FMP4 file is calculated and filled into the FMP4 metadata container (that is, the moov box).
  • the metadata encapsulated in the mvhd container is used to represent media information related to the playback of segmented media files, including location, duration, creation time, and modification time, etc .; sub-containers in the track container represent corresponding tracks in the media data References and descriptions, such as a container (recorded as tkhd box) that describes the characteristics and general information (such as duration, width and height) of the track, and records the track's media information (such as media type and sampling information).
  • Container (denoted as mdia box).
  • step 1103 the extracted media data and metadata describing the media data are correspondingly filled into a media data container in a segment container of a segmented media file and a metadata container at a segment level.
  • one or more fragments may be encapsulated in the segmented media file.
  • the media data to be filled one or more media data containers of the segmented media file ( In the mdat box), each segment is encapsulated with a segment-level metadata container (denoted as a moof box).
  • the filled metadata is used to describe the media data filled in the segment, so that the segment can be decoded independently. .
  • the metadata in the container (that is, the moof box) is correspondingly filled into the sub-containers of the moof box, where the head of the moof box is called moof box, and the filled binary data is used to indicate that the type of the container is "moof box "and the length of the moof box.
  • the write operation function of the calling class is used to complete the writing and merging of binary data in the memory buffer of the child container, and return the class The returned instance is used to merge the child container with the child container with the nested relationship.
  • each class Stream provides a memory buffer to store the binary data to be filled.
  • the buffer completes the merging and filling of the binary data to be filled into the child container; the static method provided by Stream returns a new Stream instance, which can realize the current child container's merging with other child containers with a nested relationship.
  • FIG. 12 is a schematic diagram of an optional use scenario of a media file conversion method provided by an embodiment of the present disclosure.
  • the media file conversion device provided by the embodiment of the present disclosure may be implemented as a user terminal 10.
  • the user terminal 10 (exemplarily showing the user terminal 10-1 and the user terminal 10-2) connects to the server 30 through a network 20, and the network 20 may be a wide area network or a local area network, or a combination of the two, using a wireless link Achieve data transmission.
  • the user terminal 10 plays a media file through a webpage embedded with a player, and displays the played content through a graphical interface 110 (a graphical interface 110-1 and a graphical interface 110-2 are exemplarily shown).
  • the user terminal 10 parses the metadata obtained from the server 30 encapsulated in the metadata container of the media file to obtain media information for describing the media data encapsulated in the media data container of the media file, and the media file uses a non-streaming Media format; searching for media data of a given period according to the time of the media data represented by the media information; finding the given time according to the position of the media data represented in the media information in the media file The position of the media data of the segment in the media file; extracting the media data corresponding to the position from the media data container of the media file; dividing the extracted media data and metadata describing the media data according to the
  • the encapsulation structure of segment media files is encapsulated to obtain segmented media files that can be used for independent decoding.
  • FIG. 13 is a schematic diagram of converting an MP4 file to an FMP4 file and playing it through a media source extension interface according to an embodiment of the present disclosure.
  • the player is based on the real address of the media file (http://www.toutiao.com/ a / b.mp4)
  • Obtain media data that meets a fixed duration and then construct a segmented media file based on the obtained media data, that is, convert it to a segmented MP4 format media file, and then add the segmented media file to the media source of the MSE Object (for example, implemented in the form of a file-like object (Blob))
  • the MSE creates a virtual URL corresponding to the media source object, and passes the virtual URL to the Video element, so that the Video element obtains the corresponding media source object, and implements the above extraction.
  • Media data and metadata describing the media data are packaged according to the packaging structure of the segmented media file to obtain segmented media files that can be used for independent decoding.
  • media file conversion method by parsing metadata encapsulated in a metadata container of a media file, media information for describing the media data encapsulated in the media data container of the media file is obtained, and according to the media, Time of the media data indicated by the information, find the media data of the given time period; and according to the position of the media data indicated by the media information in the media file, find the media data of the given time period in the media file.
  • the position in the media file extracting the media data corresponding to the position from the media data container of the media file; performing the extraction of the media data and the metadata describing the media data according to the packaging structure of the segmented media file Encapsulate to get segmented media files that can be used for independent decoding.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Television Signal Processing For Recording (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本公开提供了一种媒体文件转换方法、装置及存储介质，方法包括:解析媒体文件的元数据容器中封装的元数据，得到用于描述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，媒体文件采用非流媒体格式；根据媒体信息表示的媒体数据的时间，查找给定时段的媒体数据；根据媒体信息表示的媒体数据在媒体文件中的位置，查找给定时段的媒体数据在媒体文件中的位置；从媒体文件的媒体数据容器中提取对应位置的媒体数据；将所提取的媒体数据、以及描述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。

Description

媒体文件转换方法、装置及存储介质
相关申请的交叉引用
本申请基于申请号为201810530467.5、申请日为2018年05月29日的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本申请作为参考。
技术领域
本公开涉及多媒体技术，尤其涉及一种媒体文件转换方法、装置及存储介质。
背景技术
动态图像专家组(MPEG，Moving Picture Experts Group)-4封装格式(本文中简称为MP4格式)是目前广泛使用的容器格式，可以存储多种编码方式的视频数据和音频数据。
然而，MP4封装格式是一种非流媒体格式，采用MP4封装格式的文件(本文中简称MP4文件)不支持以流媒体的(即一边下载一边播放)形式播放，为此，相关技术不得不将MP4文件转换流媒体格式，诸如FLV(Flash Video)、超文本传输协议直播流(HLS，HTTP Live Streaming)等，从而实现流媒体形式的播放。
然而，相关技术缺乏将MP4文件转换为流媒体格式的分段媒体文件以适应网络传输的有效方式。
发明内容
有鉴于此，本公开实施例提供一种媒体文件转换方法、装置及存储介质，能够将非流媒体格式的媒体文件转换为适应网络传输的、能够独立解码的、更小体积的分段媒体文件。
本公开实施例的技术方案是这样实现的：
本公开实施例提供了一种媒体文件转换方法，包括：解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，所述媒体文件采用非流媒体格式；根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
本公开实施例还提供了一种媒体文件转换装置，包括：解析模块，配置为解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，所述媒体文件采用非流媒体格式；查找模块，根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；所述查找模块，配置为根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；提取模块，配置为从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；合成模块，配置为将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
又一方面，本公开实施例还提供了一种媒体文件转换装置，包括：
存储器，用于存储可执行指令；
处理器，用于执行所述存储器中存储的可执行指令实现本公开实施例提供的媒体文件转换方法。
又一方面，本公开实施例还提供了一种存储介质，存储有可执行指令，所述可执行指令被执行时，用于实现本公开实施例提供的媒体文件转换方法。
又一方面，本公开实施例还提供了一种存储介质，存储有可执行指令，所述可执行指令被执行时，用于实现本公开实施例提供的媒体文件转换方法。
本公开实施例具有如下技术效果：
能够从媒体文件中提取任意给定时段的媒体数据以构造分段媒体文件，从而，当需要播放媒体文件的给定时段时，只需要从非流媒体格式的媒体文件提取给定时间的媒体数据，封装为可独立解码的分段媒体文件即可，通过这种方式，一方面，克服了非流媒体格式文件需要完整下载后才可以独立播放的限制，播放实时性好；另一方面，由于仅需要针对给定时段构造分段媒体文件，而不是将完整的媒体文件预先转换为流媒体格式，转换延迟小因此不需要预先存储，除了原始的媒体文件不会占用额外的存储空间，显著降低了对存储空间的占用。
附图说明
图1是本公开实施例提供的用于多媒体文件封装的容器的一个可选结构示意图；
图2是本公开实施例中MP4文件的一个可选的封装结构示意图；
图3是本公开实施例中媒体文件中的媒体数据容器存储媒体数据的一个可选的结构示意图；
图4是本公开实施例中FMP4文件的一个可选的封装结构示意图；
图5是本公开实施例提供的媒体文件转换装置的一个可选的结构示意图；
图6是本公开实施例提供的媒体文件转换装置的一个可选的结构示意图；
图7是本公开实施例提供的媒体文件转换方法的一个可选的流程示意图；
图8是本公开实施例提供的从元数据容器解析得到媒体信息的一个可选的流程示意图；
图9是本公开实施例提供的从媒体文件中查找给定时段的媒体数据的一个可选的流程图；
图10是本公开实施例提供的从媒体文件中查找给定时段的媒体数据的位置的一个可选的流程示意图；
图11是本公开实施例提供的封装分段媒体文件的一个可选的流程示意图；
图12为本公开实施例提供的媒体文件转换方法的一个可选的使用场景的示意图；
图13是本公开实施例提供的MP4文件转换为FMP4文件并通过媒体源扩展接口播放的示意图。
具体实施方式
为了使本公开的目的、技术方案和优点更加清楚，下面将结合附图对本公开作进一步地详细描述，所描述的实施例不应视为对本公开的限制，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本公开保护的范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述具体的实施例的目的，不是旨在限制本公开。
对本公开进行进一步详细说明之前，对本公开实施例中涉及的名词和术语进行说明，本公开实施例中涉及的名词和术语适用于如下的解释。
1)媒体文件，以容器(Box，也称为盒子)的方式存储进行编码的媒体数据(例如音频数据和视频数据中的至少一种)的文件，其中还包括用以表达媒体信息以保证媒体数据被正确解码的元数据。
例如，采用动态图像专家组(MPEG，Moving Picture Experts Group)-4封装格式封装媒体数据的形成的媒体文件被称为MP4文件。典型地，MP4文件中存储高级视频编码(AVC，Advanced Video Coding，即H.264)或MPEG-4(Part 2)规范编码的视频数据和高级音频编码(AAC，Advanced Audio Coding)规范编码的音频数据，当然不排除视频和音频的其他编码方式。
2)容器(Box)，也称为盒子，由唯一的类型标识符和长度定义的面向对象的构件，参见图1，是本公开实施例提供的容器的一个可选的结构示意图，包括容器头部(Box Header)和容器数据(Box Data)，其中填充有用于表达各种信息的二进制数据。
容器头部包括容量(size)和类型(type)，容量指明了容器在媒体文件中所占用的长度，类型指明了容器的类型，参见图2，是本公开实施例提供的MP4文件的一个可选的封装结构示意图，MP4文件中涉及的基本容器类型包括文件类型容器(ftyp box)、元数据容器(moov box)和媒体数据容器(mdat box)。
容器数据部分可以存储具体的数据，此时容器称为“数据容器”，容器数据部分也可以进一步封装其他类型的容器，此时容器称为“容器的容器”。
3)轨道(Track)，也称为流(Stream)，媒体数据容器中按时间排序的相关的采样(Sample)，对于媒体数据来说，轨道表示一个视频帧序列或一个音频帧序列，还可以包括与视频帧序列同步的字幕轨，同一轨道中的一组连续的采样称为块(Chunk)。
4)文件类型容器，媒体文件中用于存储文件的容量(即所占用字节的长度)和类型的容器，如图2所示，文件类型容器记为“ftyp box”，其中存储的二进制数据按照规范的字节长度描述了文件的类型和兼容性。
5)元数据容器，媒体文件中用于存储元数据(即描述媒体数据容器中存储的多媒体数据的数据)的容器，在MP4文件中的元数据容器中存储的二进制数据表达的信息称为媒体信息。
如图2所示，元数据容器的头部采用二进制数据表示容器的类型为“moov box”，容器数据部分封装用于存储MP4文件的总体信息的mvhd容器，是独立于MP4文件的，并且与MP4文件的播放相关，包括时长、创建时间和修改时间等。
媒体文件的媒体数据容器中可以包括对应多个轨道的子容器，例如音频轨道容器(audio track box)和视频轨道容器(video track box)，在音频轨道容器和视频轨道容器 的子容器中都包括了相应轨道的媒体数据的引用和描述，必要的子容器包括：用于描述轨道的特性和总体信息(如时长、宽高)的容器(记为tkhd box)、记录轨道的媒体信息(比如媒体类型和采样的信息)的容器(记为mdia box)。
就mdia box中封装的子容器而言，可以包括：记录轨道的相关属性和内容的容器(记为mdhd box)，记录媒体的播放过程信息的容器(记为hdlr box)，描述轨道中媒体数据的媒体信息的容器(记为minf box)；minf box中又封装了用于解释如何定位媒体信息的子容器(记为dinf box)、以及用于记录轨道中采样的所有时间信息(解码时间/显示时间)、位置信息和编解码等信息的子容器(记为stbl box)。
参见图3，是本公开实施例提供的媒体文件中的媒体数据容器存储媒体数据的结构示意图，利用从stbl box中二进制数据所识别出的媒体信息，可以解释采样的时间、类型、容量以及在媒体数据容器中的位置，下面说明stbl box中的各个子容器。
stsd box包含了一个采样描述(sample description)表，根据不同的编码方案和存储数据的文件数目，每个媒体文件中可以有一个或多个描述表，通过描述表可以找到每个采样的描述信息，描述信息可以保证采样的正确的解码，不同的媒体类型存储不同的描述信息，例如，视频媒体而言描述信息就是图像的结构。
stts box存储了采样的时长信息，并提供表来映射时间(解码时间)和采样的序号，通过sttx box，可以定位媒体文件中任何时间的采样；stts box中还使用其他的表来映射采样的容量和指针，表中每个条目提供了在同一个时间偏移量里面连续的采样的序号，以及采样的偏移量，递增这些偏移量，可以建立一个完整的时间-采样的映射表，计算公式如下：
DT(n+1)＝DT(n)+STTS(n)         (1)
其中STTS(n)是第n个采样的时长，DT(n)是第n个采样的显示时间，采样的排列是按照时间的顺序排序，这样偏移量永远是非负的，DT一般以0开始，以第i个采样的显示时间DT(i)为例，计算公式如下：
DT(i)＝SUM(for j＝0 to i-1 of delta(j))         (2)
所有偏移量的和是轨道中媒体数据的时长。
stss box记录了媒体文件中的关键帧的序号。
sts box记录了采样与存储采样的块的映射关系，通过表来映射采样的序号和块的序号之间的关系，通过查表可以找到包含指定采样的块。
stco box定义了每个块在轨道中的位置，位置采用在媒体数据容器的起始字节的偏移量、以及相对于所述起始字节的长度(即容量)表示。
stsz box记录了媒体文件中每个采样的容量(即大小)。
6)媒体数据容器，媒体文件中用于存储多媒体数据的容器，例如，在MP4文件中媒体数据容器，如图3所示，采样是媒体数据容器中存储的单位，存储在媒体文件的块中，块和样本的长度可不相同。
7)分段媒体文件，媒体文件经过分割形成的子文件，每个分段媒体文件能够被独立解码。
以MP4文件为例，MP4文件中的媒体数据根据关键帧分割，分割后的媒体数据与对应的元数据封装形成分段MP4(FMP4，Fragmented MP4)文件，每个FMP4文件中的元数据能够保证媒体数据被正确解码。
例如，在将如图2所示的MP4文件转换为多个FMP4文件时，参见图4，是本公开实施例提供的FMP4文件的一个可选的封装结构示意图，一个MP4文件可以转换为多个FMP4文件，每个FMP4文件包括三个基本的容器：moov容器、moof容器和mdat容器。
moov容器包括了MP4文件级别的元数据，用来描述FMP4文件所来源的MP4文件中的全部媒体数据，例如MP4文件的时长、创建时间和修改时间等。
moof容器存储了分段级别的元数据，用于描述所在的FMP4文件中封装的媒体数据，保证FMP4中的媒体数据能够被解码。
1个moof容器和1个mdat容器组成分段MP4文件的1个分段，1个分段MP4文件中可以包括1个或多个这样的分段，每个分段中封装的元数据保证分段中封装的媒体数据能够被独立解码。
8)媒体资源扩展(MSE，Media Source Extensions)接口，在网页中实现的面向播放器的接口，在网页中的加载期间通过浏览器的解释器解释、执行前端编程语言(例如JavaScript)而实现，向播放器提供调用超文本标记语言(HTML)媒体元素(Media Element)的播放媒体流的功能，例如使用视频元素<video>、以及音频元素<audio>来实现视频/音频的播放功能。
9)流媒体格式，把媒体数据封装为流媒体的媒体文件，媒体文件不必完整下载、不需要额外转码，即可被解码播放，即，原生地支持一边下载，一边播放的封装技术。典型的流媒体格式的文件包括：基于HTTP直播流(HLS，HTTP Live Streaming)技术的TS媒体文件分片，FLV(Flash Video)文件等。
10)非流媒体格式，把媒体数据封装为媒体文件、且媒体文件完整下载后才可以被解码播放的封装技术，典型的非流媒体格式的文件包括：MP4文件，视窗媒体视频(WMV，Windows Media Video)文件，高级串流格式(ASF，Advanced Streaming Format)文件等。
需要指出，MP4文件原生不支持流媒体形式的播放，但是通过在线转码后向播放器转码后的媒体流、或者部分下载的MP4文件的缺失部分填充无效的二进制数据(例如，在ftyp容器和moov容器完整下载的情况下，填充mdat容器的缺失部分以无效的二进制数据代替)也能实现一边下载一边播放的技术效果，本文中将这种原生不支持流媒体播放的文件的封装格式都称为非流媒体格式。
首先说明实现本公开实施例的媒体文件转换装置，媒体文件转换装置可以提供为硬件、软件或者软硬件结合的方式。
下面说明媒体文件转换装置的软件硬件结合的实施，参见图5，图5为本公开实施例媒体文件转换装置一个可选的组成示意图，媒体文件转换装置500可以是带有包括带有视频播放功能的移动电话、计算机、数字广播终端、信息收发设备、游戏控制台、平板设备、医疗设备、健身设备、个人数字助理等。图5所示的媒体文件转换装置500包括：至少一个处理器501、存储器502、至少一个网络接口504和用户接口503。媒体文件转换装置500中的各个组件通过总线系统505耦合在一起。可理解，总线系统505用于实现这些组件之间的连接通信。总线系统505除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图6中将各种总线都标为总线系统505。
其中，用户接口503可以包括显示器、键盘、鼠标、轨迹球、点击轮、按键、按钮、触感板或者触摸屏等。
可以理解，存储器502可以是易失性存储器或非易失性存储器，也可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(ROM，Read Only Memory)、可编程只读存储器(PROM，Programmable Read-Only Memory)、可擦除可编程只读存储器(EPROM，Erasable Programmable Read-Only Memory)、电可擦除可编程只读存储器(EEPROM，Electrically Erasable Programmable Read-Only Memory)；磁表面存储器可以是磁盘存储器或磁带存储器。易失性存储器可以是随机存取存储器 (RAM，Random Access Memory)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(SRAM，Static Random Access Memory)、同步静态随机存取存储器(SSRAM，Synchronous Static Random Access Memory)、动态随机存取存储器(DRAM，Dynamic Random Access Memory)、同步动态随机存取存储器(SDRAM，Synchronous Dynamic Random Access Memory)。本公开实施例描述的存储器502旨在包括这些和任意其它适合类型的存储器。
本公开实施例中的存储器502包括但不限于：三态内容寻址存储器、静态随机存储器能够存储从服务器所获取的媒体文件数据以支持媒体文件转换装置500的操作。这些数据的示例包括：用于在媒体文件转换装置500上操作的任何计算机程序，如操作系统5021和应用程序5022；各类型媒体文件数据信息等。其中，操作系统5021包含各种系统程序，例如框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务。应用程序5022可以包含各种应用程序，例如带有媒体文件转换功能的客户端、或应用程序等。实现本公开实施例媒体文件转换方法的程序可以包含在应用程序5022中。
上述本公开实施例揭示的方法由处理器501实现。处理器501可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，本公开实施例提供的媒体文件转换方法可以通过处理器501中的硬件的集成逻辑电路或者软件形式的操作完成。上述的处理器501可以是通用处理器、数字信号处理器(DSP，Digital Signal Processor)，或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。处理器501可以实现或者执行本公开实施例中的提供的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本公开实施例所提供的方法的步骤，可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于存储介质中，该存储介质位于存储器502，处理器501读取存储器502中的信息，结合其硬件完成媒体文件转换方法。
在示例性实施例中，本公开实施例还提供了一种计算机可读存储介质，例如包括计算机程序的存储器502，上述计算机程序可由媒体文件转换装置500的处理器501执行，以完成的媒体文件转换方法。计算机可读存储介质可以是FRAM、ROM、PROM、EPROM、EEPROM、Flash Memory、磁表面存储器、光盘、或CD-ROM等存储器；也可以是包括上述存储器之一或任意组合的各种设备，如移动电话、计算机、平板设备、个人数字助理等。
下面说明媒体文件转换装置的纯硬件实施，实现本公开实施例媒体文件转换装置可以被一个或多个应用专用集成电路(ASIC，Application Specific Integrated Circuit)、DSP、可编程逻辑器件(PLD，Programmable Logic Device)、复杂可编程逻辑器件(CPLD，Complex Programmable Logic Device)、现场可编程门阵列(FPGA，Field-Programmable Gate Array)或其他电子元件实现，用于执行实现本公开实施例提供的媒体文件转换方法。
下面说明媒体文件转换装置的纯软件实施，实现本公开实施例媒体文件转换装置可以被应用程序或插件，或者采用二者结合的方式实现。
作为示例，应用程序可以是专用于进行媒体文件转换的客户端，也可以是将媒体文件转换功能作为可选功能的客户端，通过安装相应的插件实现。
作为示例，插件可以实现为应用程序的功能升级安装包，在用于媒体播放的应用程序中叠加媒体文件转换的功能；也可以是媒体播放的网页中的元素，采用前端语言实现，通过被浏览器直接解释执行而在网页中实现媒体文件转换以及媒体播放的功能。
作为媒体文件转换装置的硬件实施或软件实施的示例，媒体文件转换装置可以提供 为一系列在信号/信息/数据层面存在耦合关系的模块，下面将结合图6进行说明，参见图6，图6为本公开实施例媒体文件转换装置一个可选的组成示意图，示出了实现媒体文件转换装置包括的一系列模块，但是媒体文件转换装置的模块结构不仅限于图6所示，例如，可以对其中的模块根据所实现的不同功能而进一步拆分或者合并。下面对图6所示的模块实现的功能进行说明。
下文继续根据图5说明图6示出的各个模块的功能。
解析模块601，配置为解析媒体文件的元数据容器中封装的元数据，得到用于描述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，媒体文件采用非流媒体格式。
在本公开的一个实施例中，解析模块601，配置为解析元数据容器中子容器的嵌套结构，根据嵌套结构读取出各子容器中的二进制数据；解析模块601，配置为从所读取出的二进制数据解析出各子容器表征的媒体数据的媒体信息。结合图2所示的结构，由于容器可以为嵌套结构，因此，首先对元数据容器中子容器的嵌套结构进行解析，从mdat容器中解析出相应的二进制数据。通过本实施例所示的技术方案，可以实现对嵌套结构的元数据容器的解析和相关数据的获取。
查找模块602，根据媒体信息表示的媒体数据的时间，查找给定时段的媒体数据。
在本公开的一个实施例中，查找模块602，配置为根据媒体数据的时间查找媒体文件中的两个关键帧，两个关键帧对应给定时段的起始时间和结束时间；查找模块602，配置为根据媒体数据的时间查找与两个关键帧之间的视频帧对齐的音频帧；查找模块602，配置为根据媒体数据的位置，查找两个关键帧之间的视频帧在媒体文件中的位置，并查找音频帧在媒体文件中的位置。合图2所示的结构，通过本实施例所示的技术方案，可以实现通过对嵌套结构中的视频帧的确定，并实现对audio track容器中相应音频帧位置的定位。
查找模块602，配置为根据媒体信息表示的媒体数据在媒体文件中的位置，查找给定时段的媒体数据在媒体文件中的位置。
在本公开的一个实施例中，查找模块602，还配置为根据媒体数据的时间，查找解码时间对齐给定时段的起始时间的第一关键帧；查找模块602，还配置为查找解码时间早于给定时段的起始时间的关键帧中距离起始时间最近的第一关键帧；查找模块602，还配置为根据媒体数据的时间，查找解码时间对齐给定时段的结束时间的第二关键帧；查找模块602，还配置为查找解码时间晚于给定时段的结束时间的关键帧中距离结束时间最近的第二关键帧。
在本公开的一个实施例中，查找模块602，还配置为根据媒体数据的时间，查找解码时间处于给定时段的音频帧；
末音频帧的解码时间对齐给定时段的结束时间，或者，晚于给定时段的结束时间且距离结束时间最近。通过本实施例所示的技术方案，可以实现根据媒体数据的时间，查找解码时间处于给定时段的音频帧，由于首音频帧的解码时间对齐给定时段的开始时间，或者，首音频帧的解码时间早于给定时段的开始时间且距离开始时间最近，因此，所形成的分段媒体文件克服了MP4文件中可能出现的视频、音频时长不一致的问题，保证每帧视频播放时会有同步的音频播放，不会出现无声的现象，提升了用户的使用体验。
提取模块603，配置为从媒体文件的媒体数据容器中提取对应位置的媒体数据。
在本公开的一个实施例中，提取模块603，配置为根据两个关键帧之间的视频帧的位置对应的偏移量和容量，以及与视频帧对齐的音频帧的位置对应的偏移量和容量，确定由最小偏移量和最大容量构成的区间；提取模块603，配置为从媒体文件的媒体数据容器的相应区间内提取媒体数据。通过本实施例所示的技术方案，通过确定由最小偏移 量和最大容量构成的区间；从媒体文件的媒体数据容器的相应区间内提取媒体数据，避免了对媒体数据的二次提取，有效提升了对媒体数据的处理效率。
在本公开的一个实施例中，偏移量表征媒体数据在媒体文件的媒体数据容器中的存储起始位置，容量表征媒体数据从存储起始位置开始占用的长度。
在本公开的一个实施例中，提取模块603，配置为根据视频帧的保存位置，对视频帧进行提取，在视频帧提取完成后，根据音频帧的保存位置对音频帧进行提取。
在本公开的一个实施例中，提取模块603，配置为根据预设的取断值，在媒体数据中对视频帧和音频帧进行提取。通过本实施例所示的技术方案，可以实现通过一次提取获得相应的音频帧和视频帧，以提升对媒体文件的处理速度。
合成模块604，配置为将所提取的媒体数据、以及描述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
在本公开的一个实施例中，合成模块604，配置为将表示媒体文件的类型和兼容性的数据，执行填充操作以填充到段媒体文件的文件类型容器中；合成模块604，配置为将表示媒体文件的文件级别的元数据，填充到分段媒体文件的元数据容器中；合成模块604，配置为将所提取的媒体数据、以及描述媒体数据的元数据，对应填充到分段媒体文件的分段容器中的媒体数据容器、以及分段级别的元数据容器。结合图4所示的结构，合成模块604，配置为将表示媒体文件的类型和兼容性的数据填充到moov容器中，将表示媒体文件的文件级别的元数据填充到moof容器中，将所提取的媒体数据、以及描述媒体数据的元数据填充到mdat容器中。通过本实施例所示的技术方案，通过对媒体数据的封装与填充，可以获得完整的能够用于独立解码的分段媒体文件。
在本公开的一个实施例中，合成模块604，还配置为在分段媒体文件的子容器对应的装置中，调用类的写操作功能以完子容器与具有嵌套关系的子容器的合并。
在本公开的一个实施例中，合成模块604，还配置为调用类的写操作功能在子容器的内存缓冲区完成二进制数据的写入和合并，以及，返回类的实例，所返回的实例用于合并子容器与具有嵌套关系的子容器的合并。
在本公开的一个实施例中，合成模块604，还配置为通过类Stream提供一个内存缓冲区，用来保存写入的二进制数据；通过Stream提供的静态方法，转换待填充的多字节十进制数据到二进制数据；通过每类Stream的实例提供的写操作功能，在内存缓冲区完成待填充到子容器的二进制数据的合并；方法返回一个新的Stream实例，通过本实施例所示的技术方案，可以实现当前子容器与其他具有嵌套关系的子容器的合并，以灵活实现对子容器的调整。
在本公开的一个实施例中，解析媒体文件的元数据容器中封装的元数据之前还包括请求媒体文件的元数据，当请求原分辨率媒体文件或目标分辨率媒体文件的元数据时，首先检查所请求的缓存是否曾经请求过，如果是则继续请求媒体数据，不重复请求元数据，通过本实施例所示的技术方案，可以利用已经请求的媒体数据，节省数据获取的时间，提升操作速度。
在本公开的一个实施例中，合成模块604，还配置为根据元数据容器在媒体文件中的位置，从媒体文件的二进制数据中获取对应元数据容器的二进制数据；顺序解析所获取的二进制数据中对应元数据容器头部的规范长度的二进制数据，得到元数据容器中子容器的容器类型、以及子容器的容器数据的长度；调用与子容器的容器类型相应类型的解析器，顺序解析未解析的数据中与容器数据的长度相应的二进制数据，得到容器数据所表示的媒体信息。
在本公开的一个实施例中，对播放器获取给定时段内媒体数据的流程进行说明。当通过内嵌于网页的播放器播放一部视频文件或者一个轨道的媒体文件的时，播放器必须 能够正确的解析数据流，并对一定的时间获取对应的媒体数据并确保这一段媒体数据能够被独立解码。
将结合本公开实施例提供的媒体文件转换装置，说明实现本公开实施例的媒体文件转换方法，下文中以媒体文件为MP4文件为例进行说明，可以理解，根据下载记载的针对MP4文件的转换方案可以轻易应用于其他非流媒体格式，诸如MKV、WMV和ASF等其他非流媒体格式的媒体文件。
参见图7，图7为本公开实施例提供的媒体文件转换方法的一个可选的流程示意图，将结合图7示出的步骤进行说明。
步骤701，解析媒体文件的元数据容器中封装的元数据，得到用于描述媒体文件的媒体数据容器中封装的媒体数据的媒体信息。
在本公开的一个实施例中，媒体文件为MP4文件，解析媒体文件的元数据容器中子容器的嵌套结构，根据嵌套结构读取出各子容器中的二进制数据；从所读取出的二进制数据解析出各子容器表征的媒体数据的媒体信息。
结合图2所示的结构，MP4文件的moov容器为嵌套结构，对元数据容器中子容器的嵌套结构进行解析，确定moov容器所嵌套的子容器，例如mvhd容器、音频轨道容器和视频轨道容器等，如果子容器还嵌套有容器则继续解析，直至解析到不再嵌套有容器的子容器时，读取相应的子容器中封装的二进制数据，解析得到二进制数据所表示的媒体消息，例如stss box记录的媒体文件中的关键帧的序号，stsz box记录的媒体文件中每个采样的容量(即大小)，等。
在本公开的一个实施例中，提供根据容器类型设置解析器，根据容器类型解析元数据容器中的子容器从而得到媒体信息的方式，将结合图8进行说明。
参见图8，图8是本公开实施例提供的从元数据容器解析得到媒体信息的一个可选的流程示意图，将结合图8示出的步骤进行说明。
步骤801，定位元数据容器在媒体文件中的位置。
在一个实施例中，通过从媒体文件的二进制数据读取符合容器头部规范的二进制数据，根据读取的二进制数据中识别的容器的类型和长度，直至定位元数据容器在媒体文件中的偏移量和容量。
举例来说，对于媒体文件的二进制数据，从零字节起始的二进制数据对应文件类型容器，通过容器头部的规范长度，读取媒体文件的二进制数据的起始位置，读取符合容器头部的规范长度的二进制数据，解析读取的二进制数据可以确定媒体文件中位于文件类型容器之后的容器的类型和长度。
如果解析出的类型是文件类型容器，则可以解析出元数据容器的长度(即容量)，此时元数据容器的偏移量为文件类型容器的长度。
如果解析出的类型是媒体数据容器，则根据媒体数据容器的长度和分类型容器的长度为偏移量，继续读取符合容器头部的规范长度的二进制数据，从而可以解析出元数据容器的长度(即容量)，此时元数据容器的长度为文件类型容器的长度与媒体数据容器的长度的加和。
媒体文件中除了起始的容器是文件类型容器，后续容器的封装顺序没有规范，通过上述解析方式，不管媒体文件中容器的封装顺序是文件类型容器、元数据容器和媒体数据容器，还是文件类型容器、媒体数据容器和元数据容器，都能够准确高效定位文件类型容器在媒体文件中的位置。
步骤802，根据元数据容器在媒体文件中的位置从媒体文件的二进制数据中获取对应元数据容器的二进制数据。
元数据容器在媒体文件中的位置以偏移量和容量表示，从媒体文件中读取以偏移量 对应的位置开始读取二进制数据，直至读取的二进制数据的长度符合元数据容器的容量，从而读取出元数据容器对应的二进制数据。
步骤803，顺序解析元数据容器二进制数据中对应容器头部的规范长度的二进制数据，得到元数据容器中子容器的容器类型、以及子容器的容器数据的长度。
在一个实施例中，针对元数据容器中嵌套多个子容器的情况，每次读取二进制数据的偏移量都是已经识别出子容器长度的加和，所读取的二进制数据的长度符合容器头部的规范长度，从而可以解析出当前所处理子容器的类型和长度。
例如，首次读取时，从元数据容器的二进制数据的零字节开始读取二进制数据，且所读取的二进制数据的长度符合容器头部的规范长度，从而可以解析出第一个子容器的类型和长度；第二次读取时，以首次读取的子容器的长度为偏移量，开始读取二进制数据，且所读取的二进制数据的长度符合容器头部的规范长度，从而可以解析出第二个子容器的类型和长度。
通过上述方式读取二进制数据不会出现多读导致回退的情况，也不会出现及少读而二次读取的情况，解析的效率和正确率得以保证。
步骤804，调用与子容器的容器类型相应类型的解析器，顺序解析未解析的数据中与容器数据的长度相应的二进制数据，得到容器数据所表示的媒体信息。
在一个实施例中，对元数据容器中嵌套的典型容器类型进行预标记，用于表示容器是直接用于封装二进制数据还是进一步封装有容器，例如对于图2示出的mvhd box、audio track box和video track box等标记进一步封装有容器，对于图2示出的stts box、stsd box等标记为直接封装有二进制数据。
对于标记为直接封装二进制数据的容器类型，设置与容器类型一一对应的解析器，解析器用于根据二进制数据解析出所表示的媒体信息；在步骤804中，将在步骤803中解析出的子容器的容器类型与预标记的容器类型比对时，涉及到以下两种情况。
情况1)当通过比对确定子容器的容器类型被预标记、且被预标记为用于直接封装二进制数据时，则调用与子容器的容器类型对应的解析器，通过解析器解析子容器中的容器数据，得到容器数据所表示的媒体信息。
情况2)当通过比对确定子容器的容器类型被预标记、且被预标记为用于继续封装容器时，根据媒体文件中容器头部的规范长度，递归解析子容器所对应的二进制数据，直至解析到子容器中所封装容器的容器类型被预标记、且被预标记为用于直接封装二进制数据,调用与子容器中所封装容器的容器类型相应的解析器，逐字节解析二进制数据，所解析二进制数据的长度与子容器中所封装容器的容器数据的长度对应，以得到子容器中所封装容器的容器数据所表示的媒体信息。
在一个实施例中，对解析元数据容器的过程中记录媒体信息的方式进行说明，在顺序解析元数据容器二进制数据中对应容器头部的规范长度的二进制数据，得到元数据容器中子容器的容器类型时，根据子容器与所归属的容器之间的嵌套关系、以及子容器与所封装的容器嵌套关系建立对象，当子容器的容器类型被预标记为用于直接封装二进制数据时，在对应子容器所建立的对象中存储包括有媒体信息的数组，所存储的媒体信息由子容器的容器数据表示。
例如，在图2中，当解析的子容器的类型为stts box时，由于stts box被预标记为直接封装二进制数据，在对应stts box所建立的对象中存储包括有媒体信息的数组，这里的媒体信息是stts box的容器数据所表示的时长信息。
在一个实施例中，对解析元数据容器的过程中记录子容器之间的嵌套关系的方式说明，在顺序解析元数据容器二进制数据中对应容器头部的规范长度的二进制数据，得到元数据容器中子容器的容器类型时，如果容器类型被预标记为直接封装二进制数据， 则在被调用的解析器中记录所解析的子容器；将所记录的子容器的实例设置到子容器属性中，子容器属性包括于子容器所归属的容器，用于描述子容器与所归属的容器之间的嵌套关系。
例如，在图2中，当解析的子容器的类型为stsd box时，由于stsd box被预标记为直接封装二进制数据，在stsd box所对应的解析器中记录stsd box，将stsd box的实例设置到stbl box子容器属性中，以此类推，最后在stsd box的子容器属性中记录了stsd box、stts box、stsc box等多个嵌套于stbl box的子容器。
在一个实施例中，通过比对确定子容器的容器类型没被预标记、或者被预标记为直接封装二进制数据但是没有调用到相应类型的解析器时，忽略解析子容器对应的二进制数据，根据子容器的长度，跳转到二进制数据中对应下一子容器的部分继续解析。
事实上，媒体文件中会出现自定义的容器类型，通过跳转的方式不会影响元数据容器的整体解析的进度，同时通过设置解析器的方式，当元数据容器的容器类型发生变动时，通过对相应类型的解析器的增加、删除和修改，可以快速实现对最新的元数据容器的兼容解析，具有升级灵活快速的特点。
步骤702，根据媒体信息表示的媒体数据的时间，查找给定时段的媒体数据。
在本公开的一个实施例中，参见图9，图9是本公开实施例提供的从媒体文件中查找给定时段的媒体数据的一个可选的流程图，将结合图9示出的步骤进行说明。
步骤901，根据媒体数据的时间查找媒体文件中的两个关键帧，两个关键帧对应给定时段的起始时间和结束时间。
在本公开的一个实施例中，在根据媒体数据的时间查找媒体文件中的两个关键帧时根据媒体数据的时间，查找解码时间对齐给定时段的起始时间的第一关键帧的序号，或者，查找解码时间早于给定时段的起始时间的关键帧中距离起始时间最近的第一关键帧的序号；根据媒体数据的时间，查找解码时间对齐给定时段的结束时间的第二关键帧的序号，或者，查找解码时间晚于给定时段的结束时间的关键帧中距离结束时间最近的第二关键帧的序号。
例如，媒体文件的时长为120秒，其中包含若干关键帧，当给定时间段为第20至40秒时，根据帧的解码时间与帧的序号的映射关系，首先查找是否有解码时间恰好处于第20秒的关键帧，如果存在，则解码时间恰好处于第20秒的关键帧作为第一关键帧；如果不存在，则查找解码时间早于第20秒、且距离起始时间第20秒最近的一个关键帧作为第一关键帧。
其次，查找是否有解码时间恰好处于第40秒的关键帧，如果存在，则解码时间恰好处于第40秒的关键帧作为第二关键帧；如果不存在，则查找解码时间晚于第40秒、且距离第40秒最近一个关键帧作为第二关键帧。
通过查找视频帧中的关键帧的方式，能够避免起始时间所对应的是普通帧而无法播放所发生跳帧的情况，同时最大程度减少了获取无用的数据，在网络传输时节约流量。
步骤902，根据媒体数据的时间查找与两个关键帧之间的视频帧对齐的音频帧。
在一实施例中，播放器可通过如下方式从媒体数据的时间中查找与给定时段对齐的音频帧：从媒体数据的时间中查找解码时间根据给定时段分布的音频帧，以视频帧为基准，在视频帧定位时间同步的音频帧；其中，当播放点的时间对应存在音频帧时，首音频帧的解码时间对齐给定时段的开始时间；当播放点的时间不存在对应的音频帧时，首音频帧的解码时间早于给定时段的开始时间且距离开始时间最近，保证首音频帧的解码时间不晚于首视频帧(上述第一关键帧)的解码起始时间；当给定时段的结束时间对应存在音频帧时，末音频帧的解码时间对齐给定时段的结束时间；当给定时段的结束时间不存在对应的音频帧时，末音频帧的解码时间晚于给定时段的结束时间且距离结束时间 最近，保证末音频帧的解码时间不早于末视频帧(上述第二关键帧)的解码时间。
通过上述查找音频帧的方式，所形成的包括视频帧和相应音频帧的分段媒体文件克服了可能出现的视频、音频时长不一致的问题，保证每帧视频播放时会有同步的音频播放，不会出现无声的现象。
步骤903，根据媒体数据的位置，查找两个关键帧之间的视频帧在媒体文件中的位置，并查找音频帧在媒体文件中的位置。
在一个实施例中，根据步骤703，媒体数据在媒体文件中的位置可以表示为两个区间：第一个区间，即两个关键帧之间的视频帧在媒体文件中的位置，包括在媒体文件的媒体数据容器中的偏移量和容量；第二个区间，即两个音频帧之间的音频帧在媒体文件中的位置，包括在媒体文件的媒体数据容器中的偏移量和容量。其中，偏移量表征媒体数据在媒体文件的媒体数据容器中的存储起始位置，容量表征媒体数据从存储起始位置开始占用的长度。
在另一实施例中，媒体数据在媒体文件中的位置表示为一个区间，根据两个关键帧之间的视频帧的位置对应的偏移量和容量，以及与视频帧对齐的音频帧的位置对应的偏移量和容量，确定由最小偏移量和最大容量构成的区间。
例如，第一关键帧和第二关键帧之间的视频帧在目标分辨率媒体文件的位置的偏移量，对应区间为[a，b](地址为升序)，音频帧在目标分辨率媒体文件的位置的偏移量，对应区间为[c，d](地址为升序)，那么，取位置的上限和下限构成的区间即为[min(a，c)，max(b，d)]。
通过上述的确定一个区间的方式，能够从媒体文件中一次性提取给定时间的媒体数据，减少了播放器的读取次数，提高了处理效率。如此，播放器发送携带目标区间的偏移量和容量的网络请求给服务器，以请求目标区间的媒体数据，服务器基于目标区间的偏移量和容量提取媒体文件中的媒体数据后一次性的返回目标区间的媒体数据，无须二次获取，减少了播放器的请求次数，提高了处理效率。
结合图2所示的结构，通过本实施例所示的技术方案，可以实现通过对嵌套结构中的视频帧的确定，并实现对音频轨道(audio track)容器中相应音频帧位置的定位。
当通过内嵌于网页的播放器播放媒体文件的给定时段时，对于基于给定时段的媒体数据所构造的分段媒体文件时，需要获知部分媒体数据在媒体文件中的位置，包括偏移量和容量，从而能够正确地提取出媒体数据以构造分段媒体文件从而能够正确解码。
步骤703，根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置。
下面，以媒体文件采用图4示出的封装结构为例，对从媒体文件中查找给定时段的媒体数据在媒体文件中的位置说明，参见图10，图10是本公开实施例提供的从媒体文件中查找给定时段的媒体数据的位置的一个可选的流程示意图，将结合图10示出的步骤进行说明。
步骤1001，确定待获取的媒体数据对应的时段。
时段是接续播放点的一段时间，播放点对应的时间是相对于媒体时间坐标系统(以媒体文件的播放开始时间为时间原点)的时间度量，时段的长度小于媒体文件的长度，例如媒体文件长度的预定比例5％，或者是设定的长度如10分钟。
步骤1002，检查stts容器(box)来确定解码时间处于给定时段内的采样(包括视频帧和音频帧)的序号。
例如，对于音频帧，检查stts box来确定解码时间对应给定时段的音频帧的序号；对于视频帧，由于采用压缩算法，给定时段内的首帧如果不是关键帧，还需要根据时间顺序向给定时段的开始时间之前追溯到关键帧，以保证给定时段内的帧能够被解码。
步骤1003，根据采样的序号查询stsc box，确定包括采样的块的序号。
步骤1004，根据块的序号，从stco box中查找块的偏移量。
步骤1005，根据采样的序号查找stsz box，找到采样在块内的偏移量和采样的容量。
在stsz box记录了采样的序号以及容量，将块的偏移量与采样在块内的偏移量累加，得到采样在mdat容器(box)中的偏移量。
步骤1006，以偏移量最小的块所对应的偏移量、以及相应采样在块内的偏移量累加，作为给定时段的媒体数据的偏移量。
步骤1007，以偏移量最大的块所对应的偏移量、相应采样在块内的偏移量、以及相应采样的容量累加，作为能够覆盖媒体数据的长度，并与媒体数据的偏移量做差值，得到媒体数据的容量。
对于步骤1006和步骤1007而言，计算给定时段内的音频帧的位置(包括偏移量和容量)，以及给定时段内的视频帧的位置(包括偏移量和容量)，分别得到视频帧和音频帧在mdat box中存储时占用的区间。
步骤704，从媒体文件的媒体数据容器中提取对应位置的媒体数据。
在本公开的一个实施例中，当媒体文件数据在媒体数据容器的位置使用根据步骤703记载的两个区间表示时，即分别使用视频帧以及音频帧对应的区间表示时，根据视频帧的位置，从媒体数据容器中提取视频帧对应的二进制数据；根据音频帧的位置从媒体数据容器中提取对应的二进制数据。
在本公开的一个实施例中，当媒体文件数据在媒体数据容器的位置使用上述实施例记载的一个区间表示时，从媒体文件的媒体数据容器的相应区间内一次性提取媒体数据，有效提升了对媒体数据的处理效率，尤其在需要网络传输时有显著节约对连接的占用、以及对流量的消耗。
通过本实施例所示的技术方案，可以实现通过一次提取获得相应的音频帧和视频帧，以提升对媒体文件的处理速度。
步骤705，将所提取的媒体数据、以及描述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
在本公开的一个实施例中，参见图11，图11是本公开实施例提供的封装分段媒体文件的一个可选的流程示意图，将结合图11示出的步骤进行说明。
步骤1101，将表示分段媒体文件的类型和兼容性的数据，填充到分段媒体文件的文件类型容器中。
例如，以封装形成如图4时所示的封装结构的FMP4文件为例，在FMP4文件的文件类型容器即ftyp box的头部填充容器的类型和长度(表示ftyp box的整体长度)，在ftyp box的数据部分填充生成表示文件类型为FMP4以及兼容协议的数据(二进制数据)。
步骤1102，将表示分段媒体文件的文件级别的元数据，填充到分段媒体文件的元数据容器中。
在一个实施例中，根据向分段媒体文件的封装结构待填充的媒体数据，根据分段媒体文件中的元数据容器的嵌套结构，计算填充嵌套结构所需要的描述媒体数据的元数据。
仍以图4为例，计算表示FMP4文件的文件级别的元数据，并填充到FMP4的元数据容器(即moov box)中，在moov box中嵌套有mvhd、track和视频扩展(mvex，movie extend)三个容器。
其中，mvhd容器中封装的元数据用于表示与分段媒体文件的播放相关的媒体信息，包括位置、时长、创建时间和修改时间等；track容器中嵌套的子容器表示媒体数据中相应轨道的引用和描述，例如track容器中嵌套有描述轨道的特性和总体信息(如时长、 宽高)的容器(记为tkhd box)、记录轨道的媒体信息(比如媒体类型和采样的信息)的容器(记为mdia box)。
步骤1103，将所提取的媒体数据、以及描述媒体数据的元数据，对应填充到分段媒体文件的分段容器中的媒体数据容器、以及分段级别的元数据容器中。
在一个实施例中，分段媒体文件中可以封装有一个或多个分段(fragment)，对于待填充的媒体数据而言，可以填充到分段媒体文件的一个或分段的媒体数据容器(即mdat box)中，每个分段中封装有分段级别的元数据容器(记为moof box)，其中填充的元数据用以描述分段中填充的媒体数据，使分段能够被独立解码。
结合图4，以将待填充的媒体数据填充到FMP4文件的封装结构的2个分段中为例，填充到每个分段媒体数据；计算需要填充到相应分段的分段级别的元数据容器(即moof box)中的元数据，并对应填充到moof box嵌套的子容器中，其中在moof box的头部称为moof box，其中填充的二进制数据用于表示容器的类型为“moof box”以及moof box的长度。
在步骤1101至步骤1103中填充数据到相应容器的一个实施例中，当执行填充操作时，调用类的写操作功能在子容器的内存缓冲区完成二进制数据的写入和合并，以及，返回类的实例，所返回的实例用于合并子容器与具有嵌套关系的子容器的合并。
作为填充数据的一个示例，建立用于实现封装功能的类MP4，将分段媒体文件中的每个子容器封装为类Stream的静态方法；建立用于实现二进制数据操作功能的类Stream，每个类Stream提供有一个内存缓冲区，用来保存待填充的二进制数据；通过Stream提供的静态方法，转换待填充的多字节十进制数据到二进制数据；通过类Stream的实例提供的写操作功能，在内存缓冲区完成待填充到子容器的二进制数据的合并以及填充；Stream提供的静态方法返回一个新的Stream实例，可以实现当前子容器与其他具有嵌套关系的子容器的合并。
图12为本公开实施例提供的媒体文件转换方法的一个可选的使用场景的示意图，参见图12，本公开实施例提供的媒体文件转换装置可以实施为用户终端10，为实现支撑一个示例性应用，用户终端10(示例性示出了用户终端10-1和用户终端10-2)通过网络20连接服务器30，网络20可以是广域网或者局域网，又或者是二者的组合，使用无线链路实现数据传输。
用户终端10通过嵌入有播放器的网页进行媒体文件播放，并通过图形界面110(示例性示出了图形界面110-1和图形界面110-2)显示所播放的内容，在播放的过程中，用户终端10解析媒体文件的元数据容器中封装的从服务器30处获取的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，所述媒体文件采用非流媒体格式；根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
图13是本公开实施例提供的MP4文件转换为FMP4文件并通过媒体源扩展接口播放的示意图，参见图13，播放器基于媒体文件的真实地址(图中http://www.toutiao.com/a/b.mp4)获取满足固定时长的媒体数据，然后基于获取的媒体数据构造分段媒体文件，即将其转换为分段MP4格式的媒体文件，然后将分段媒体文件添加到MSE的媒体源对象(例如采用类文件对象(Blob)的方式实现)，MSE创建对应媒体源对象的虚拟URL，并向Video元素传递该虚拟URL，以使Video元素获取相应的媒体源对象，实现上述将所提取的媒体数据、以及描述所述媒体数据的元数据根据分 段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
通过本公开所提供的媒体文件转换方法，通过解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。能够实现从媒体文件中提取任意给定时段的媒体数据以构造分段媒体文件，从而，当需要播放媒体文件的给定时段时，只需要从非流媒体格式的媒体文件提取给定时间的媒体数据，封装为可独立解码的分段媒体文件即可，由此，克服了非流媒体格式文件需要完整下载后才可以独立播放的限制，播放实时性好；同时由于仅需要针对给定时段构造分段媒体文件，而不是将完整的媒体文件预先转换为流媒体格式，转换延迟小因此不需要预先存储，除了原始的媒体文件不会占用额外的存储空间，显著降低了对存储空间的占用。
以上所述，仅为本公开的较佳实施例而已，并非用于限定本公开的保护范围，凡在本公开的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本公开的保护范围之内。

Claims (20)

  1. 一种媒体文件转换方法，包括：
    解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，所述媒体文件采用非流媒体格式；
    根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；
    根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；
    从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；
    将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
  2. 根据权利要求1所述的方法，其中，所述根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据，包括：
    根据所述媒体数据的时间查找所述媒体文件中的两个关键帧，所述两个关键帧对应所述给定时段的起始时间和结束时间；
    根据所述媒体数据的时间查找与所述两个关键帧之间的视频帧对齐的音频帧；
    根据所述媒体数据的位置，查找所述两个关键帧之间的视频帧在所述媒体文件中的位置，并查找所述音频帧在所述媒体文件中的位置。
  3. 根据权利要求2所述的方法，其中，所述根据所述媒体数据的时间查找所述媒体文件中的两个关键帧，包括：
    根据所述媒体数据的时间，查找解码时间对齐所述给定时段的起始时间的第一关键帧，或者，查找解码时间早于所述给定时段的起始时间的关键帧，并在所查找到的关键帧中查找解码时间距离所述起始时间最近的第一关键帧；
    根据所述媒体数据的时间，查找解码时间对齐所述给定时段的结束时间的第二关键帧，或者，查找解码时间晚于所述给定时段的结束时间的关键帧，并在所查找到的关键帧中查找解码时间距离所述结束时间最近的第二关键帧。
  4. 根据权利要求3所述的方法，其中，所述根据所述媒体数据的时间查找与所述两个关键帧之间的视频帧对齐的音频帧，包括：
    根据所述媒体数据的时间，查找解码时间处于所述给定时段的音频帧；
    其中，首音频帧的解码时间对齐所述给定时段的开始时间，或者，早于所述给定时段的开始时间且距离所述开始时间最近；
    末音频帧的解码时间对齐所述给定时段的结束时间，或者，晚于所述给定时段的结束时间且距离所述结束时间最近。
  5. 根据权利要求1所述的方法，其中，所述解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，包括：
    解析所述元数据容器中子容器的嵌套结构，根据所述嵌套结构读取出各子容器中的二进制数据；
    从所读取出的二进制数据解析出各子容器表征的所述媒体数据的媒体信息。
  6. 根据权利要求1所述的方法，其中，所述从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据，包括：
    根据所述两个关键帧之间的视频帧的位置对应的偏移量和容量，以及与所述视频帧对齐的音频帧的位置对应的偏移量和容量，确定由最小偏移量和最大容量构成的区间；
    从所述媒体文件的媒体数据容器的相应区间内提取所述给定时段的媒体数据。
  7. 根据权利要求6所述的方法，其中，
    所述偏移量表征所述媒体数据在所述媒体文件的媒体数据容器中的存储起始位置，所述容量表征所述媒体数据从所述存储起始位置开始占用的长度。
  8. 根据权利要求1所述的方法，其中，所述将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，包括：
    将表示所述分段媒体文件的类型和兼容性的数据，填充到所述分段媒体文件的文件类型容器中；
    将表示所述分段媒体文件的文件级别的元数据，填充到所述分段媒体文件的元数据容器中；
    将所提取的媒体数据、以及描述所述媒体数据的元数据，对应填充到所述分段媒体文件的分段容器中的媒体数据容器、以及分段级别的元数据容器。
  9. 根据权利要求1至8任一项所述的方法，其中，所述将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，包括：
    调用类的写操作功能在所述分段媒体文件的子容器的内存缓冲区完成二进制的媒体数据的写入和合并，以及，
    合并所述子容器与具有嵌套关系的子容器。
  10. 一种媒体文件转换装置，包括：
    解析模块，配置为解析媒体文件的元数据容器中封装的元数据，得到用于描述所述媒体文件的媒体数据容器中封装的媒体数据的媒体信息，所述媒体文件采用非流媒体格式；
    查找模块，根据所述媒体信息表示的所述媒体数据的时间，查找给定时段的媒体数据；
    所述查找模块，配置为根据所述媒体信息表示的所述媒体数据在所述媒体文件中的位置，查找所述给定时段的媒体数据在所述媒体文件中的位置；
    提取模块，配置为从所述媒体文件的媒体数据容器中提取对应所述位置的媒体数据；
    合成模块，配置为将所提取的媒体数据、以及描述所述媒体数据的元数据根据分段媒体文件的封装结构进行封装，得到能够用于独立解码的分段媒体文件。
  11. 根据权利要求10所述的装置，其中，
    所述查找模块，配置为根据所述媒体数据的时间查找所述媒体文件中的两个关键帧，所述两个关键帧对应所述给定时段的起始时间和结束时间；
    所述查找模块，配置为根据所述媒体数据的时间查找与所述两个关键帧之间的视频帧对齐的音频帧；
    所述查找模块，配置为根据所述媒体数据的位置，查找所述两个关键帧之间的视频帧在所述媒体文件中的位置，并查找所述音频帧在所述媒体文件中的位置。
  12. 根据权利要求11所述的装置，其中，
    所述查找模块，还配置为根据所述媒体数据的时间，查找解码时间对齐所述给定时段的起始时间的第一关键帧；
    所述查找模块，还配置为找解码时间早于所述给定时段的起始时间的关键帧，并在所查找到的关键帧中查找解码时间距离所述起始时间最近的第一关键帧；
    所述查找模块，还配置为根据所述媒体数据的时间，查找解码时间对齐所述给定时段的结束时间的第二关键帧；
    所述查找模块，还配置为查找解码时间晚于所述给定时段的结束时间的关键帧，并 在所查找到的关键帧中查找解码时间距离所述结束时间最近的第二关键帧。
  13. 根据权利要求12所述的装置，其中，
    所述查找模块，还配置为根据所述媒体数据的时间，查找解码时间处于所述给定时段的音频帧；
    其中，首音频帧的解码时间对齐所述给定时段的开始时间，或者，早于所述给定时段的开始时间且距离所述开始时间最近；
    末音频帧的解码时间对齐所述给定时段的结束时间，或者，晚于所述给定时段的结束时间且距离所述结束时间最近。
  14. 根据权利要求10所述的装置，其中，
    所述解析模块，配置为解析所述元数据容器中子容器的嵌套结构，根据所述嵌套结构读取出各子容器中的二进制数据；
    所述解析模块，配置为从所读取出的二进制数据解析出各子容器表征的所述媒体数据的媒体信息。
  15. 根据权利要求10所述的装置，其中，
    所述提取模块，配置为根据所述两个关键帧之间的视频帧的位置对应的偏移量和容量，以及与所述视频帧对齐的音频帧的位置对应的偏移量和容量，确定由最小偏移量和最大容量构成的区间；
    所述提取模块，配置为从所述媒体文件的媒体数据容器的相应区间内提取所述给定时段的媒体数据。
  16. 根据权利要求15所述的装置，其中，
    所述偏移量表征所述媒体数据在所述媒体文件的媒体数据容器中的存储起始位置，所述容量表征所述媒体数据从所述存储起始位置开始占用的长度。
  17. 根据权利要求10所述的装置，其中，
    所述合成模块，配置为将表示所述分段媒体文件的类型和兼容性的数据，执行填充操作以填充到所述分段媒体文件的文件类型容器中；
    所述合成模块，配置为将表示所述分段媒体文件的文件级别的元数据，填充到所述分段媒体文件的元数据容器中；
    所述合成模块，配置为将所提取的媒体数据、以及描述所述媒体数据的元数据，对应填充到所述分段媒体文件的分段容器中的媒体数据容器、以及分段级别的元数据容器。
  18. 根据权利要求10至17所述的装置，其中，
    所述合成模块，还配置为调用类的写操作功能在所述分段媒体文件的子容器的内存缓冲区完成二进制的媒体数据的写入和合并；
    所述合成模块，还配置为合并所述子容器与具有嵌套关系的子容器。
  19. 一种媒体文件转换装置，包括：
    存储器，配置为存储可执行指令；
    处理器，配置为执行所述可执行指令时，实现如权利要求1至9任一项所述的媒体文件转换方法。
  20. 一种存储介质，存储有可执行指令，所述可执行指令被执行时，配置为实现如权利要求1至9任一项所述的媒体文件转换方法。
PCT/CN2018/103581 2018-05-29 2018-08-31 媒体文件转换方法、装置及存储介质 Ceased WO2019227751A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2020552076A JP7068489B2 (ja) 2018-05-29 2018-08-31 メディアファイル変換方法、装置及び記憶媒体
US16/913,821 US11064269B2 (en) 2018-05-29 2020-06-26 Conversion method, device and storage medium for media file

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810530467.5 2018-05-29
CN201810530467.5A CN110545490B (zh) 2018-05-29 2018-05-29 媒体文件转换方法、装置及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/913,821 Continuation US11064269B2 (en) 2018-05-29 2020-06-26 Conversion method, device and storage medium for media file

Publications (1)

Publication Number Publication Date
WO2019227751A1 true WO2019227751A1 (zh) 2019-12-05

Family

ID=68697358

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/103581 Ceased WO2019227751A1 (zh) 2018-05-29 2018-08-31 媒体文件转换方法、装置及存储介质

Country Status (4)

Country Link
US (1) US11064269B2 (zh)
JP (1) JP7068489B2 (zh)
CN (1) CN110545490B (zh)
WO (1) WO2019227751A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873176A (zh) * 2021-10-27 2021-12-31 北京奇艺世纪科技有限公司 一种媒体文件合并方法及装置

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114598915B (zh) * 2020-12-03 2025-03-18 南京中兴软件有限责任公司 一种媒体服务方法、装置、设备及计算机存储介质
CN112507148B (zh) * 2020-12-16 2024-10-11 北京华宇信息技术有限公司 媒体文件自动合成方法及装置
CN112584061B (zh) * 2020-12-24 2023-08-01 咪咕文化科技有限公司 多媒体通用模板生成方法、电子设备及存储介质
CN112687375A (zh) * 2021-01-06 2021-04-20 武汉联影医疗科技有限公司 Dicom文件传输方法、系统、装置、服务器和存储介质
CN113973222A (zh) * 2021-12-02 2022-01-25 成都统信软件技术有限公司 一种视频播放方法、装置及计算设备
CN114845135B (zh) * 2022-04-24 2025-07-15 腾讯音乐娱乐科技（深圳）有限公司 流媒体数据处理方法、流媒体播放方法、设备和程序产品
US12149769B2 (en) * 2022-06-15 2024-11-19 Microsoft Technology Licensing, Llc Self-driven adaptive upload
US12341844B2 (en) * 2022-06-15 2025-06-24 Microsoft Technology Licensing, Llc Self-driven adaptive upload
CN116110410B (zh) * 2023-04-14 2023-06-30 北京算能科技有限公司 音频数据处理方法、装置、电子设备和存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1949876A (zh) * 2006-11-21 2007-04-18 华为技术有限公司 一种支持多种编码格式的媒体数据的方法及系统
US20080010658A1 (en) * 1997-03-31 2008-01-10 Abbott Michael J System and method for media stream indexing and synchronization
CN102413358A (zh) * 2011-08-12 2012-04-11 青岛海信传媒网络技术有限公司 流媒体文件存储和播放的方法、装置及系统
CN102510519A (zh) * 2011-10-11 2012-06-20 成都市华为赛门铁克科技有限公司 流媒体数据的处理方法、播放方法以及装置

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003114845A (ja) * 2001-10-03 2003-04-18 Hitachi Ltd メディア変換方法およびメディア変換装置
JP2006129078A (ja) * 2004-10-28 2006-05-18 Canon Inc データファイル編集方法及び装置及び制御プログラム及び記憶媒体
JP2008199387A (ja) * 2007-02-14 2008-08-28 Hitachi Software Eng Co Ltd 動画部分配信システム
CN103051953B (zh) * 2012-12-30 2016-01-20 乐视网信息技术（北京）股份有限公司 一种利用视频播放器对视频文件进行解码方法
JP6411862B2 (ja) * 2013-11-15 2018-10-24 パナソニック株式会社 ファイル生成方法およびファイル生成装置
CN107979783B (zh) * 2016-10-25 2020-03-24 杭州海康威视数字技术股份有限公司 一种流式数据解析方法、装置及电子设备
CN106961613A (zh) * 2017-03-30 2017-07-18 上海七牛信息技术有限公司 一种流式实时转码点播方法及系统
US10694241B2 (en) * 2017-06-08 2020-06-23 T-Mobile Usa, Inc. Capturing border metadata while recording content

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080010658A1 (en) * 1997-03-31 2008-01-10 Abbott Michael J System and method for media stream indexing and synchronization
CN1949876A (zh) * 2006-11-21 2007-04-18 华为技术有限公司 一种支持多种编码格式的媒体数据的方法及系统
CN102413358A (zh) * 2011-08-12 2012-04-11 青岛海信传媒网络技术有限公司 流媒体文件存储和播放的方法、装置及系统
CN102510519A (zh) * 2011-10-11 2012-06-20 成都市华为赛门铁克科技有限公司 流媒体数据的处理方法、播放方法以及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113873176A (zh) * 2021-10-27 2021-12-31 北京奇艺世纪科技有限公司 一种媒体文件合并方法及装置
CN113873176B (zh) * 2021-10-27 2024-03-08 北京奇艺世纪科技有限公司 一种媒体文件合并方法及装置

Also Published As

Publication number Publication date
US11064269B2 (en) 2021-07-13
US20200329283A1 (en) 2020-10-15
JP7068489B2 (ja) 2022-05-16
CN110545490B (zh) 2021-03-16
CN110545490A (zh) 2019-12-06
JP2021508429A (ja) 2021-03-04

Similar Documents

Publication Publication Date Title
CN110545490B (zh) 媒体文件转换方法、装置及存储介质
US11178452B2 (en) Playing method, device and storage medium of webpage-based media file
US10924801B2 (en) Method and device for playing media file while switching resolution in webpage and storage medium
US11012759B2 (en) Webcasting method, device and storage medium of media file
CN110545456B (zh) 媒体文件的同步播放方法、装置及存储介质
WO2019227742A1 (zh) 媒体播放的加载控制方法、装置及存储介质
US11025991B2 (en) Webpage playing method and device and storage medium for non-streaming media file
CN110545460A (zh) 一种媒体文件的预加载方法、装置及存储介质
JP7174762B2 (ja) メタデータ容器の解析方法、装置及び記憶媒体
CN110545471B (zh) 基于离线转换的播放控制方法、装置及存储介质
CN110545463B (zh) 基于媒体文件转换的播放控制方法、装置及存储介质
CN110545461A (zh) 一种媒体文件的分辨率切换方法、装置及存储介质
CN110545467B (zh) 一种媒体文件加载控制方法、装置及存储介质
CN110545464A (zh) 一种媒体文件分辨率切换方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18920199

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020552076

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18/03/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18920199

Country of ref document: EP

Kind code of ref document: A1