WO2011049413A2

WO2011049413A2 - Multimedia content processing method and apparatus

Info

Publication number: WO2011049413A2
Application number: PCT/KR2010/007324
Authority: WO
Inventors: 장의선; 김현규
Original assignee: 한양대학교 산학협력단
Priority date: 2009-10-23
Filing date: 2010-10-25
Publication date: 2011-04-28
Also published as: WO2011049413A3; KR20110044726A; KR101182851B1

Abstract

Disclosed is a multimedia content processing device for one or more types of multimedia content, which identifies the types of identification information in a storage unit which stores identification information, and the types of multimedia content received from applications which reference different types of identification information. The media processing apparatus comprises a media identification unit which summons a specified API (Application Programming Interface) depending on type of media content, and at least one API to handle the summoned multimedia content.

Description

Method and apparatus for processing multimedia content

The present invention relates to a technique for processing multimedia content, and more particularly, to a method of encoding or decoding multimedia content and providing the same to an application.

The ISO / IEC 23006 standard (MPEG Extensible Middleware, MXM) is a standard for defining intermediary devices to enable faster implementation of a variety of multimedia-based applications.

The MXM standard is a technology that provides a number of multimedia processing engines and makes it easy to encode or decode various multimedia contents in an application by calling an API (application-programming interface) included in the engine. This makes MXM easier and faster to implement the various multimedia applications that exist in the multimedia value chain.

However, the multimedia processing engine defined by the MXM standard provides a module that performs a plurality of encodings and decodings. The process of identifying the multimedia content to be encoded or decoded and calling an API appropriate for the identified multimedia content can be performed by the application. It should be done. Accordingly, the creator of an application had to directly implement a process of identifying multimedia content and selecting an API to be used in the application.

An object of the present invention is to provide a multimedia content processing apparatus having a media frame engine capable of identifying multimedia content.

According to an aspect of the present invention, the identification information storage unit for storing the type identification information for identifying the type of one or more multimedia content with reference to the type identification information to identify the type of multimedia content received from the application, the multimedia content There is provided a multimedia content processing apparatus including a media identification unit for calling a specified API (Application Programming Interface) and a media processing unit for providing one or more APIs for processing the multimedia content.

The media identification unit extracts type identification information from the multimedia content according to any one or more designated parsing methods, and selects a type corresponding to the same type identification information as the type identification information among one or more designated type identification information. Can be identified by type.

When one of the type identification information and the type identification information is the same, the media identification unit may call an API corresponding to the same type identification information as the type identification information.

If the same type identification information as the type identification information does not exist, the media identification unit re-extracts the type identification information from the multimedia content according to a parsing method other than the parsing method previously performed among one or more designated parsing methods. Can be.

When the type identification information equal to the type identification information indicates a container bitstream, the media identification unit extracts one or more element bitstreams from the multimedia content, identifies each type of the element bitstream, The API may be called according to the type of each element bitstream.

According to another aspect of the present invention, a method for processing multimedia content in a multimedia content processing apparatus, the method comprising: receiving multimedia content from an application, identifying the type of the multimedia content; Programming interface), encoding or decoding the multimedia content through the API, generating output data, and transmitting the output data to the application.

The identifying of the type of the multimedia content may include extracting type identification information from the multimedia content according to any one or more designated parsing methods, and determining whether the type identification information is the same as any of the specified type identification information. It may include.

Calling an API designated according to the type of the multimedia content may call an API corresponding to the same type identification information as the type identification information when any one of the type identification information and the type identification information is the same. have.

The identifying of the type of the multimedia content may further include reselecting a parsing method other than the parsing method previously performed among one or more designated parsing methods when the same type identification information as the type identification information does not exist. After the step of reselecting the parsing method, the process from the step of extracting the type identification information may be performed again.

If the type identification information equal to the type identification information indicates a container bitstream, further comprising extracting one or more element bitstreams from the multimedia content, wherein each step after extracting the element bitstream Extracting the type identification information for the element bitstream or transmitting the output data to the application may be performed.

1 is a diagram schematically illustrating a configuration of a multimedia content processing apparatus according to the MXM standard.

2 is a view for explaining a process according to the multimedia content transmission between the multimedia content processing apparatus according to the MXM standard.

3 is a block diagram schematically illustrating a configuration of a media framework engine included in a multimedia content processing apparatus according to an embodiment of the present invention.

4 is a diagram illustrating a process of determining a type of multimedia content by using the type identification information stored in the identification information storage unit in the media identification unit of the multimedia content processing apparatus according to an embodiment of the present invention.

5 is a block diagram schematically illustrating a configuration of a media framework engine included in a multimedia content processing device according to an embodiment of the present invention.

6 is a flowchart illustrating a process of a multimedia content processing apparatus processing multimedia content.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to FIG. 1, a multimedia content processing apparatus includes a processor 101, a memory 102, and an operating system 103. The multimedia content processing apparatus includes a multimedia content engine 105 (Orchestrator Engine, MPEG21 File Engine, Digital Item Engine, REL Engine, IPMP Engine, Security Engine, Content Metadata Engine, Media Framework) according to the MXM standard. Engine, Content Search Engine, etc.) may be installed, and the API included in the engine for content may be provided according to a call of the application 110 that is resident on the memory 102 and executed by the processor 101. For example, an application receives a bitstream encoded according to the MPEG-4 VIDEO standard from a user and calls an API included in the Media Framework Engine to decode the bitstream. The application generates and plays the video data obtained by decoding the bitstream using the called API. The process of decoding and playing the above-described bitstream is performed by an application, and the API for decoding the bitstream is determined according to the application.

FIG. 2 is a diagram for describing a process of transmitting multimedia content between multimedia content processing apparatuses according to the MXM standard.

The first multimedia content processing device 210 having the above-described configuration with reference to FIG. 1 may be configured according to an encoding of video data or an arbitrary standard through the multimedia content engine 215 defined in the MXM standard by the first application 213. The decoding of the encoded bitstream may be performed. The second multimedia content processing device 220 may perform encoding of the video data through the multimedia content engine 225 defined in the MXM standard or decoding the bitstream encoded according to any standard by the second application 223. Can be. In this case, the first multimedia content processing device and the second multimedia content processing device may be different types of platforms, and thus, each

operating system

216 and 226 may be different operating systems.

Accordingly, the multimedia contents processing apparatuses according to the above-mentioned MXM standard can perform encoding and decoding using multimedia technology related standard technology of the same method even if the devices of different platforms are used. The device may prevent the encoding and decoding schemes of the bitstream such that the decoding cannot be performed are not matched.

3 is a block diagram schematically illustrating a configuration of a media framework engine included in a multimedia content processing apparatus according to an embodiment of the present invention, and FIG. 4 is a media identification unit of the multimedia content processing apparatus according to an embodiment of the present invention. Is a diagram illustrating a process of determining the type of multimedia content by using the type identification information stored in the identification information storage unit.

The media framework engine 300 includes a media identification unit 310 and a media processing unit 330.

The media identification unit 310 provides an API (hereinafter, referred to as an application call function) that is called from the application 110. The media identifier 310 receives the multimedia content from the application 110 through an application call function. The media identification unit 310 analyzes the header of the multimedia content to identify the type of the multimedia content.

The identification information storage unit 320 may identify identification information (hereinafter referred to as type identification information) for identifying the type of multimedia content according to a standard that may be processed by the media processing unit 330, and a bitstream of a type corresponding to each type identification information. Stores identification information (hereinafter referred to as interface identification information) of an API for processing. In this case, the identification information storage unit 320 may further store information indicating syntax parsing rules for extracting type identification information from multimedia content other than the identification information (hereinafter referred to as syntax information).

The media identification unit 310 identifies the type of multimedia content by referring to the type identification information stored in the identification information storage unit 320. For example, referring to FIG. 4, the media identification unit 310 may extract arbitrary syntax information from the identification information storage unit 320. The media identification unit 310 may extract data (hereinafter referred to as type identification information) for identifying the type of the multimedia content by comparing the type identification information with the multimedia content according to the syntax information. That is, if the multimedia content is a bitstream encoded according to the MPEG4 standard, and the syntax information indicates a syntax parsing rule for extracting the first 32 bits of the multimedia content, the media identifier 310 identifies the type of the first 32 bits from the multimedia content. Extract information ("00 00 01 B0"). The media identification unit 310 checks whether the same type identification information 410 as the type identification information 405 is stored in the identification information storage unit 320. The media identification unit 310 confirms that the extracted type identification information is the same as the type identification information corresponding to MPEG-4, and identifies the interface identification information corresponding to the type identification information (ie, decoding the MPEG-4 bitstream. Interface identification information corresponding to the API) is extracted from the identification information storage unit 320. The media identification unit 310 transmits the multimedia content to the media processing unit 330 through an API corresponding to the interface identification information.

As described above, the media identifier 310 extracts data from the bitstream according to the syntax information. In this case, the identification information storage unit 320 may store a plurality of syntax information. The media identifier 310 may sequentially extract one of the plurality of syntax information. Accordingly, the media identification unit 310 extracts the data according to any one of the plurality of syntax information and identifies the type until it confirms that the same type identification information as the data extracted according to the syntax information is stored in the identification information storage unit 320. The process of checking the information can be repeated. For example, there is a standard in which data other than the first 32 bits is used as the type identification information. When the media identification unit 310 fails to check the type of the multimedia content according to the syntax information extracting the first 32 bits of the multimedia content, the media identification unit 310 stores the data in the multimedia content according to the syntax information stored in the identification information storage unit 320. The extracted and extracted data may be used to identify the type of multimedia content.

The media processor 330 provides an API required in the process of encoding or decoding the multimedia content. In this case, the API provided by the media processing unit 330 may be an API for encoding and decoding of a standard as shown in Table 1 below. At this time, it is apparent that the API provided by the media processing unit 330 may additionally provide other APIs in addition to APIs for encoding and decoding of the following standard.

Table 1

Standard	Profile or Technology
MPEG-1 Audio	layer II
MPEG-1 Audio	layer III
MPEG-4 AAC
MPEG-4 HE-AAC
MPEG-1 Video
MPEG-2 Video	Main profile
MPEG-4 Visual	Simple Profile
	Advanced Simple Profile
MPEG-4 AVC
Codec Configuration Representation
Video tool library
MPEG-1 Systems
MPEG-2 Systems	Transport stream
	Program Stream
MPEG-4 M4Mux
ISO Base Media File Format
Geometry
Appearance
Texture
BBA Animation
FAMC Animation
LASeR	Core
BIFS

The media processor 330 encodes or decodes the multimedia content by performing the above-described API. The media processor 330 transmits data (hereinafter, referred to as output data) obtained by encoding or decoding multimedia content to the media identifier 310 through the API.

The media identification unit 310 transmits output data received from the media processing unit 330 to the application 110.

Up to now, a process of encoding or decoding multimedia content according to a standard by a multimedia content processing apparatus has been described.

The multimedia content may be data in a format according to one standard as described above, but may be a container bitstream in which bitstreams according to each standard are merged. The container bitstream may include an element bitstream for each standard, such as a standard for encoding video data and a standard for encoding audio data. Accordingly, in order to decode the container bitstream, it is necessary to extract each element bitstream from the container bitstream.

Hereinafter, a process of processing a container bitstream by the multimedia content processing apparatus will be described with reference to FIG. 5. The following container bitstream is assumed to include a video element bitstream for video data and an audio element bitstream for audio data.

The media identifier 310 receives a container bitstream from the application 110 through an identification function. The media identification unit 310 extracts the type identification information from the container bitstream by using the syntax information stored in the identification information storage unit 320 and confirms the same type identification information as the type identification information. The media identification unit 310 confirms that the same type identification information as the type identification information indicates the container bitstream. The media identifier 310 extracts an element bitstream (video element bitstream and audio element bitstream) included in the container bitstream. The media identification unit 310 extracts the type identification information for each of the video element bitstream and the audio element bitstream to confirm the same type identification information, and identifies the interface identification information corresponding to each type identification information. ). The media identifier 310 transmits each element bitstream to the API corresponding to the extracted interface identification information. That is, when the type identification information corresponding to the video element bitstream indicates the MPEG-4 VISUAL standard, the media identifier 310 transmits the video element bitstream to the API 510 of the MPEG-4 VISUAL standard. In addition, when the type identification information corresponding to the audio element bitstream indicates the MPEG AUDIO standard, the media identification unit 310 transmits the audio element bitstream to the API 520 of the MPEG AUDIO standard. The media function 220 decodes each of the video element bitstream and the audio element bitstream received by each

API

510 or 520 and transmits the video data and the audio data to the media identifier 310. The media identifier 310 transmits the video data and the audio data to the application 110.

6 is a flowchart illustrating a process of processing a multimedia content by the multimedia content processing device. In this case, as described above with reference to FIGS. 1 to 5, the multimedia content processing apparatus may store, in advance, type identification information corresponding to various standards and interface identification information corresponding to each type identification information.

Referring to FIG. 6, in operation 610, the multimedia content processing apparatus receives multimedia content from an application.

In operation 620, the multimedia content processing apparatus may extract the type identification information by parsing the multimedia content according to the previously stored syntax information.

In operation 630, the multimedia content processing apparatus determines whether the same type identification information as the type identification information exists among the previously stored type identification information.

If the same type identification information as the type identification information does not exist in operation 630, in operation 640, the multimedia content processing apparatus selects the next syntax information and performs a process from operation 620 using the selected syntax information.

If the same type identification information as the type identification information exists in step 630, the multimedia content processing apparatus transmits the multimedia content to the API according to the interface identification information corresponding to the type identification information in step 650.

In operation 660, the multimedia content processing apparatus generates an output data by performing a process of encoding or decoding the multimedia content received through the API. In this case, the output data may be a bitstream in which the raw data is encoded when the multimedia content is raw data such as an original video. The output data may be raw data calculated by decoding the multimedia content when the multimedia content is bitstream encoded according to an arbitrary standard.

In operation 670, the multimedia content processing apparatus transmits the output data to the application.

So far I looked at the center of the present invention with respect to the embodiment. Many embodiments other than the above-described embodiments are within the claims of the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

According to the present invention, there is an advantage that can automatically identify and process multimedia content.

Claims

Identification information storage unit for storing the type identification information to identify the type of one or more multimedia content

A media identification unit for identifying a type of multimedia content received from an application with reference to the type identification information and calling an API (Application Programming Interface) designated according to the type of the multimedia content;

And a media processing unit for providing one or more APIs for processing the multimedia contents.
According to claim 1,

The media identification unit

Extracting type identification information from the multimedia content according to any one or more specified parsing methods,

And identifying a type corresponding to the same type identification information as the type identification information among one or more designated type identification information as the type of the multimedia content.
The method of claim 2,

The media identification unit

And when one of the type identification information and the type identification information is the same, calling the API corresponding to the same type identification information as the type identification information.
The method of claim 2,

The media identification unit

If the same type identification information as the type identification information does not exist, the type identification information is re-extracted from the multimedia content according to a parsing method other than the parsing method previously performed among one or more designated parsing methods. Multimedia content processing device.
According to claim 1,

The media identification unit extracts one or more element bitstreams from the multimedia content when the type identification information identical to the type identification information indicates a container bitstream.

Identify the type of each said element bitstream,

And calling the API according to the type of each element bitstream.
In the multimedia content processing apparatus for processing multimedia content,

Receiving multimedia content from an application

Identifying the type of multimedia content

Calling an application programming interface (API) designated according to the type of the multimedia content;

Generating output data by encoding or decoding the multimedia content through the API; and

And transmitting the output data to the application.
The method of claim 6,

Identifying the type of multimedia content

Extracting type identification information from the multimedia content according to any one or more designated parsing methods;

And determining whether the type identification information is the same as any of the specified type identification information.
The method of claim 7, wherein

Calling the specified API according to the type of the multimedia content

And if one of the type identification information and the type identification information is the same, calling an API corresponding to the same type identification information as the type identification information.
The method of claim 7, wherein

Identifying the type of multimedia content

Re-selecting a parsing method other than the parsing method previously performed, among the one or more designated parsing methods, when the same type identification information as the type identification information does not exist;

And after the reselecting of the parsing method, the process from the step of extracting the type identification information is performed again.
The method of claim 7, wherein

If the type identification information identical to the type identification information indicates a container bitstream, extracting one or more element bitstreams from the multimedia content;

Extracting the type identification information for each of the element bitstreams and transmitting the output data to the application after extracting the element bitstream.