JP2002077855A - System and method for processing multimedia information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To receive and decode encoded bit stream information transmitted from various different digital systems by networked digital system. SOLUTION: A method for processing multimedia information comprises the steps of accessing to a part of the encoded multimedia information from a database for storing the encoded multimedia information in response to a request signal from a client, accessing to contents information for describing encoding formatting of a part of the encoded multimedia information and client's decoding format, and automatically transcodes the part of the encoded multimedia information to generate paste transcode multimedia information to client information interchangeable with the client's decoding format by the contents and the transcode multimedia information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

FIELD OF THE INVENTION The present invention relates to the field of multimedia electronic systems, and more particularly to storage media and / or storage media.
Alternatively, the present invention relates to a method of decoding an encoded bit stream generated from an electronic network and encoding and decoding multimedia information and transmission comprising a system for displaying multimedia content.

[0002]

2. Description of the Related Art Audio / visual (AV) materials are increasingly being accumulated by using digital data.
Transmission is becoming possible. The digital video display of AV material facilitates its use with computer control electronics and also facilitates high quality images and sound reproduction. Digital AV material
Digital data is typically compressed ("encoded") to reduce the computer resources required to store and transmit it. Systems for sending multimedia content encode and / or compress the content to make efficient use of their transmission channels. This is because multimedia content, especially video, is very large in size. Digital AV material can be encoded using many well-known standard methods.
For example, DV (Digital Video)
Standard format, MPEG (Movie Expert Group) standard format,
JPEG standard, H.264 H.261 standard, H.264. 263 standard system and the motion JPEG standard system.
The coding standard also specifies the associated decoding process. Multimedia content is typically stored on a storage medium and sent as a bitstream.

[0003] Compression H. 261 and H.E. The H.263 standard is suitable for audio and visual data.
These standard systems are based on ITU-T (International Telecommunication Union)
Described by H. While the H.261 standard is used for TV conference systems, The H.263 standard is used for mobile communication.

[0004] H. 261 and H.E. The H.263 standard employs hybrid coding of motion compensation (MC) and discrete cosine transform (DCT). For more information on these specifications,
This is explained in the well-known ITU-T recommendation.

[0005] JPEG is a compression standard used for still images. It is ISO-IEC / JTC1
/ SC29 / WG1 document. JPE
The G standard uses the transform coding (DCT) used in the JPEG standard. WG1 is currently JPEG
2000, which is a more efficient coding technique for still image processing. The JPEG2000 standard uses wavelet transform. Motion JPE
G is a de facto standard for video processing.
In motion JPEG, individual frames of a video are encoded by JPEG. It is used in the digital video (DV) standard, which is used on the market today. Frame correlation is not used for compression.

[0006] MPEG is a compression standard for audio, video, and graphics information, and includes, for example, MPEG 1, 2, 4, and 7. It is ISO-
It is standardized in the IEC / JTC1 / SC29 / WG11 document. MPEG1 is a standard method for encoding audio and video data for storage devices in a CD-ROM (Compact Disc Read Only Memory) device. The MPEG1 standard is IS-11393.
Described in a standard way.

[0007] MPEG2 is a storage medium such as a DVD.
(Digital Video Disk) and a standard method for encoding / decoding audio / video data for transmission in digital broadcasting. Also M
PEG2 supports interlaced video that MPEG1 does not support. Therefore, MPEG2
Used for high quality video displaying on television equipment. The MPEG2 standard is described in IS-13818.

The MPEG4 standard is used to decode and encode the sending audio, video and computer graphics data. It supports content-based bitstream manipulation and representation. The standard is described in IS14496.

[0009] MPEG7 is a standard system for meta information of multimedia (MM) contents. The metadata in the MM content describes the data or allows the identification of related information, such as other authors, creator information, directors, actors, and the like.

[0010] The MPEG7 standard is still in the process of being standardized and is available as a draft. The draft specification is ISO-IEC-JTC1 / SC29 / WG1
Explained in one document.

In the past, multimedia systems were closed applications. One example of an existing closed multimedia information retrieval system is a DVD system. Audio / visual content is MP
It is encoded by EG2 and stored on a DVD storage medium (disk). A DVD player reads a bitstream from a DVD disc, decodes the bitstream, and then displays the associated decoded content on a television. The DVD player is "closed". Therefore, the decoder circuit in the DVD unit always
Since all content on a DVD disc is encoded using a single compression technique, one decoding technique sets a unique need. For example, DV
D players are only required to decode bitstreams encoded using DVD.

It is also necessary and sufficient to have a well-developed STB to decode a group of bit streams that use a predetermined encoding technique to be decoded by a set-top box (STB). . If, for example, the STB could not decode it during the decoding process (eg, it was encoded using a different compression technique), the received information of the STB might be corrupted. Thus, in a known client system, all bitstreams are encoded in a closed application to ensure decoding according to a known decoding format.

Recently, many companies have created a wide variety of digital multimedia systems using various compression techniques, but to date, digital AV multimedia systems have been used only for professional products. Was.

Advances in LSI and signal processing techniques have recently reduced the cost of digital systems, and as a result, AV multimedia systems have been used in consumer products. Examples of such digital consumer products are Video CD, DVD, DV (Digital Video),
Such as digital cameras.

[0015] Digital computer technology and digital network technology have also advanced significantly over the last decade. Computers are interconnected by a digital communication network and can share network resources and information. The Internet has become very popular,
It is now an indispensable tool for many business and home users. Users can access various types of information and services through the Internet. In business, intranets are very popular. With an intranet, all computers in an office are connected to each other, and their users can share information and resources located in the same office.

[0016] These network technologies not only allow information or resources and also possible access, but also distribute the computing power to some terminals of the network. Distributed systems are very useful for terminals that have limited computing power. This is because it is possible to reduce the power of computer processing required to execute some applications.

[0017] Network technology has also been introduced into the consumer electronics field. As many companies develop products for this communication standard, the standardization of high-speed networks (such as the serial communication standard IEEE 1394) has been very successful. In view of the explosive use of the growth of consumer electronics connected to each other via the Internet, intranets, and IEEE 1394 communication standards, many digital systems are now retrieving information from a variety of different digital systems. You can receive.

[0018]

When a server receives digital information, it sends the information in the form of a digital bit stream. But the sender (server)
May cause a problem that the encoding format used by the receiver does not match the decoding format used by the receiver (client).

A case arises in which the client may not be able to decode the requested multimedia content sent by the sender. What is desired is an efficient system that allows a networked digital system to receive and decode encoded bitstream information sent from a variety of different digital systems.

[0020]

SUMMARY OF THE INVENTION Accordingly, in accordance with the present invention, a method and system are provided wherein a networked digital system receives encoded bitstream information sent from a variety of different digital systems. Provided to allow decoding.

The present invention automatically converts (transcodes) coded bit stream information from one format to another, and provides a client (receiver)
Include a format converter capable of displaying the received information. The format converter of the present invention also automatically allocates the necessary computer resources to convert to the appropriate format.

The multimedia information retrieval system and method include a method and system for automatic format conversion. The invention includes a data structure ("content information") associated with each requested multimedia bitstream. The data structure identifies the encoding format, for example, the compression technique used in the encoded multimedia bitstream that can be initiated by the content server. The automatic format conversion process then queries information from the client system (requester) and also receives a data structure identifying the encoding format. The client information identifies a decoding format compatible with the client.

Automatic format conversion then determines the transcoding process required to convert the bitstream from its encoded format to a format recognized by the client system. Thus, in order to save storage time and computer resources before transcoding is performed, the format conversion process of the present invention also determines whether decoding is needed in those cases where decoding is not needed.

Further, the format conversion process automatically determines the computer memory (frame memory) size required to execute the transcoding process efficiently using the computer memory resources. The format converter is implemented in the application as software and integrated into the data access server.

The data access server can be integrated in the client system or in the content server. The inventive format converter is particularly useful for electronic devices connected by a communication network where the encoding format of the sender may not be compatible with the decoding format of the receiver, thus requiring transcoding between formats. And

According to one embodiment of the present invention, a method of processing multimedia information includes a method of processing encoded multimedia information from a database storing the encoded multimedia information in response to a request signal from a client. Accessing multimedia information to access a part thereof, and client information accessing information describing client information describing a coding format of a part of the encoded multimedia information and a decoding format of the client Accessing and automatically transcoding the encoded multimedia information according to the content and the client information to generate transcoded multimedia information in which the transcoded multimedia information is compatible with the decoding format of the client. Tiger And a scan code step.

Further, the transcoding step of the embodiment includes:
Automatically determining, based on the content and the client information, whether the encoded multimedia information needs to be decoded, and, when decoding is required, extracting a portion of the encoded multimedia information. Determining the automatic memory resources required for decoding, and, if decoding is necessary, decoding and generating the multimedia information by decoding a portion of the encoded multimedia information; And transcoding the decoded multimedia information into transcoded multimedia information according to the content and the client information.

[0028]

DETAILED DESCRIPTION OF THE INVENTION The multimedia information retrieval system and method of the present invention, including the method and system for automatic format conversion, are described in the following detailed description.
Provide complete understanding.

An embodiment of the present invention is associated with an audiovisual encoder for encoding, such as a transfer encoder, a transcoding method, and comprises a storage system and a decoder including the storage method, for decoding an information retrieval system.

The present invention is a digital system for decoding and displaying a coded bitstream arriving from a storage medium and / or a network communication system. Related to multimedia content. Examples of these systems are:
Includes TV conferencing systems, videophone systems, broadcasted systems, and multimedia database systems. A terminal (also called a client system) accesses multimedia content from a network and / or storage medium, plays the decrypted content, and / or edits the information. The invention includes a transcoding device, for example, a data access manager located between the content server and the client system to facilitate automatic format conversion.

FIG. 1 illustrates an embodiment of a multimedia content access system 200 of the present invention. System 200 includes a multimedia content server (“server”) 210 that is a sender of multimedia (MM) information in the form of a digital bitstream. Client system (client) 230
Or is the recipient of the bitstream, and also requests the multimedia content server 210 for the bitstream. Clients 230 include electronic devices such as, for example, set-top boxes, CDs, DVDs, computer systems, television units, VCRs, and the like.

The data access server 220 of the present invention is configured between the client 230 and the server 210. The data access server 220 can also be integrated within the client 230 or, alternatively,
Can be integrated within 0. The data access server 220 has a communication channel (network or transmission channel) 24
0b, and connected to the client 230 via a communication channel (network or transmission channel) 240a. Many well-known digital communication networks and / or standard schemes
It can be used to implement channels 240a and 240b according to the present invention. In one embodiment, the communication channel is the Internet.

In another embodiment, the communication channel is IEEE
This is the E1394 serial communication standard system.

Client information and content request 242
Is the data access server 220 from the client 230
Is transmitted to The MM content (for example, a bit stream) 244 is transmitted from the data access server 220 to the client 230. The MM contents 244 are transcoded contents. The content information request and the content request 248 are transmitted from the data access server 220 to the server 210, and are initiated from the client 230 in response to the request.

Content information and MM content 246
Is transmitted from the server 210 to the data access server 220. The MM content 246 is an encoded bit stream.

The function of the data access server 220 of the present invention is to convert the encoded bit stream 246 into a transcoded version 2
44 format.

The multimedia contents are stored in the multimedia contents server 210. The content is stored as column data or a bit stream and is well known in a format standard, for example MPE
G, JPEG, H. 263; 261, Motion J
It can be formatted with PEG, video CD, DVD, DV (digital video), digital camera, and client 230 is the terminal of multimedia content access system 200. The user accesses the content from the client 230.

The client 230 sends a content request 1 signal 242, which is a request for certain content, for example, a particular image, audio program, or title.

Another example of a content request 1 signal is a movie title. In one embodiment, for example, MPEG
In the 7 standard system, the content request 1 signal semantic information is described in the content. Client 230 may include information described by the client system (including client resources), such as decoding formats supported by client 230 (eg, compatible with it), client information signal 242, memory size, display size, computer processing. Transfer power, buffer size, etc.

The data access server 220 of FIG. 1 receives a content request 1 signal and a client information signal 242 from the client 230. The data access server 220 sends a content information request signal 248 to the server 210, which is a request for content information about the requested multimedia bitstream.

The data access server 220 also sends a content request 2 signal 248, which is a request for MM content (bit stream), to the server 210. The server 210 is a multimedia server, for example, both a bitstream form and a database of information describing their contents, such as titles, directors called "content information", information about authors, authors, actors, etc. Store content.

The server 210 receives the content information request signal, and returns a content information 246 signal to the data access server 220. The content information signal includes information describing the MM content stored in server 210. An example of a content information signal is a file name. The content information signal is the meaning and syntax information of the content (a pointer to an appropriate place such as a file name or a bit stream) described in the MPEG7 standard.

The content information also includes various information about the encoding format used by its associated MM content (as exemplified in Tables 1, 2 and 3).

The data access server 220 receives the content request 1 signal, and finds the appropriate content in the server 210 in an appropriate format. The data access server 220 then sends the content request 2 signal 2
Send 48, which requests the appropriate MM content (bitstream) from the server 210. Content request 2
An example of a signal is a file name. The content request 2 signal is syntax information of content (such as a pointer to an appropriate location such as a file name or a bit stream) described in MPEG7.

The server 210 sends the requested MM content 246 to the data access server 220. The data access server 220 receives the content information signal and the MM content from the server 210. The data access server 220 converts (transcodes) the MM content into a format suitable for the client 230 using the client information signal and the content information signal. The data access server 220 transmits the converted (transcoded) MM content 244 to the client 23.
Send to 0.

In FIG. 1, since the data is independent of the server 210 and the client 230, it accesses the server 220 which processes the format conversion of the MM contents. However, it is possible to implement the module of the data access server 220 of the server 210 or the client 230.

FIG. 2 illustrates an embodiment 210a of the multimedia content server 210 of FIG. Content information and other metadata (“metadata”) are stored in metadata storage 320. Multimedia
The content is a multimedia content storage device 33
Stored at zero. The storage devices 320 and 330 can be implemented using a suitable digital storage medium such as a digital memory or a hard disk drive. The content information is metadata and includes not only the other information described in Tables 1 to 3 but also information about the meaning of the related MM content.

When the encoded bit stream making up the movie is MM content, examples of the meaning information are the title and director of the movie. Examples of syntax information are bitstream file names, URL locators, and appropriate locations. An example of content information is
This is metadata described in the MPEG7 standard. As further described below, the content information is:
It also contains specific information used to determine the encoding technique used for the MM content.

The MM content, for example, digital bit stream data, has a well-known format,
For example, MPEG, H.264. 261, H .; 263, motion JPEG, DV, DVD, CD, etc., and are stored in the multimedia content storage device 330 of FIG. The multimedia content server 210a of FIG. 2 also includes a metadata manager 310 and an MM content manager 325. The content information request signal 248a is received from a data access server 220 and input to a metadata manager 310 that processes content information stored in a metadata storage 320.

The metadata manager 310 sends a content information request signal to the metadata storage 320 to query the storage 320 for the specified content information as required by the client 230.
The metadata storage 320 sends the appropriate content information to the metadata manager 310. At that time, the metadata manager 310 sends a content information signal 246a to the data access server 220.

The link 325 is used to associate the metadata content of the storage device 320 with its corresponding bit stream information of the storage device 330 and, as appropriate, synchronizing the transmission of the information.

The content request 2 signal 248 b is transmitted to the multimedia content manager 335 which processes the multimedia content in the multimedia content storage device 330 by the data access server 220.
And input. Multimedia content manager 335 sends a content request 2 signal to multimedia content storage 330 to identify a particular bitstream for transmission.

Multimedia content storage device 33
0 sends the appropriate encoded MM content to the multimedia content manager 335. then,
The multimedia content manager 335 sends the specified MM content 246b to the data access server 220.

M from the MM content manager 335
M content 246b is in an encoded form.

FIG. 3 shows the data access server 2 of the present invention.
20 shows the twenty embodiment 220a. The server 220a comprises a device (or process) 350 and a transcode manager 340 that encodes a transcode tool library 360. The client information signal 242a received from the client 230 is an input to the transcode manager 340 and
Informs the transcode manager 340 of the decoding standard (output format) supported by

Multimedia content server 210
a is received from transcode manager 340 and the associated M
It contains information about the encoding standard (input format) used by the M content 246b.

From now on, the transcode manager 34
0 produces an input / output format signal 362 that informs the transcoding device of the input and output formats involved.

The transcode manager 340 of the present invention
Determines the output format of the MM content using the client information signal 242a, and determines the input format of the MM content using the content information 246a of the present invention.

The transcode manager 340 determines where to convert the input MM content 246b to a converted bit stream 244b.

The transcode manager 340 forwards the transfer to the transcoding device 350
4 is coded. The transcode type information signal 364 specifies the MM content and the output format of the transcoding process to generate the converted bit stream 244b. The process used to determine transcode type information 364 is described further below.

The transcode manager 340 of FIG.
Also sends an available and content information signal 244a to the client 230 (FIG. 1). Transcode manager 340 sets available signal 244a to "0" if there is no content at multimedia content server 210a or no content encounters the specified client request. Transcode manager 340 sets available signal 244a to "1" if multimedia content server 210a has content.

The transcoding device 350 converts the format of the MM content 246b (based on the transcode type information signal 364) to generate a converted bit stream 244b to be transferred to the client 230.

An embodiment 210b of the multimedia content server 210 of FIG. 1 will be described with reference to FIG.

The embodiment 210b includes a multiplexer 305
The configuration is the same as that of the embodiment 210a (FIG. 2) except for the configuration and the demultiplexer 307. Multiplexer 305 concurrently transmits content information signal 246a over a single communication channel, indicated as 250, by MM content bitstream 246b. In this case, a single communication channel transmits information from the multimedia content server 200b to the data access server 220b.
Used to communicate to

On the other hand, the demultiplexer 307 functions on the content request 2 signal 248b and the content information request signal 248a from the single communication channel 260 to be demultiplexed. In this case, a single communication channel is used to convey information from data access server 220 to multimedia content server 200b.

According to FIG. 5, the data access server 2 of FIG.
Twentieth embodiment 220b will be described.

Embodiment 220a except for multiplexers 372 and 376 and demultiplexers 374 and 378.
(FIG. 2) is similar to embodiment 220b. Multiplexer 372 simultaneously transmits content information request signal 248a and content request 2 signal 248b (not shown in FIG. 5) over a single communication channel 260. The demultiplexer 374 outputs the content information signal 2
46a and the MM content signal 246b from a single communication channel 250 are demultiplexed.

Further, the multiplexer 376 includes an available, content information signal 244a, and a converted bit stream signal 244 over a single communication channel 270.
b is transmitted simultaneously. In this case, a single communication channel 270 is used to communicate information from the data access server 220b to the client system 230 (FIG. 1). Demultiplexer 378 demultiplexes client information signal 242a from communication channel 280 (connected to client system 230).

In this case, a single communication channel 280 is used to communicate information from client system 230 (FIG. 1) to data access server 220b.

FIG. 6 illustrates one embodiment of a transcoding device 350.

In FIG. 6, transcoding device 350
Transcoding tools, e.g.
All connections are implemented. In this embodiment, transcoding device 350 operates as switch 410 to select one of tools 412-418 to perform the appropriate transcoding function based on the transcoding type information signal. Tools 412-418
Although it is planned to perform a predetermined conversion process and can be implemented using hardware in general, software is also used.

Switch 410 operates to provide an input MM content signal to one of inputs 412a-418a 246b. Therefore, only the selected one of the transcoding tools 412-418 is active at any given time, and therefore their output 4
12b-418b can be connected to provide a single transcode output 244b.

For example, if the input bit stream 246b of the MPEG2 format and the output bit stream format needs to be in the DV format, the input MPEG2 bit stream is converted to a “MPEG2 to DV” converter module 414.
Is input to

The "MPEG2 to DV" converter module therefore converts to a bit stream in DV format, and the bit stream 244b is output to the client 230 of FIG.

The individual tools 412-418 need to perform the transcoding process for any required decoding.

A transcoding process is not necessarily required for decoding, for example a DV to MPEG2 tool is not required to decode the input bitstream.

FIGS. 3 and 5 illustrate a more general embodiment of a transcoding device 350 implemented using software tools, rather than predetermined connections as illustrated in FIG.

In these embodiments, transcoding device 350 is generally a processor of a computer system (112 in FIG. 11) or a digital signal processor (D
SP).

In this embodiment, transcoding device 350 uses a suitable transcoding tool stored in transcoding tool library 360 implemented in a computer readable storage element of computer system 112. All expected format connections are made to use the transcode tool library 360. Transcode type information signal 364
, Based on the information from
M content 246b and converted bit stream 244b
A tool request signal 366 (based on the transcode type information 364) indicating the software tools needed to perform the appropriate changes during.

A different software tool is used for each different connection of the input / output format expected to be applied to the transcoding process performed by the transcoding device 350. The transcoding tool library 360 then sends the selected transcoding tool 368, which is a software module, back to the transcoding device 350. Transcoding device 3
50 then allocates the appropriate resources (such as the size of memory, etc.) in system 112 (FIG. 11) to support the selected transcoding tool 368. It is noted that transcode type information 364 indicates whether transcoding manager 340 needs to decode MM content 246b before transcoding can function.

In order to determine whether or not decoding is possible, the transcoding device 35 is programmed once with an appropriate transcoding tool, and is set with appropriate memory resources.
0 performs a transcoding process to generate a converted bit stream 244b.

Tables 1, 2 and 3 below illustrate examples of content information signals 246a associated with individual MM content 246b bitstreams. The content information described below initially comprises its complete bitstream 24
6b without decoding the encoded bitstream (MM
Satisfy 246b). By providing information via the content information 246a, the present invention does not always require decoding on the MM content 246b before transcoding works, but achieves a very efficient transcoding. Allow code process.

[0083]

[0085]

According to the present invention, data access server 2
20 is used to perform automatic format conversion so that the content server can send information to the client without requiring compatibility with the decoding techniques supported by the client.

In order for an automatic format conversion to be performed according to the present invention, the dispatch application identifies the encoding format used in the bitstream.

The descriptors shown in Table 1 are for identifying the application used to encode the MM content bit stream 246b.

The ApplicationDescriptor of the present invention includes an application identifier, and specifies the application that encoded the bit stream. This information is used so that the decoder can identify whether or not decoding is possible. Table 1 shows an example of an application descriptor including an ApplicationID field and a SizeofServiceInfo field. Ap
plicationID is the encoding application, eg D
This is a flag used to identify V, MPEG2, DVD, or the like. SizeofServiceInfo is a flag used to specify the maximum size (in bytes) of service information found in the user data. It is generally multiplexed in the bitstream,
Used for buffer management. For example, if the user data is large, the decoder must break.
Thus, the decoder can use the fields to check the maximum size of the user data.

Table 2 shows the MaxBitRate field and AverageB
5 shows an example of a VariableRateInfo descriptor of the present invention including an itRate field. For MM content that uses variable bit rate coding, the decoder only knows that the bit stream is encoded with variable bit rate rate control from inspection of the information in the bit stream. The decoder can use this descriptor to know the maximum, minimum and average bit rate of the bitstream.

Table 3 shows the BitstreamParameterDescri of the present invention.
Here is an example of ptor. BitstreamParameterDescriptor is a descriptor that specifies encoding parameters that are not described differently in the bitstream itself. In the bitstream, only the information needed to decode the bitstream is described. However, sometimes that information alone (of the bitstream) is not enough to identify how the bitstream was coded and transmitted efficiently.

For this reason, the present invention provides specific coding parameters to allow efficient transcoding. If, for example, all pictures of the bitstream are encoded in the MPEG2 format as intra-type, it is possible to access all frames independently. In this case, it is possible to transfer the code of the bit stream to DV format without decoding the DCT coefficients.

However, the BitstreamParameterDesc of the present invention
Without a riptor, it is initially impossible to identify whether all pictures are coded as intra-type without decoding the entire bitstream.

[0094] The isI_Picture field is a 1-bit flag that specifies whether or not an I-picture exists in the bit stream. If an I-picture is present in the bitstream, then this flag is set to "1" (normal), otherwise "0"
Set to (error).

[0095] The isP_Picture field is a 1-bit flag that specifies whether a P picture exists in the bit stream. This flag is set to "1" (normal) if a P-picture is present in the bitstream, otherwise it is set to "0" (error).

[0096] The isB_Picture field is a 1-bit flag that specifies whether or not a B picture exists in the bit stream. This flag is set to "1" (normal) if a B picture is present in the bitstream, otherwise it is set to "0" (error).
If all bitstreams are coded with intra pictures only, then the isI_Picture field is equal to "1", the isP_Picture field is equal to "0", and the isB_Picture field is equal to "0".

[0097] The isFrameStruct field is a 1-bit flag that specifies whether or not the frame structure exists in the bit stream. This flag is set to "1" (normal) if there is at least one frame structure picture, otherwise it is set to "0" (error). The isField_struct field is a 1-bit flag used to specify whether the field structure exists in the bitstream. If there is at least one field structure picture, the flag is set to "1" (normal), otherwise it is set to "0" (error).

IsFrameMC is a 1-bit flag used to specify whether or not frame motion compensation (MC) is used in the bit stream. The flag is set to "1" (normal) if at least one frame MC is used, otherwise it is set to "0" (error).

[0099] The isField_MC field is a 1-bit flag that specifies whether or not the field structure exists in the bit stream. If at least one field MC is used, the flag is set to "1" (normal),
Otherwise, set to "0" (error).

The isSpecial_MC field is a 1-bit flag that specifies whether another type of MC, for example, dual prime, is used in the bit stream.

If at least one special MC is used,
The flag is set to "1" (normal), otherwise it is set to "0" (error).

[0102] The isFrameDCT field is a 1-bit flag that specifies whether or not a frame discrete cosine transform (DCT) is used in the bit stream. The flag is set to "1" (normal) if at least one frame DCT is used, otherwise "0".
Set to (error).

[0103] The isField_DCT field is a 1-bit flag that specifies whether or not the field structure exists in the bit stream. The flag is set to "1" (normal) if at least one field DCT is used, otherwise it is set to "0" (error).

Automatic Format Conversion Process of the Data Access Server of the Present Invention FIG. 7 was performed by the transcode manager 340 and the associated transcoding device 350 for an embodiment that implements the software transcoding process. Step flowchart 510 will be described. Step 510 can store the procedure in a computer readable storage element of system 112 (FIG. 11),
Implemented in one embodiment to be executed by processor 101. At step 515, the data access server 220 accesses the encoded bitstream (eg, MM content 246b) transferred from the multimedia content server 210 in response to a client request for multimedia information. At step 515, certain descriptors in the content information signal are also received as described in Tables 1, 2, and 3. In step 520, the data access server 220
Obtains certain client information from the client system 230 specifying the decryption format supported by the client system 230.

At step 525, the data access server 220 analyzes the descriptor information of the content information together with the client information. To determine (1) whether decoding is required for MM content before transcoding, and (2) the required buffer memory size to perform transcoding.

This information is supplied to the transcoding device 350 in the form of a transcode type information signal. At step 530, if the decoding indicated by the decision worked at step 525, then the MM content is decoded at step 535 using the encoding process specified in the content information, otherwise , Step 540 is input without decoding. If decryption works, step 535 generates a decrypted version of the MM content.

At step 540, the transcoding device 3
50 requests the appropriate transcoding tool, receives it from the transcoding tool library 360, and performs the transcoding process of the MM content on the transcoding bitstream. If decryption is not required, then MM content 246b is
0 may be in the transcoding bitstream, otherwise the transcoding process 540
Acts on the decrypted version of the MM content,
Go directly and be coded.

In step 540, any of a number of well-known transcoding processes can be used to perform the transcoding process. At step 545, data access server 220 sends the transcoded bitstream to client system 230 for display or other representation. Step 5
At 50, process 510 is repeated for the next bitstream.

The process 525 of FIG. 7 is described in more detail with respect to FIGS. 8, 9 and 10.

FIG. 8 shows the VisualBitstreamParameter in Table 3.
4 illustrates a process for determining a transcode type information signal using a Descriptor. In step 610, if isI_Picture is equal to 1, isP_Picture and isB_Pictur
If e is both 0, then all frames of the bitstream are coded as intra-picture type. Thus, in any frame of the bitstream, all frames of MM content can be accessed without decoding other special frames (eg, random access) and those that can be edited, eg, cut and・ Can paste.

Furthermore, it is possible to transcode the bit stream into other formats using DCT transcoding (without decoding the DCT coefficients of the MM content, so that no frame memory is required for the transcoding process).

Accordingly, entering step 615,
And a flag is set to indicate that no decoding is needed and that no frame buffer resources are needed. Applications can use this information to identify whether this bitstream is edited and accessible in any frame. After step 615, process 525 ends.

Referring to FIG. 3, the transcode manager 340 transfers the coding type information signal 364 to the coding device 350 as “transcoding is possible without decoding”. When transcoding device 350 receives it, it does not allocate any frame memory space to the transcoding process.

In FIG. 9, in step 620, if is is
I_Picture is equal to 1, isP_Picture is equal to 1, is
If B_Picture is 0, there is no B picture in the MM content, and at least one P picture is used in the bit stream.

Step 625 is entered. In this case,
Inter-frame prediction is used to encode this bitstream. If the output format (as specified by the client information) also supports inter-frame coding, then
Step 650 (FIG. 9) is input to enable transcoding the MM content without decoding its DCT coefficients.

However, if the output format does not support interframe coding, ie, DV format, JPEG, MPEG2, or MP
In EG4, DCT coefficients requiring MM content need to be decoded, and step 635 is entered to set a flag to decode and allocate one frame in the buffer memory.

At step 635, the output format does not support inter-frame coding, and the transcoding device 350
The coefficients need to be decoded. In this case, at least one frame memory is needed for transcoding.

Referring to FIG. 3, the transcode manager 340 transcodes the type information signal 364 as “one frame memory request”. After step 635, process 525 ends. Step 625
So, if the output format supports inter-frame coding, the decision process follows the rules of FIG.

Referring to FIG. 9, in step 650,
isFrameStruc is equal to 1, isFieldStruct is equal to 0, isFrameMC is equal to 1, isFieldMC is equal to 0,
isSpecialMC is equal to 0, isFrameDCT is equal to 1, i
If sFieldDCT is equal to 0, step 665 is entered.

In this case, no field-based coding is used in the bitstream. Thus, transcoding is possible without decoding the DCT coefficients, and no decoding process is required. After step 665, process 525 ends.

Referring to FIG. 3, transcode manager 340 transfers coding type information signal 364 to coding apparatus 350 as “transcoding is possible without decoding”. When the transcoding device 350 receives it, no frame memory is assigned to the transcoding process.

If isFieldStruct is equal to 1, then i
If sFieldMC is equal to 1 or isFieldDCT is equal to 1, then step 670 of FIG. 9 is entered because field-based coding is used in the bitstream. In step 670, if the output format supports field-based coding, then the code can be transferred without decoding the DCT coefficients, and step 660 is entered. At step 660, if isSpecialMC is equal to 0, DC
It is possible to transfer the code without decoding the T coefficient, and step 665 is entered.

Referring to FIG. 3, the transcode manager 340 determines that the coding type information signal 364 is “transcoding possible without decoding”, and
To transfer. When transcoding device 350 receives it, it does not allocate any frame memory to the transcoding process.

Step 670 of FIG. 9 enters step 675 if the output format does not support field coding. In step 675, the transcoding device 350 sends the DC of the MM content
It requires decoding the T coefficients and requires at least one frame memory to transfer the code. Referring to FIG. 3, the transcode manager 340 transfers the coded type information signal to the transcoding device 350 as "request 1 frame memory", and sets a flag indicating a decoding request. After steps 665 and 675, process 525 ends.

Referring to FIG. 8, if isI_Picture is equal to 1, isP_Picture is equal to 1 and isB_Picture is equal to 1, then step 630 is entered. In this case, the B picture is used in the bitstream (MM content). If the output format also supports B pictures, then step 680 of FIG. 10 is entered where it is possible to transcode without decoding the DCT coefficients of the MM content. However, the output format only supports I / P pictures. 261 and H.E. The H.263 format then enters step 640 and the DCT
The coefficients need to be decoded.

In step 640, the output format does not support B pictures, and the transcoding device 350 needs to decode the DCT coefficients of the MM content. Furthermore, in this case, at least two frame memory spaces are needed for transcoding. Referring to FIG. 3, the transcode manager 340 causes the "request 2 frame memory" coded transfer to be up to the transcoding device 350 to provide the type information signal. After step 640, process 525 ends.

In step 630 of FIG. 8, if the output format supports B pictures, the decision process follows the rules of FIG. For FIG. 10, if isFrameStruc is equal to 1, isFieldStruct is equal to 0, isFrameMC is equal to 1, isFieldMC is equal to 0, isSpecialMC is equal to 0, isFrameDCT is equal to 1 and isFieldDCT is equal to 0, the step 69
5 is input. In this case, no field-based coding is used in the bitstream.
Therefore, it is possible to transfer the code without decoding the DCT coefficients. Referring to FIG. 3, the transcode manager 340 transfers the transfer coding type information signal to the coding device 350 as “transferable without decoding”. When transcoding device 350 receives it, it does not allocate any frame memory to the transcoding process. After step 695, process 525 ends.

Referring to FIG. 10, if isFieldStruct, isF
If one of ieldMC or isFieldDCT is equal to 1, then step 710 is entered. In this case, field-based coding is used for the bitstream. If the output format supports field-based coding, then step 690 is entered where the DCT of the MM content
The code can be transferred without decoding the coefficients.

At step 690, if isSpecialMC is 0
If it is, the DCT coefficients and those that can be transferred without decoding step 695 are entered. Referring to FIG. 3, the transcode manager 340 transfers the coding type information signal to the coding device 350 as “transferable without decoding”.

When the transcoding device 350 receives it, no frame memory is assigned to the transcoding process. After step 695, process 525 ends.

In step 715 of FIG. 10, the transcoding device 350 needs to decode the DCT coefficients of the MM content and needs at least two frame memory spaces. Referring to FIG. 3, transcode manager 340 sends a type information signal to transcoding device 350 as a "two frame memory request".

Computer System Platform FIG. 11 illustrates a general-purpose computer system 112 that may be embedded within a data access server 220, according to one embodiment of the present invention. Computer system 112
Is used as a platform to implement the process of FIG. 7, FIGS. 8, 9 and 10.

The computer system 112 includes an address / data bus 100 for transmitting information, a central processor 101 associated with the bus 100 for storing static information and procedures for the processor 101 and information and procedures. And a volatile memory 10 associated with a bus 100 for storing information and procedures for a non-volatile memory 103 (eg, a read-only memory ROM).
2 (eg, random access memory RAM) a central processor 10 associated with a bus for processing
Including 1. The computer system 112 includes a magnetic, optical disk, data storage device 104 such as a hard disk coupled to the bus 100 for storing information, procedures, and a display device 105 connected to the bus 100 for displaying information to a computer user. (“Disk subsystem”).

The computer system 11 shown in FIG.
2 is a processor 10 that is focused on information and command selection.
1 has an optional alphanumeric input device 106 including comprehensive alphanumeric and function keys connected to the bus 100 for communication to the device. The system 112 also includes an optional cursor movement control or guidance device 107 connected to the bus for communicating user input information and command selections to the central processor 101. The cursor introduction device 107 can be implemented using many well-known devices, such as a trackball, trackpad, electronic pad, and stylus, optical tracker, touch screen, mouse, and the like, and the computer system 112 is a display device. 105 is an optional liquid crystal device, cathode ray tube (CRT), field emission device (FED is also referred to as flat panel CRT), or suitable for creating recognizable graphic images and alphanumeric characters for the user. Make other display devices available.

Thus, suitable embodiments of the present invention, multimedia information retrieval systems and methods, including methods and systems for automatic format conversion, are described above. The present invention is not limited to the embodiments of the present invention, but can be variously modified, and the scope of rights should be recognized by the claims.

[Brief description of the drawings]

FIG. 1 is a block diagram of a multimedia content access system according to one embodiment of the present invention.

FIG. 2 is a block diagram illustrating an embodiment of a multimedia content server according to the present invention.

FIG. 3 is a block diagram for explaining an embodiment of a data access server of the present invention including a transcoding device and a transcoding device manager.

FIG. 4 is a block diagram illustrating an embodiment of the multimedia content server of the present invention utilizing a multiplexed transmission.

FIG. 5 is a block diagram illustrating an embodiment of a data access server of the present invention utilizing a multiplexed transmission including a transcoding device and a transcoding device manager.

FIG. 6 is a block diagram illustrating one embodiment of a transcoding device according to the present invention.

FIG. 7 is a flowchart illustrating steps performed by the data access manager of the present invention for performing transcoding on an input bit stream of multimedia information.

FIG. 8 is a part of a flowchart showing a processing procedure of the embodiment of the data access server of the present invention.

FIG. 9 is a part of a flowchart showing a processing procedure of the embodiment of the data access server of the present invention.

FIG. 10 is a part of a flowchart showing a processing procedure of the embodiment of the data access server of the present invention.

FIG. 11 is a block diagram of a computer system platform on which embodiments of the present invention may be implemented.

 ──────────────────────────────────────────────────続 き Continuation of the front page (71) Applicant 500404214 Regent of California University 1111 Oakland Franklin Street, California, United States of America (72) Inventor Teruhiko Suzuki 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Yoichi Yagasaki 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo Sony Corporation F-term (reference) 5C059 KK41 LB18 MA00 MA04 MA05 MA23 PP05 PP06 PP07 RA09 RB09 RC00 RC32 SS07 SS13 SS14 SS30 UA02 UA05 UA29 5C064 BA07 BB05 BC18 BC21 BC23 BD01 BD02 BD08

Claims (29)

[Claims] 1. A method for processing multimedia information, comprising: accessing a portion of encoded multimedia information from a database storing encoded multimedia information in response to a request signal from a client. A media information access step; a content information access step of accessing content information describing a part of an encoded format of the encoded multimedia information and a decoding format of a client; automatically, content and transcoding Transcoding a portion of the encoded multimedia information to generate the base transcoded multimedia information into client information whose multimedia information is compatible with the client's decoding format. Multimedia information processing method. 2. The transcoding step includes: automatically determining, based on the content and the client information, whether or not it is necessary to decode a part of the encoded multimedia information; and decoding is required. Determining the automatic memory resources required to decode a portion of the encoded multimedia information, if necessary; and, if decoding is necessary, encoding to generate the multimedia information. Decoding a part of the decoded multimedia information, and transcoding the decoded multimedia information to provide transcoded multimedia information based on the content and the client information. The multimedia information processing method according to claim 1, wherein 3. The transcoding step includes: automatically determining, based on the content and the client information, that decoding of a part of the encoded multimedia information is not necessary; 2. The method according to claim 1, further comprising the step of transcoding the encoded multimedia information to provide the transcoded multimedia information. 4. The multimedia information processing method according to claim 1, further comprising the step of transmitting transcoded multimedia information to a client. 5. The method of claim 1, wherein the step of accessing multimedia information comprises the step of receiving client information from a client, wherein a portion of the encoded multimedia information is a digital bit stream. Multimedia information processing method. 6. The multimedia according to claim 1, wherein said content information comprises picture type information implying a coded picture type in a part of the coded multimedia information. Media information processing method. 7. The picture type information includes a flag for an I-picture, a flag for a P-picture, and B
7. The multimedia information processing method according to claim 6, comprising a flag for a picture. 8. The multimedia information processing according to claim 6, wherein the content information further comprises picture structure information indicating a type of structure used in a part of the encoded multimedia information. Method. 9. The multimedia information according to claim 6, wherein the content information further comprises motion compensation information indicating a type of motion compensation used in a part of the encoded multimedia information. Processing method. 10. The multimedia information processing method according to claim 9, wherein said motion compensation information comprises a frame and a field motion compensation flag. 11. The multimedia according to claim 6, wherein the content information comprises discrete cosine transform (DCT) information indicating a type of DCT coefficient used in a part of the encoded multimedia information. Media information processing method. 12. The DCT information includes a frame and a field D.
2. The multimedia information processing method according to claim 1, comprising a CT flag. 13. A multimedia information system, comprising: a processor connected to a bus; and a memory connected to the bus, wherein the memory stores encoded multimedia information in response to a request signal from a client. A multimedia information access step for accessing a part of the encoded multimedia information in the stored database; a part of the encoded format of the encoded multimedia information is accessed; and a decoding format of the client is described. Accessing the client information, accessing the client information, and encoding the content and transcoded multimedia information to generate base transcoded multimedia information into client information compatible with the decoding format of the client. And a transcoding step of automatically transcoding a part of the multimedia information. 14. The transcoding step comprises: determining whether or not a part of the encoded multimedia information is necessary, based on the content and the client information, and automatically decoding; Automatically determining the memory resources needed to decode a portion of the encoded multimedia information; and, if decoding is required, re-allocating the portion of the encoded multimedia information. Decoding to generate multimedia information by decoding; and transcoding the decoded multimedia information to provide transcoded multimedia information based on the content and the client information. The multimedia information system according to claim 13, wherein: 15. The transcoding step comprises: automatically determining, based on the content and the client information, that decoding of a part of the encoded multimedia information is not necessary; and 14. The multimedia information system of claim 13, further comprising: transcoding the encoded multimedia information to provide transcoded multimedia information. 16. The multimedia information system according to claim 13, further comprising the step of communicating transcoded multimedia information to a client. 17. The method of claim 13, wherein the step of accessing multimedia information further comprises the step of receiving client information from a client, wherein a portion of the encoded multimedia information is a digital bit stream. Multimedia information system. 18. The multimedia information system according to claim 13, wherein said content information comprises picture type information indicating a coded picture type of a part of the coded multimedia information. 19. The multimedia information system according to claim 18, wherein the picture type information comprises a flag for an I-picture, a flag for a P-picture, and a flag for a B-picture. . 20. The multimedia information according to claim 18, wherein the content information further comprises picture structure information indicating a type of structure used in a part of the encoded multimedia information. system. 21. The multimedia of claim 18, wherein the content information further comprises motion compensation information indicating a type of motion compensation used in a part of the encoded multimedia information. Information system. 22. The motion compensation information according to claim 1, wherein the motion compensation information comprises a frame and a field motion compensation flag.
9. The multimedia information system according to item 8. 23. The method of claim 18, wherein the content information further comprises discrete cosine transform (DCT) information indicating a type of DCT coefficient used in a portion of the encoded multimedia information. The multimedia information system as described. 24. The multimedia information system according to claim 23, wherein said DCT information comprises a frame and a field DCT flag. 25. A multimedia information system, comprising: a client system; a multimedia content server including multimedia information describing the multimedia information and encoded content information; and a client system and a multimedia content server. Connected to a portion of the encoded multimedia information from the multimedia content server in response to a request signal from a client system for accessing the content information describing the access client information; A data access server for accessing a part of an encoded format of the encoded multimedia information and a decoding format of a client system, wherein the data access server also has an encoded multimedia information. To generate transcoded multimedia information based on the content and client information to automatically transcode a portion of the information, where the multimedia information is a multimedia compatible with the decoding format of the client system. Media information system. 26. It is automatically determined based on the content and the client information whether or not a part of the encoded multimedia information needs to be decoded. A transcode manager that automatically determines the memory resources needed to decode a portion of the media information, and a portion of the encoded multimedia information to generate decoded multimedia information when decoding is required 26. The transcoder of claim 25, further comprising: a trans encoder for further transcoding the decoded multimedia information to decode the multimedia information according to the content and the client information and further provide the transcoded multimedia information. Multimedia information system. 27. The multimedia information according to claim 25, wherein the content information further comprises picture type information indicating an encoded picture type in a part of the encoded multimedia information. system. 28. The multimedia information according to claim 27, wherein the content information further comprises picture structure information indicating a type of structure used in a part of the encoded multimedia information. system. 29. The multimedia of claim 27, wherein the content information further comprises motion compensation information indicating a type of motion compensation used in a part of the encoded multimedia information. Information system.