KR20020032862A - An object-based multimedia service system and a service method using a moving picture encoding - Google Patents

Abstract

PURPOSE: An object-base multimedia service system and service method using motion picture coding are provided to offer motion pictures having high picture quality to a user by introducing a shape selector. CONSTITUTION: A multimedia service system that allows motion picture data to be transmitted from a terminal of a service provider to a user terminal through a network includes an interactive interface(250) and a shape selector(110). The interactive interface controls output of at least one motion picture at the request of a user, and allows the user to be able to select desired shape information. The shape selector selects the shape information the user wants from the motion picture data.

Description

OBJECT-BASED MULTIMEDIA SERVICE SYSTEM AND A SERVICE METHOD USING A MOVING PICTURE ENCODING

The present invention relates to multimedia, and more particularly, to a multimedia service system and a service method for encoding and decoding a video using shape information of a specific ROI, and playing back a video using such shape information.

Multimedia has recently attracted attention from many people in many fields and industries. Multimedia is not limited to PC, but is understood as a broader concept such as communication, broadcasting, home appliances, and computers. Media (consensus) is a generic term for a means of expressing information attributes (expression media) and a means of physically delivering information (delivery media). Thus, multimedia refers to the integrated handling of multiple expression media by the same delivery media. Such multimedia has important features in terms of expression as well as delivery. In particular, in terms of information delivery, attention has been focused on how to efficiently compress and deliver the expression media. Image compression has been developed by the Moving Picture coding Experts Group (MPEG) algorithm in the conventional JPEG (Joint Photographic Coding Experts Group).

Hereinafter, a conventional video method will be briefly described with reference to the accompanying drawings.

1A and 1B schematically illustrate a conventional video encoding and decoding method.

As shown in FIG. 1A, the conventional video encoding and decoding method simply encodes and transmits a 2D video as it is, receives it, decodes it, and plays it back. As an improvement over this method, MPEG-4 introduces a shape encoder / decoder as shown in FIG. The objects in the image can be separated and processed independently.

However, such a conventional encoding and decoding method needs to process shape information extraction on a meaningful object in an accurate and fast time, and processing a large amount of data in a limited system results in poor encoding and decoding efficiency, resulting in poor image quality. There is a problem.

In order to solve such a problem, the technical problem to be achieved by the present invention is to introduce a shape selector so that the user can be provided with a high-definition video service.

In addition, by providing a real-time object-based multimedia service, the user can select and adjust the number, size, location, etc. of the video content provided through the interactive interface.

1A and 1B schematically illustrate a conventional video encoding and decoding method.

2 is a block diagram illustrating an interactive multimedia service system according to a first embodiment of the present invention.

3 is a diagram illustrating an interactive multimedia service method according to a first embodiment of the present invention through a screen;

4 is a block diagram illustrating an interactive multi-object multimedia service system according to another example of the first embodiment of the present invention.

5 is a diagram illustrating an interactive multi-object multimedia service method according to another example of the first embodiment of the present invention through a screen.

6 is a diagram illustrating an interactive multi-object multimedia service method that can be reproduced according to a user's condition on a screen.

7 is a block diagram illustrating an interactive multi-object videophone / videoconference terminal according to a second embodiment of the present invention.

Fig. 8 is a block diagram showing an upstreaming interactive multi-object videophone / videoconference terminal according to another example of the second embodiment of the present invention.

An object-based multimedia service system using video encoding according to an aspect of the present invention for achieving the above object,

A multimedia service system for transmitting video data from a terminal of a service provider to a user terminal through a network.

An interactive interface for receiving a user's requirements to control output of one or more videos and allowing the user to select desired shape information; And

A shape selector which receives a result of determining shape information from the interactive interface and selects shape information according to the user's intention from the video data;

It includes.

Object-based multimedia service method using video encoding according to another aspect of the present invention,

A service method using a multimedia service system that supports transmission of video data from a terminal of a service provider to a user terminal through a network,

Receiving a request from a user from an interactive interface and causing the shape selector to select a shape according to the user request;

Synthesizing the shape with one or more multimedia data by multi-receiving and decoding information transmitted through the network when the image and the image within the shape are encoded and multi-transmitted.

It includes.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a block diagram illustrating an interactive multimedia service system according to a first embodiment of the present invention.

As shown in FIG. 2, the interactive multimedia service system according to the first embodiment of the present invention decodes the encoded video from the user terminal 200 when the encoded video is transmitted from the service provider terminal 100 through a network (not shown). Display.

The service provider terminal 100 may include: a shape selector 110 which receives a video from a video inputter (not shown) and determines a default value of an image or a system selected by a user, or analyzes the input image; Receives video, audio, and text information about the image determined by the shape selector 110, respectively, and uses the MPEG-4 technology to construct video, audio, and text information in an object unit (AVO: Audio / Visual Object) structured in a hierarchical structure. An encoding unit 120 configured to encode a complex; An encoding buffer 130 for storing encoded image data; It includes a multiple transmitter 140 for transmitting the tuned video, voice, text information in accordance with the information transmission timing for the encoded video, voice, text information.

The shape selector 110 may include a default shape 111 using a predetermined specific default shape; An image shape analyzer 112 that uses the selected shape determined by shape analysis of the input video; The shape extracting unit 113 extracts and uses the selected shape; The user select shape 114 uses the shape selected by the user through the interactive interface 260.

The encoder 120 includes a shape encoder 121 for encoding a shape selected from the shape selector 110; The apparatus may further include an image encoder 122 that obtains and encodes image data in the selected shape from the input video data by using the selected shape.

The user terminal 200 may include a multiple receiver 210 configured to receive and reconstruct a shape and video bitstream transmitted from a multiple transmitter 140 through a network; A decoding buffer 220 for storing the bitstream transmitted by the multiple receiver 210 and transferring the bitstream to the following decoder 230 in consideration of the reproduction timing; A decoder 230 for decoding a bitstream of shape and image data input from the decoding buffer 220; A shape image synthesizer 240 constituting an image object to be displayed by synthesizing the shape and image data decoded by the decoder 230; A video player (not shown) for outputting and playing the synthesized image object, and an interactive interface 250 for controlling the shape selector 110 by applying a video output method selected according to a user's intention.

Hereinafter, the operation of the interactive multimedia system according to the first embodiment of the present invention will be described in detail with reference to the accompanying drawings.

3 is a diagram illustrating an interactive multimedia service method according to a first embodiment of the present invention through a screen;

The user selects the shape desired by the user through the interactive interface 250. The user selection feature 114 of the shape selector 110 extracts the user selection shape transmitted from the interactive interface 250 and transmits the shape information to the encoder 120. The shape encoder 121 performs encoding on the selected shape as shown in FIG. 3, and the image encoder 122 obtains and encodes image data in the selected shape from the input video data by using the selected shape. The encoder 120 stores the compressed bitstream output from the encoder 120 in the encoding buffer 130 in consideration of synchronization (or timing) and stores the compressed bitstream for information transmission. Output to 140. The multiple transmitter 140 reconstructs the input bitstream and transmits the received bitstream to the multiple receiver 210 of the user terminal 200.

The multiple receiver 210 separates and reconstructs the transmitted compressed bitstream for each object, and the decoding buffer 220 is stored in consideration of the reproduction timing of the bitstream received from the multiple receiver 210 and then stored in the decoder 230. To pass. The shape decoder 231 decodes the bitstream associated with the shape information input from the decoding buffer 220, and the image decoder 232 decodes the shape information and the decoding buffer 220 output from the shape decoder 231. Decode the image information by using a bitstream related to the image information input through.

4 is a block diagram illustrating an interactive multi-object multimedia service system according to another example of the first embodiment of the present invention, and FIG. 5 illustrates a method of an interactive multi-object multimedia service according to another example of the first embodiment of the present invention. FIG. 6 is a diagram illustrating an interactive multi-object multimedia service method that can be reproduced according to a user's condition on a screen.

The interactive multi-object multimedia service system according to another example of the first embodiment of the present invention uses the same codes as the components of the first embodiment of the present invention. Is omitted.

As shown in FIG. 4, when one or more videos are input, one or more shape selectors 110 select a shape based on a result determined by the interactive interface 250, and one or more encoders 120 each shape. The shape selected by the selector 110 is encoded. Each encoding buffer 130 stores the encoded shape and image data, and the multi-transmitter 140 combines the shape and image data with one or more multimedia contents and transmits them. The multiple receiver 210 separates each of the received shape, image data, and multimedia content into a decoding buffer 220, and the decoding unit 230 reads information about the data from the decoding buffer 220 and decodes it. The object image synthesizer synthesizes the decoded shape, the image data and the multimedia content in each decoder 230 and displays the same through the video output unit 260.

As shown in FIG. 6, the encoder 120 and the decoder 230 of the present invention use an MPEG-4 core profile image and shape information encoding / decoding technique. The MPEG-4 Core Profile includes a shape information encoder in addition to the conventional two-dimensional video encoding method, and separates and encodes and decodes meaningful objects in a video. It is a technology that can guarantee additional functionality. Therefore, in the present invention, object-based shape encoding / decoding and image encoding / decoding of MPEG-4 are performed on shape information indicating a region of interest and image information therein. In addition, in the service method as shown in FIG. 5, the small advertisement image (multimedia content B) uses the base layer by using temporal scalability and spatial scalability. Service, and either or both of them as an enhancement layer, depending on the user's choice. At this time, temporal hierarchical coding is a method of hierarchically encoding the number of frames to be serviced per second. For example, if the original video can service 30 frames per second, the base layer samples 10 videos per second. In addition, the additional enhancement layer refers to an encoding technique that adds 20 pieces of information per second to allow a total of 30 pieces of service, and spatial layer encoding hierarchically increases the spatial resolution of a video from low to high size. For example, if the original video has a resolution of 352x288 pixels, the base layer is sampled and coded into a video having a resolution of 176x144 pixels as the base layer, and an additional enhancement layer is a base layer video. Original resolution of 352x288 by adding additional information to the information It is an encoding technique that enables complete service with degrees.

In FIG. 6, when encoding / decoding the image object 2 using the shape of MPEG-4 and the image spatial scalability, the information of the image object 1 is used to increase the encoding / decoding efficiency. Accordingly, the user may not only receive the service as the image object 1 but also receive the image object 2 by using an enhancement layer bitstream having an optimal amount of information according to the user's selection.

Hereinafter, the operation of the interactive multi-object multimedia service system according to the second embodiment of the present invention will be described with reference to the accompanying drawings.

In the interactive multi-object multimedia service system according to the second embodiment of the present invention, description of the same parts as in the first embodiment of the present invention will be omitted.

7 is a block diagram illustrating an interactive multi-object videophone / videoconference terminal according to a second embodiment of the present invention.

Interactive multi-object video telephony / video conferencing terminal according to a third embodiment of the present invention, the video input unit 210 receives video information through a camera or the like; A content encoding / transmitting unit 220 which processes encoding of shape and image data determined by a user's selection of one or more contents and transmission of the shape and image data; A content decoding / receiving unit 230 which receives and decodes the transmitted shape and image data; A display unit 240 for representing the received shape and image data; It includes an interactive interface 250 that allows a user to determine the desired shape, location, resolution, picture quality, and bitstream format.

The content encoder / transmitter 220 may determine a shape by analyzing the input value or the default value of the content or system selected from the interactive interface 250 according to the user's decision about the one or more contents ( 221); An encoder 222 which receives data determined by the shape selector 221 and configures and encodes image, voice, and text information into a complex of object units having a hierarchical structure; An encoding buffer 223 for storing encoded image data; It includes a multiple transmitter 224 for transmitting the tuned video, audio, text information in accordance with the information transmission timing for the encoded video, audio, text information.

The content decoding / receiving unit 230 includes: a multiple receiver 231 for receiving and reconstructing a shape and video bitstream transmitted from the multiple transmitter 224 through a network; A decoding buffer 232 for storing the bitstream transmitted by the multiple receiver 231 and transferring the bitstream to the following decoding unit 233 in consideration of the reproduction timing; A decoder 233 for decoding a bitstream of shape and image data input from the decoding buffer 232; And an object image synthesizer 234 constituting an image object to be displayed by synthesizing the shape and image data decoded by the decoder 233.

The video input unit 210 receives a video from a camera or the like, and the shape selector 221 receives a user's selection from the interactive interface 250 to shape, position, resolution, image quality, and bit of the video according to the user's selection. Determine the stream format. The determined shape and image data are encoded by the encoder 222. The multiple transmitter 224 mixes the objectified shape and image data with the content and transmits the content as a bitstream. The multi-receiver 231 separates the transmitted bitstream information into object units, and the decoder 233 changes object selection, object position, resolution, and quality from object data stored in the decoding buffer 232. Decode each object that corresponds to the same request. The object image synthesizer 234 synthesizes each decoded object data, and the display unit 240 displays the synthesized multimedia image.

Fig. 8 is a block diagram showing an upstreaming interactive multi-object videophone / videoconference terminal according to another example of the second embodiment of the present invention.

Hereinafter, a description of the same parts as the components of the interactive multi-object videophone / videoconference terminal according to another example of the second embodiment of the present invention will be omitted.

The interactive multi-object video telephony / video conferencing terminal according to another example of the second embodiment of the present invention is an up-streaming transmitter 260 that informs a counterpart sender of a receiver's selection shape and a receiver's reception environment and delivers optimal bitstream information. ); The apparatus further includes an upstreaming receiver 270 that grasps a request of a sender and receives optimal bitstream information from a counterpart sender.

As shown in FIG. 8, by using upstreaming, the user can receive the optimal bitstream information suitable for the receiver by informing the counterpart receiver of the selection and reception environment of the mixed video information through the upstream transmitter 260. The upstream receiver 270 receives the optimal bitstream information according to the receiver's reception environment.

Of course, many modifications and variations are possible in the configuration of the embodiments of the present invention without departing from the gist of the present invention, and the present invention is not limited to the embodiments.

As described above, the object-based multimedia service system and service method using the video encoding of the present invention is designed in a structure that can encode and decode only a specific region of interest of the user in real time, thereby improving the encoding and decoding efficiency, the same amount of information It is possible to provide a user with multimedia content with improved image quality and resolution. In particular, by using a shape selector that allows a service provider, a service user, a sender or a receiver on a video communication to select shape information through an interactive interface, it is possible to provide content that is optimal for a service environment.

In addition, the user can select and adjust the number, size, position, etc. of the image content provided through the interactive interface.

Claims (12)

In a multimedia service system supporting transmission of video data from a terminal of a service provider to a user terminal through a network, An interactive interface for receiving a user's requirements to control output of one or more videos and allowing the user to select desired shape information; And A shape selector which receives a result of determining shape information from the interactive interface and selects shape information according to the user's intention from the video data; Object-based multimedia service system using video encoding comprising a. In claim 1, An encoder which encodes at least one shape selected by the shape selector and an image in the at least one selected shape; An encoding buffer for storing bitstream data of the encoded shape and the image output from the encoder; A multiplexing unit configured to reconstruct input bitstream data of the encoding buffer and transmit the same through the network; A multiple receiver configured to receive and reconstruct bitstream data transmitted through the network; A decoding buffer which stores the reconstructed bitstream from the multiple receivers in consideration of reproduction timing; A decoder which decodes one or more shape and image data input from the decoding buffer; A shape image synthesizer which synthesizes and reproduces the shape data and the image data. Object-based multimedia service system using a video encoding further comprising. The method of claim 1, wherein the shape selector A default shape portion for selecting and using a predetermined specific default shape; An image shape analyzer configured to analyze the input video; A shape extraction unit which extracts and uses a shape of an object at the request of a user; User-selected features that use one or more selection features that have been selected by the user or service provider. Object-based multimedia service system using video encoding comprising a. The method of claim 2, wherein the encoder A shape encoder for encoding a shape selected by the shape selector; An image encoder which obtains and encodes image data in the selection shape from the input video data by using the selection shape selected by the shape selector Object-based multimedia service system using a video encoding further comprising. The method of claim 2, wherein the decoding unit A shape decoder for decoding a bitstream related to shape data input from the decoding buffer; An image decoder for decoding a bitstream associated with the configuration information output from the shape decoder and the image data input from the decoding buffer via the multiple receiver. Object-based multimedia service system using a video encoding further comprising. In claim 2, An upstream transmitter for receiving the user's shape selection request, content object selection, and receiving environment information from the interactive interface; And Receives the shape selection request, content object selection, and receiving environment information transmitted from the upstream transmitter through the network, and obtains optimal shape and image data suitable for the result information through the shape selector, shape encoder, and image encoder. An upstream receiver that controls the multiple transmitters to transmit an optimal bitstream by encoding Object-based multimedia service system using a video encoding further comprising. In the service method using a multimedia service system that supports the transmission of video data from the terminal of the service provider to the user terminal through the network, Receiving a request from a user from an interactive interface and causing the shape selector to select a shape according to the user request; If the selected shape and the image in the shape are encoded, and the encoded information and the additional multimedia content information are multi-transmitted, the information having the selected shape and the additional multimedia content are decoded by receiving and decoding the information transmitted through the network. Synthesizing Object-based multimedia service method using video encoding comprising a. The method of claim 7, wherein the shape selection step A method of using a selection shape selected by a user or a service provider from one or more selection shapes, a method of using a predetermined predetermined default shape, a method of using a selection shape determined through analysis of an input video, and extracting a shape Object-based multimedia service method using the video encoding, characterized in that it can be selectively used from the method using the extracted shape. The method of claim 8, wherein the synthesis step The upstream transmitter transmits the user's request, the user's selection, and the user's receiving environment through the network, and the upstream receiver identifies the needs of the other user, the user's selection, and the user's receiving environment. And causing the upstream transmitter to transmit an optimal bitstream. The method of claim 9, wherein the synthesis step An object-based multimedia service method using video encoding, characterized in that the object is processed with a change in size, position, and information link of an output multimedia object, together with one or more other multimedia objects under an environment set by the user's selection. The method of claim 9, wherein the synthesis step In addition to the main object to be provided to the user, at least one object or all objects selected by the user among the other multimedia objects that can be simultaneously provided are simultaneously played with the main object or according to the user's selection order, An object-based multimedia service method using video encoding, which may use various types of information including an advertisement video, a still image, or 2D / 3D graphics instead of the main object. The method of claim 9, wherein the synthesis step If the user wants to play the full video of the general format, select and provide a bitstream of the first video object that can play the video of the general format, If the user wants to play the internal region of the shape information selected by the shape selector at a high resolution, a scalable encoding bitstream for encoding the second image object, which is the corresponding selection region of the first image object, with a high resolution is used. Object-based multimedia service method using video encoding, characterized in that the.