KR101599875B1 - Method and apparatus for multimedia encoding based on attribute of multimedia content, method and apparatus for multimedia decoding based on attributes of multimedia content - Google Patents

Abstract

The present invention relates to a method and apparatus for receiving multimedia data and analyzing multimedia data to detect characteristic information for managing or searching for multimedia based on a predetermined characteristic of the multimedia content and using the characteristic information for managing or searching for multimedia, Based encoding scheme based on the encoding scheme.

Descriptor, MPEG-7

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia encoding method and apparatus based on content characteristics of multimedia and a multimedia decoding method and apparatus based on content characteristics of multimedia,

The present invention relates to coding and decoding of multimedia data.

Multimedia descriptors include a description of content characteristics for retrieving or managing multimedia information. Typically, a descriptor of MPEG-7 (Moving Picture Experts Group-7) is used. A user is provided with various information on multimedia according to the MPEG-7 video decoding method using an MPEG-7 descriptor, and the user can search multimedia desired by the user.

The present invention proposes the encoding or decoding of multimedia based on the content characteristics of multimedia.

According to an embodiment of the present invention, there is provided a multimedia encoding method based on content characteristics of multimedia, the multimedia encoding method comprising: receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia.

According to an embodiment of the present invention, there is provided a method of encoding multimedia data according to a coding scheme based on characteristics of the multimedia; And generating a bitstream including the encoded multimedia data.

The multimedia encoding method may further include encoding the property information for management or searching of the multimedia into a descriptor for managing or searching for multimedia based on the multimedia content, And generate a bitstream including a descriptor for management or search of the encoded multimedia data and multimedia based on the multimedia content.

The characteristic information detecting step of the multimedia encoding method according to an exemplary embodiment may detect and analyze color characteristics of image data as a predetermined characteristic of the multimedia content. The color characteristic of the image data may include at least one of a color layout of the image and a cumulative distribution by color bin.

The encoding method determination step of the multimedia encoding method according to an exemplary embodiment may include a step of measuring a change amount between pixel values of current image data and reference image data using color characteristics of the image data.

The encoding method determination step of the multimedia encoding method according to an exemplary embodiment may further include compensating a pixel value of the current image data using a change amount between a pixel value of the current image data and a pixel value of the reference image data . The multimedia encoding method may further include compensating a change amount of the pixel values of the current image data subjected to motion compensation and encoding the current image data.

A multimedia coding method according to an exemplary embodiment is a descriptor for managing or searching multimedia based on the multimedia content. The multimedia coding method includes metadata related to a color layout, a color structure color structure and meta data relating to a scalable color of the image data.

The characteristic information detecting step of the multimedia encoding method according to an exemplary embodiment may detect texture characteristics of the multimedia data by analyzing texture characteristics of the multimedia data. The texture property of the image data may include at least one of homogeneity, smoothness, regularity, edge directionality, and density of the image texture.

The encoding method determination step of the multimedia encoding method according to an exemplary embodiment may include determining a size of a data processing unit for motion estimation of current image data using texture characteristics of the image data .

In the encoding method determination step of the multimedia encoding method according to an exemplary embodiment, the size of the data processing unit may be determined such that the size of the data processing unit is larger as the current image data is more uniform based on the uniformity among the texture characteristics of the image data.

In the encoding method determination step of the multimedia encoding method according to an exemplary embodiment, the size of the data processing unit may be determined such that the size of the data processing unit becomes larger as the current image data becomes smoother based on the smoothness of the texture characteristics of the image data.

The encoding method determination step of the multimedia encoding method according to an exemplary embodiment may determine that the size of the data processing unit is larger as the pattern of the current image data is more regular based on the texture property of the image data.

The multimedia encoding method may further include performing motion estimation or motion compensation on the current image data using a data processing unit whose size is determined with respect to the image data.

The encoding method determination step of the multimedia encoding method according to another embodiment may include determining an intra prediction mode that can be performed on the current image data using the texture property of the image data .

The encoding method determination step of the multimedia encoding method according to another embodiment may include determining a type of a predictable intra prediction mode that can be performed on the current image data based on the directionality of an edge among the texture characteristics of the current image data, Priority can be determined.

The multimedia encoding method according to another embodiment may further include performing motion estimation on the current image data using an intra prediction mode determined for the current image data.

A multimedia coding method according to an exemplary embodiment is a descriptor for managing or searching multimedia based on the multimedia content. The descriptor may include metadata related to an edge histogram, texture the metadata for browsing and the metadata for texture homogeneity of texture.

The characteristic information detection step of the multimedia encoding method according to an exemplary embodiment may detect and analyze the fastness characteristic of the sound data as a predetermined characteristic of the multimedia content. The speed characteristics of the sound data may include tempo information of sound.

The step of determining a coding scheme of the multimedia coding method according to an exemplary embodiment includes a step of determining a length of a data processing unit for frequency transformation of current sound data using the fastness characteristic of the sound data .

The determining step of the encoding method of the multimedia encoding method according to an exemplary embodiment may further determine that the length of the data processing unit is shorter as the current sound data is faster based on the tempo information among the speed characteristics of the sound data.

The multimedia encoding method of the multimedia encoding method according to an embodiment may include frequency conversion of the current sound data using a data processing unit whose length is determined for the sound data.

The multimedia encoding method of the multimedia encoding method according to an embodiment is a descriptor for managing or searching multimedia based on the multimedia content. The multimedia encoding method includes metadata related to an audio tempo, And encoding at least one of semantic description information and side information.

In the encoding method determination step of the multimedia encoding method according to an exemplary embodiment, when the useful information is not extracted as the fastness characteristic of the sound data, the length of the data processing unit for frequency conversion of current sound data may be determined to be a fixed length have.

According to an embodiment of the present invention, there is provided a method of decoding multimedia based on a content characteristic of multimedia, the method comprising: receiving a multimedia data bitstream and parsing the bitstream to classify encoded data of the multimedia and information on the multimedia ; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And determining a decoding method based on the characteristics of the multimedia using the characteristic information for management or searching of the multimedia.

According to an embodiment of the present invention, there is provided a multimedia decoding method comprising: decoding encoded multimedia data according to a decoding method based on characteristics of the multimedia; And reconstructing the decoded multimedia data.

The characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment of the present invention includes parsing the bitstream and extracting a descriptor for managing or searching multimedia based on the multimedia content; And extracting the characteristic information from the descriptor.

In the characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment, the color characteristic of the image data may be extracted as a predetermined characteristic of the multimedia content.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may include a step of measuring a change amount between a pixel value of current image data and reference image data using a color characteristic of the image data.

According to an embodiment of the present invention, there is provided a multimedia decoding method comprising: performing motion compensation on current image data subjected to inverse frequency transform; And compensating a pixel value of the motion-compensated current image data using a change amount between the pixel value of the current image data and the pixel value of the reference data.

The characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment parses the bit stream to extract at least one of metadata relating to color layout, metadata relating to a color structure, and metadata related to a hierarchical color from the descriptor ; And extracting color characteristics of the image data from the extracted at least one descriptor.

The characteristic information extraction step of the multimedia decoding method according to an exemplary embodiment may extract the texture characteristic of the image data as a predetermined characteristic of the multimedia content.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may include determining a size of a data processing unit for motion estimation of the current image data using the texture property of the image data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may determine that the size of the data processing unit is larger as the current image data is more uniform based on the uniformity among the texture characteristics of the image data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may determine that the size of the data processing unit is larger as the current image data is smoother based on the smoothness of the texture characteristics of the image data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may determine that the size of the data processing unit is larger as the pattern of the current image data is more regular based on the normality among the texture characteristics of the image data.

The multimedia decoding method may further include performing motion estimation or motion compensation on the current image data using a data processing unit whose size is determined with respect to the image data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may include determining an intra prediction mode that can be performed on the current image data using the texture characteristic of the image data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may determine the type and priority of the intra prediction mode that can be performed on the current image data based on the directionality of edge among the texture characteristics of the current image data .

The multimedia decoding method may further include performing motion estimation on the current image data using an intra prediction mode determined for the current image data.

The characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment parses the bitstream and extracts at least one of metadata related to an edge histogram, metadata for texture browsing, and metadata about texture uniformity from the descriptor step; And extracting a texture characteristic of the image data from the extracted at least one descriptor.

In the characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment, the characteristic of the multimedia data may be extracted as a characteristic of the multimedia data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may include determining a length of a data processing unit for performing inverse frequency conversion of current sound data using the fastness characteristic of the sound data.

The decoding method determination step of the multimedia decoding method according to an exemplary embodiment may further determine that the length of the data processing unit is shorter as the current sound data is faster based on the tempo information among the speed characteristics of the sound data.

The method of multimedia decoding according to an exemplary embodiment may include performing reverse frequency conversion on the current sound data using a data processing unit whose length is determined for the sound data.

The characteristic information extracting step of the multimedia decoding method according to an exemplary embodiment of the present invention includes parsing the bitstream and extracting at least one of metadata, semantic attribute information, and side information related to an audio tempo from the descriptor; And extracting a temporal characteristic of the acoustic data from the extracted at least one descriptor.

In the decoding method determination step of the multimedia decoding method according to an exemplary embodiment, when the useful information is not extracted as the fastness characteristic of the sound data, the length of the data processing unit for the inverse frequency conversion of the current sound data may be determined to be a fixed length have.

According to an embodiment of the present invention, an apparatus for encoding multimedia based on a content characteristic of multimedia includes an input unit for inputting multimedia data; A characteristic information detector for analyzing the multimedia data and detecting characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; A coding mode determining unit for determining a coding mode based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia; And a multimedia data encoding unit for encoding the multimedia data according to a coding scheme based on the characteristics of the multimedia.

The multimedia encoding apparatus according to one embodiment may further include a descriptor encoding unit for encoding the property information for management or search of the multimedia into a descriptor for managing or searching for multimedia based on the multimedia content.

According to an embodiment of the present invention, an apparatus for decoding multimedia based on content characteristics of multimedia includes a receiver for receiving a multimedia data bitstream and parsing the bitstream to classify encoded data of multimedia and information about the multimedia ; A characteristic information extracting unit for extracting characteristic information for managing or searching multimedia from information about the multimedia; A decoding method determining unit for determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia; And a multimedia data decoding unit decoding the encoded data of the multimedia according to a decoding method based on the characteristics of the multimedia.

The multimedia decoding apparatus may further include a decompression unit for decompressing the decoded multimedia data.

The present invention includes a computer-readable recording medium on which a program for implementing a multimedia encoding method based on content characteristics of multimedia according to an embodiment of the present invention is recorded.

The present invention includes a computer-readable recording medium on which a program for implementing a multimedia decoding method based on content characteristics of multimedia according to an embodiment of the present invention is recorded.

Hereinafter, a multimedia encoding method, a multimedia encoding apparatus, a multimedia decoding method, and a multimedia decoding apparatus based on content characteristics of multimedia according to an embodiment of the present invention will be described with reference to FIG. 1 to FIG.

1 is a block diagram of a multimedia encoding apparatus based on content characteristics of multimedia according to an embodiment of the present invention.

The multimedia encoding apparatus 100 based on the content characteristic of multimedia according to an exemplary embodiment includes an input unit 110, a characteristic information detector 120, a coding scheme determination unit 130, and a multimedia data coding unit 140.

The input unit 110 receives the multimedia data and outputs the multimedia data to the characteristic information detector 120 and the multimedia data encoder 140. The multimedia data may include image data, sound data, and the like.

The characteristic information detection unit 120 analyzes the inputted multimedia data and detects characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content. In one embodiment, the predetermined characteristics of the multimedia content may include color characteristics of the image data, texture characteristics of the image data, and fastness characteristics of the sound data.

For example, the color characteristic of the image data may include a color layout of the image, a cumulative distribution of each color bin (hereinafter, referred to as a 'color histogram'), and the like. The color characteristics of the image data will be described later with reference to Figs. 8 and 9.

For example, the texture properties of the image data may include homogeneity, smoothness, regularity, edge orientation, coarseness, etc. of the image texture. The texture characteristics of the video data will be described later with reference to Figs. 16, 17, 18, 24, 25, and 26.

For example, the speed characteristics of the sound data may include tempo information of sound, and the like. The characteristics of the acoustic data will be described later with reference to Fig. 33.

The coding mode determination unit 130 may determine a coding mode based on the characteristics of the multimedia using the characteristic information for managing or searching multimedia extracted by the characteristic information detecting unit 120. [

The predetermined encoding method determined according to the characteristic information may be an encoding method for one of various operations of the encoding process. For example, the encoding method determination unit 130 can determine the compensation value of the luminance change amount according to the color characteristics of the image data. The coding mode determination unit 130 may determine the size and the estimation mode of the data processing unit used in the inter prediction according to the texture characteristics of the video data. In addition, the type and direction of the intra prediction mode that can be used can be determined according to the texture characteristics of the image data. The encoding method determination unit 130 can determine the length of the data processing unit for frequency conversion according to the speed characteristics of the sound data.

The encoding method determination unit 130 according to the embodiment can measure the amount of change between the pixel value of the current video data and the pixel value of the reference video data, that is, the luminance change amount, based on the color characteristics of the video data.

The encoding method determination unit 130 may determine the size of the data processing unit for motion estimation of the current image data using the texture property of the image data. The data processing unit for temporal motion estimation, which is determined by the coding method determination unit 130 according to the embodiment, may be a block such as a macroblock.

The coding mode determination unit 130 may determine that the size of the data processing unit for motion estimation is larger as the current image data is more uniform based on the uniformity among the texture characteristics. In addition, the size of the data processing unit may be determined to be larger as the current image data is smoother based on the smoothness of the texture characteristics. In addition, the more regular the pattern of the current image data is, the greater the size of the data processing unit may be determined based on the regularity among the texture characteristics.

The encoding method determination unit 130 may determine the type and direction of the intra prediction mode that can be performed on the current image data using the texture property of the image data. The type of intra prediction mode may include a directional prediction mode and a DC average value mode, and the direction of the intra prediction mode includes a vertical direction, a horizontal direction, a lower left direction, a lower right direction, a vertical right direction, a lower horizontal direction, .

The encoding method determination unit 130 analyzes the edge components of the current video data using the texture characteristics of the video data and determines intra prediction modes that can be performed among the various intra prediction modes based on the edge components . The encoding method determination unit 130 according to an embodiment determines a priority among intra prediction modes that can be performed according to a dominant edge of image data and generates an intra prediction mode table that can be performed on the image data can do.

The encoding method determination unit 130 according to the embodiment can determine a data processing unit for frequency conversion of current sound data by using the fastness characteristic of the sound data. A data processing unit for frequency conversion of sound data includes a frame, a window, and the like.

The encoding method determination unit 130 can determine that the length of the data processing unit becomes shorter as the current acoustic data is faster based on the tempo information among the acoustic characteristics of the acoustic data.

The multimedia data encoding unit 140 encodes the multimedia data input to the input unit 110 based on the encoding method determined by the encoding method determination unit 130. [ The multimedia encoding apparatus 100 according to an exemplary embodiment may output encoded multimedia data in a bit stream format.

The multimedia data encoding unit 140 may encode multimedia data by performing operations such as motion estimation, motion compensation, intra prediction, frequency conversion, quantization, and entropy encoding. The multimedia data encoding unit 140 may perform at least one of motion estimation, motion compensation, intra prediction, frequency conversion, quantization, and entropy encoding in consideration of multimedia content characteristics.

The multimedia data encoding unit 140 may encode the current image data in which the pixel value is compensated using the change amount between the pixel values determined based on the color characteristics of the image data. When there is a sudden change in luminance between the current image and the reference image, a large number of residual components are generated, resulting in a negative result in encoding using the temporal similarity of the image sequence. Accordingly, the multimedia encoding apparatus 100 can perform more efficient encoding by compensating for the luminance change amount of the reference image data and the current image data with respect to the current image data on which motion compensation has been performed.

The multimedia data encoding unit 140 may perform motion estimation or motion compensation on the current image data using the data processing unit of the inter prediction mode determined based on the texture characteristic. Video coding performs inter prediction with various data processing units for the current image data and determines an optimal data processing unit. Therefore, the more kinds of data processing units are, the more accurate the inter prediction can be, but the calculation burden is increased.

The multimedia encoding apparatus 100 according to an exemplary embodiment may perform more efficient encoding by performing error rate optimization on the current image data using a data processing unit determined based on the texture component of the current image.

The multimedia data encoding unit 140 according to an exemplary embodiment may perform motion estimation on the current image data using the intra prediction mode determined based on the texture property. Video coding performs intraprediction on various types of prediction directions and intraprediction with respect to the current video data, and determines an optimal prediction direction and a type of an intra prediction mode. Therefore, the more the types of the intra prediction direction and the intra prediction mode are, the more the calculation burden is increased.

The multimedia encoding apparatus 100 according to an exemplary embodiment of the present invention performs intraprediction on the current image data by using the intra prediction direction and the type of the intra prediction mode determined based on the texture characteristic of the current image, have.

The multimedia data encoding unit 140 according to an exemplary embodiment may perform frequency conversion on the current sound data using a data processing unit whose length is determined with respect to the sound data. In audio coding, the length of the window in time for frequency conversion can determine the resolution of the frequency and the change in the representable temporal sound. The multimedia encoding apparatus 100 according to an exemplary embodiment may perform more efficient encoding by performing frequency conversion on the current sound data using the determined window length based on the current sound speed characteristics.

The multimedia data encoding unit 140 may determine the length of the data processing unit for frequency conversion of the current sound data to be a fixed length when useful information can not be extracted as the speed characteristics of the sound data. In the case of irregular sound such as natural sound, since the constant speed characteristic is not extracted, the multimedia data encoding unit 140 can perform frequency conversion with a data processing unit of a predetermined length.

The multimedia encoding apparatus 100 according to an exemplary embodiment of the present invention is a multimedia encoding apparatus that encodes feature information for multimedia management or search into a descriptor for multimedia management based on multimedia content or a search (hereinafter, referred to as a multimedia content descriptor) And a characteristic descriptor encoding unit (not shown).

The multimedia contact characteristic descriptor encoding unit may classify metadata related to a color layout, metadata related to a color structure, and scalable color to represent color characteristics of image data. ) Can be encoded.

The multimedia contact characteristic descriptor encoding unit may include metadata related to an edge histogram, metadata for texture browsing, and homogeneity of texture to represent texture characteristics of image data. Can be encoded.

The multimedia contact characteristic descriptor encoding unit may encode at least one of metadata related to an audio tempo, semantic description information, and side information to indicate a speed characteristic of sound data. Can be encoded.

The multimedia management content characteristic descriptor may be included in the bitstream into which the encoded multimedia data is inserted, or a bitstream separate from the encoded multimedia data may be generated.

The multimedia encoding apparatus 100 based on the content characteristic of the multimedia according to an embodiment can efficiently encode multimedia data based on the characteristics of the multimedia content.

In order to efficiently decode multimedia or manage and retrieve multimedia contents, information on the characteristics of the multimedia contents may be separately provided in a descriptor form. In particular, in this case, the multimedia encoding apparatus 100 according to an exemplary embodiment may extract the content characteristic using a descriptor for information management or search based on the multimedia content characteristic. Accordingly, the multimedia encoding apparatus 100 according to the embodiment can efficiently encode multimedia data using the content characteristics of the multimedia without analyzing the additional content characteristics.

In the multimedia encoding apparatus 100 according to an embodiment, various embodiments exist depending on the content characteristic and the encoding method to be determined. The case where the luminance change amount compensation value is determined according to the color characteristic of the image data among the various embodiments of the multimedia encoding apparatus 100 will be described below with reference to FIG.

The case where the data processing unit for inter prediction is determined according to the texture characteristic of the image data among the various embodiments of the multimedia encoding apparatus 100 will be described below with reference to FIG.

The case where the type and direction of the intra prediction mode are determined according to the texture characteristic of the image data among the various embodiments of the multimedia encoding apparatus 100 will be described below with reference to FIG.

The case where the length of the data processing unit for frequency conversion is determined according to the speed characteristics of the sound data among various embodiments of the multimedia encoding apparatus 100 will be described below with reference to FIG.

FIG. 2 illustrates a block diagram of a multimedia decoding apparatus based on content characteristics of multimedia according to an embodiment of the present invention.

The multimedia decoding apparatus 200 based on the content characteristic of multimedia according to an exemplary embodiment of the present invention includes a receiving unit 210, a characteristic information extracting unit 220, a decoding method determining unit 230 and a multimedia data decoding unit 240.

The receiving unit 210 receives and parses the multimedia data bitstream to classify the encoded data and multimedia information of the multimedia. Multimedia can include all kinds of data such as video and sound. The information on the multimedia may include metadata, a content property descriptor, and the like.

The characteristic information extracting unit 220 extracts characteristic information for managing or searching for multimedia from the information about the multimedia inputted from the receiving unit 210. [ The characteristic information for management or retrieval of multimedia may be information based on the content characteristic of the multimedia.

For example, the color characteristics of the video data among the content characteristics of the multimedia may include a color layout of the image, a color histogram, and the like. The texture characteristics of the video data among the multimedia content characteristics may include uniformity, smoothness, regularity, edge directionality, and density of the image texture. Among the content characteristics of the multimedia, the speed characteristics of the sound data may include tempo information of sound, and the like.

The characteristic information extracting unit 220 may extract the characteristic information of the multimedia content from the descriptor for managing and searching the multimedia information based on the multimedia content characteristic.

For example, the characteristic information extracting unit 220 may extract color characteristic information of the image data from at least one of a color layout descriptor, a color structure descriptor, and a hierarchical color descriptor. In addition, the characteristic information extracting unit 220 may extract the texture characteristic information of the image data from at least one of the edge histogram descriptor, the texture browsing descriptor, and the uniform texture descriptor. In addition, the characteristic information extracting unit 220 may extract the characteristic information of the acoustic data from at least one of the audio tempo descriptor, the semantic attribute information, and the side information.

The decoding method determining unit 230 determines a decoding method based on the characteristics of the multimedia using the characteristic information for managing or searching multimedia extracted from the characteristic information extracting unit 220. [

The decoding method determining unit 230 according to the embodiment can measure the amount of change between the pixel value of the current image data and the pixel value of the reference image data, that is, the luminance change amount, based on the color characteristics of the image data.

The decoding method determination unit 230 may determine the size of the data processing unit for motion estimation of the current image data using the texture property of the image data. The data processing unit for motion estimation of the inter prediction may be a block such as a macroblock.

The decryption method determining unit 230 may determine that the size of the data processing unit for inter-prediction of the current image data is greater as one of the uniformity, smoothness, and regularity among the texture characteristics of the current image data is higher have.

The decoding method determining unit 230 according to an embodiment analyzes the edge components of the current video data using the texture characteristics of the video data and determines the intra prediction modes among the various intra prediction modes based on the edge components . The decoding method determination unit 230 according to an exemplary embodiment may determine the priority among the intra prediction modes that can be performed according to the main edge of the video data to generate an intra prediction mode table that can be performed on the video data.

The decoding method determining unit 230 according to an embodiment can determine the data processing unit for frequency conversion of the current sound data by using the speed characteristics of the sound data. A data processing unit for frequency conversion of sound data includes a frame, a window, and the like. The decryption method determining unit 230 according to the embodiment can further determine that the length of the data processing unit is shorter as the current sound data is faster based on the tempo information among the speed characteristics of the sound data.

The multimedia data decoding unit 240 decodes the encoded data of the multimedia inputted from the receiving unit 210 according to the decoding method based on the characteristics of the multimedia determined by the decoding method determining unit 230. [

The multimedia data decoding unit 240 basically decodes the multimedia data by performing operations such as motion estimation, motion compensation, intra prediction, inverse frequency conversion, inverse quantization, and entropy decoding. The multimedia data decoding unit 240 may perform at least one of motion estimation, motion compensation, intra prediction, inverse frequency conversion, inverse quantization, and entropy decoding in consideration of multimedia content characteristics.

The multimedia data decoding unit 240 may perform motion compensation on the current image data subjected to the inverse frequency transform and output the current image data using the variation between the pixel values determined based on the color characteristics of the image data. The pixel value can be compensated.

The multimedia data decoding unit 240 may perform motion estimation or motion compensation on the current image data according to the inter prediction mode in which the size of the data processing unit determined based on the texture characteristic is determined.

The multimedia data decoding unit 240 according to an exemplary embodiment can perform intra prediction on the current video data according to the intra prediction mode determined based on the texture characteristic and the intra prediction mode in which the type of the intra prediction mode is determined.

The multimedia data decoding unit 240 according to an exemplary embodiment may perform inverse frequency conversion on the current sound data as the length of the data processing unit for frequency conversion is determined based on the fastness characteristics of the sound data.

The multimedia data decoding unit 240 according to an embodiment determines the length of a data processing unit for reverse frequency conversion of current sound data to be a fixed length when useful information can not be extracted as a speed characteristic of the sound data, Conversion can be performed.

The multimedia decoding apparatus 200 according to an exemplary embodiment of the present invention may further include a decompression unit (not shown) for decompressing and outputting the decoded multimedia data.

The multimedia decoding apparatus 200 according to an exemplary embodiment of the present invention extracts content characteristics of a multimedia using a descriptor provided for managing and searching for multimedia information in order to decode the multimedia contents considering the content characteristics of the multimedia. Therefore, the multimedia decoding apparatus 200 according to an embodiment can efficiently decode multimedia without performing an additional task of directly analyzing the content characteristic of the multimedia or new additional information.

In the multimedia decoding apparatus 200 according to an exemplary embodiment, various embodiments exist depending on the content characteristic and the decoding method to be determined. The case where the luminance change amount compensation value is determined according to the color characteristics of the image data among the various embodiments of the multimedia decoding apparatus 200 will be described below with reference to FIG.

The case where the data processing unit for inter prediction is determined according to the texture characteristic of the image data among the various embodiments of the multimedia decoding apparatus 200 will be described below with reference to FIG.

The case where the type and direction of the intra prediction mode are determined according to the texture characteristics of the image data among the various embodiments of the multimedia decoding apparatus 200 will be described below with reference to FIG.

The case where the length of the data processing unit for the inverse frequency conversion is determined according to the speed characteristics of the sound data among various embodiments of the multimedia decoding apparatus 200 will be described below with reference to FIG.

The multimedia coding apparatus 100 and the multimedia decoding apparatus 200 according to the embodiment described above with reference to FIGS. 1 and 2 may be a video coding / decoding apparatus based on spatial prediction or temporal prediction or a video coding / decoding apparatus using such a video coding / And is applicable to all image processing methods and apparatuses.

For example, the processes of the multimedia encoding apparatus 100 and the multimedia decoding apparatus 200 according to an exemplary embodiment may be applied to a mobile communication device such as a mobile phone, a video camera such as a camcorder, a digital camera, a multimedia player, or a portable multimedia player ), A multimedia playback apparatus such as a next generation DVD, and a software video codec.

In addition, the multimedia encoding apparatus 100 and the multimedia decoding apparatus 200 according to the embodiment can be applied to the next generation image compression standard as well as the current image compression standard such as MPEG-7 and H.26X.

The processes of the multimedia encoding apparatus 100 and the multimedia decoding apparatus 200 according to the embodiment may be applied not only to an image compression function but also to a media application that provides a search function used independently or simultaneously with image compression.

Metadata contains information that effectively represents the content, and some of the information contained in the metadata includes some information useful for encoding or decoding the multimedia data. Therefore, although the syntax information of the meta data is provided for information retrieval, the encoding / decoding efficiency of the sound data can be increased by using the close relation between the syntax information and the sound data.

3 shows a block diagram of a conventional video coding apparatus.

The conventional video coding apparatus 300 includes a frequency transform unit 340, a quantization unit 350, an entropy coding unit 360, a motion estimation unit 320, a motion compensation unit 325, an intra prediction unit 330, A frequency transform unit 370, a deblocking filtering unit 380, and a buffer 390.

The frequency transform unit 340 transforms the residual components of the predetermined image and the reference image in the input sequence 305 into frequency domain data and the quantization unit 350 transforms the frequency transformed data into a finite number . ≪ / RTI > The entropy encoding unit 360 outputs the bitstream 365 in which the input sequence 305 is encoded by lossless encoding the quantized value.

In order to utilize the temporal similarity between different images in the input sequence 305, a motion estimation unit 320 estimates motion between different images, and the motion compensation unit 325 estimates a motion The motion of the current image can be compensated.

Also, in order to utilize the spatial similarity of the different regions of one of the input sequences 305, the intra predictor 330 predicts the reference region most similar to the current region of the current image.

Accordingly, the reference image for obtaining the residual component of the current image may be a motion compensated image by the motion compensation unit 325 based on temporal redundancy. Alternatively, the reference image may be an image predicted in the intra-prediction mode through the intra-prediction unit 330 based on the spatial redundancy in the same image.

The deblocking filtering unit 380 transforms the quantized values into spatial domain data by the inverse frequency transform unit 370 and performs frequency transformation, quantization, motion estimation, and the like on the image data plus the reference image data Thereby reducing blocking artifacts on the boundary of the data processing unit of FIG. The deblocking filtered decoded picture may be stored in a buffer 390.

4 shows a block diagram of a conventional video decoding apparatus.

The conventional video decoding apparatus 400 includes an entropy decoding unit 420, an inverse quantization unit 430, an inverse frequency transform unit 440, a motion estimation unit 450, a motion compensation unit 455, an intra prediction unit 460, A deblocking filtering unit 470, and a buffer 480.

The input bit stream 405 is lossless decoded and inverse quantized through the entropy decoding unit 420 and the inverse quantization unit 430. The inverse frequency transform unit 440 performs inverse frequency transformation on the inverse quantized data, Area video data.

The motion estimation unit 450 and the motion compensation unit 455 compensate temporal motion between different images using the deblocked reference image and the motion vector and the intra prediction unit 460 compensates the temporal motion between the deblocked reference image and the reference index To perform intra prediction.

A motion compensated or intra predicted reference image and a residual component reverse-frequency-transformed into a spatial domain are added to generate current image data. By blocking the current image data through the deblocking filtering unit 470, blocking artifacts occurring at the boundary of data processing units such as inverse frequency conversion, inverse quantization, and motion estimation are reduced. The decoded and deblock-filtered pictures may be stored in the buffer 480.

Conventional video encoding apparatus 300 and conventional video decoding apparatus 400 use temporal similarity between consecutive images and spatial similarity between adjacent regions in one image to reduce the amount of data for expressing an image, The characteristics of the system are not considered at all.

Hereinafter, a first embodiment for encoding or decoding image data based on color characteristics among content characteristics will be described in detail with reference to FIGS. 5 to 11. FIG.

Hereinafter, a second embodiment for encoding or decoding image data based on the texture characteristic among the content characteristics will be described in detail with reference to FIGS. 12 to 20. FIG.

Hereinafter, a third embodiment of encoding or decoding image data based on the texture characteristic among the content characteristics will be described in detail with reference to FIGS. 21 to 29. FIG.

Hereinafter, with reference to FIGS. 30 to 35, a fourth embodiment for encoding or decoding the sound data based on the fast characteristic among the content characteristics will be described in detail.

Hereinafter, a first embodiment for encoding or decoding image data based on color characteristics among content characteristics will be described in detail with reference to FIGS. 5 to 11. FIG.

FIG. 5 is a block diagram of a multimedia encoding apparatus based on the color characteristics of multimedia according to the first embodiment of the present invention.

The multimedia encoding apparatus 500 according to the first embodiment includes a color characteristic information detector 510, a motion estimator 520, a motion compensator 525, an intra predictor 530, a frequency transformer 540, An entropy encoding unit 560, an inverse frequency conversion unit 570, a deblocking filtering unit 580, a buffer 590, and a color characteristic descriptor encoding unit 515. [

The overall encoding process of the multimedia encoding apparatus 500 according to the first embodiment is performed by using temporal similarity of consecutive images of the input sequence 505 and spatial similarity in an image, 565).

That is, inter prediction and motion compensation are performed through the motion estimation unit 520 and the motion compensation unit 525, intraprediction is performed through the intra prediction unit 530, and a frequency transform unit 540, a quantization unit 550) and the entropy encoding unit 560 are generated. The blocking effect that may occur during the encoding operation can be eliminated through the inverse frequency transform unit 570 and the deblocking filtering unit 580. [

The multimedia encoding apparatus 500 according to the first embodiment further includes a color characteristic information detector 510 and a color characteristic descriptor encoder 515 as compared to the conventional video encoding apparatus 300. [ The operation of the motion compensation unit 525 using the color characteristic information detected by the color characteristic information detection unit 510 is distinguished from the motion compensation unit 325 of the conventional video coding apparatus 300. [

The color characteristic information detection unit 510 according to an embodiment analyzes the input sequence 505 to extract a color histogram or a color layout. For example, in accordance with the YCbCr color specification, the color layout includes discrete cosine transformed coefficient values for each Y, Cb, and Cr color component for each sub-image.

The color characteristic information detection unit 510 can measure a luminance change amount between the both images using the color histogram or color layout of the current image and the reference image. The current image and the reference image may be consecutive images.

The motion compensation unit 525 can compensate for the sudden change in luminance by adding the luminance change amount to the predicted area after motion compensation. For example, the color characteristic information detection unit 510 may add the measured luminance variation amount to an average value of pixels in the predicted area.

The sudden change in luminance increases the residual, which may reduce the efficiency of image data coding. Accordingly, the amount of change between pixel values of continuous image data is measured using the color characteristic, the pixel value of the current image data is compensated by using the amount of change between the pixel values of the previous image data and the current image data, So that efficient coding can be achieved.

When the color characteristic detected by the color characteristic information detecting unit 510 according to the embodiment is a color layout, the color characteristic descriptor coding unit 515 according to an embodiment uses the color layout information to generate the color layout information Can be encoded. For example, in an environment based on the MPEG-7 standard compression standard, an example of metadata regarding the color layout may be a color layout descriptor.

Alternatively, when the color characteristic detected by the color characteristic information detecting unit 510 according to an exemplary embodiment is a color histogram, the color characteristic descriptor encoding unit 515 may use the color histogram information, Can be encoded as metadata relating to data or hierarchical color.

For example, in an environment based on the MPEG-7 standard compression standard, an example of the metadata regarding the color structure may be a color structure descriptor. Also, in an environment based on the MPEG-7 standard compression standard, an example of the metadata regarding the hierarchical color may be a scalable color descriptor.

The metadata regarding the color layout, the metadata about the color structure, and the metadata about the hierarchical color correspond to the descriptors for information management and search of the multimedia contents, respectively.

The color layout descriptor is a descriptor that schematically shows the color characteristics. The input image is converted into the color space of YCbCr, divided into small areas of 8 × 8 pixels, and an average of pixel values is obtained for each area. Color features can be extracted by performing 8x8 discrete cosine transform on each color component of the generated small area Y, Cb, and Cr and selecting the number of transformed coefficients.

The color structure descriptor is a descriptor that indicates the spatial distribution of color bin values of an image. The local histogram is extracted using an 8 × 8 window mask based on the image of the CIF size (352 pixels in horizontal size, 288 pixels in vertical size). If there are color bin values of the local histogram, the final histogram is updated, so that cumulative spatial distribution of the corresponding color bin can be analyzed for each color bin.

The hierarchical color descriptor is a color descriptor that secures and expresses a hierarchical level by Haar transform of a color histogram as a modified form of the color histogram descriptor.

The color characteristic descriptor encoded by the color characteristic descriptor encoding unit 515 may be included in the bitstream 565 as encoded multimedia data. Or may be output as a bitstream different from the encoded multimedia data.

The input sequence 505 corresponds to an image input through the input unit 110 and the characteristic information detection unit 120 and the encoding method determination unit 130 are provided with color The characteristic information detecting unit 510 may be corresponded. The multimedia data encoding unit 140 includes a motion estimation unit 520, a motion compensation unit 525, an intra prediction unit 530, a frequency conversion unit 540, a quantization unit 550, an entropy encoding unit 560, The frequency transforming unit 570, the deblocking filtering unit 580, and the buffer 590, respectively.

After the motion compensation, the motion compensation unit 525 adds the luminance change amount compensation value measured by the color property information detection unit 510 to the motion-compensated image to increase the residual component due to the abrupt luminance change or increase the number of intra estimation .

Another embodiment of the color characteristic information detection unit 510 may determine whether to perform inter prediction or intra prediction according to the degree of luminance change between the both images, using the extracted color characteristics of the reference image and the current image. For example, it is possible to perform intra prediction if the luminance change between the reference image and the current image is smaller than a predetermined threshold, and to perform inter prediction if the luminance change between the reference image and the current image is equal to or greater than a predetermined threshold value.

6 is a block diagram of a multimedia decoding apparatus based on color characteristics of multimedia according to a first embodiment of the present invention.

The multimedia decoding apparatus 600 according to the first embodiment includes a color characteristic information extractor 610, an entropy decoder 620, an inverse quantizer 630, an inverse frequency transformer 640, a motion estimator 650, A motion compensation unit 655, an intra prediction unit 660, a deblocking filtering unit 670, and a buffer 680.

The overall decoding process of the multimedia decoding apparatus 600 according to the first embodiment is for generating a reconstructed image using the encoded multimedia data of the input bitstream 605 and all the information about the multimedia data.

That is, the bitstream 605 is losslessly decoded through the entropy decoding unit 620, and the residual components of the spatial domain are decoded through the inverse quantization unit 630 and the inverse frequency transformation unit 640. The motion estimation unit 650 and the motion compensation unit 655 perform temporal motion estimation and motion compensation using the reference image and the motion vector and the intra prediction unit 660 performs intra prediction using the reference image and index information can do.

An image obtained by adding the residual component and the reference image may be subjected to deblocking filtering 670 to reduce the blocking effect that may occur during the decoding operation. The decoded pictures and the like can be stored in the buffer 680.

The multimedia decoding apparatus 600 according to the first embodiment further includes a color characteristic information extracting unit 610 as compared with the conventional video decoding apparatus 400. [ The operation of the motion compensation unit 655 using the color characteristic information extracted by the color characteristic information extraction unit 610 is distinguished from the motion compensation unit 455 of the conventional video decoding apparatus 400. [

The color characteristic information extracting unit 610 may extract color characteristic information using the color characteristic descriptor classified from the input bitstream 605. [ For example, a color layout or a color histogram can be extracted if the color characteristic descriptor is one of metadata about a color layout, metadata about a color structure, and metadata about a hierarchical color.

For example, in an MPEG-7 standard compression standard environment, the metadata about the color layout, the metadata about the color structure, and the metadata about the hierarchical color may be color layout descriptors, color structure descriptors and hierarchical color descriptors, respectively .

The color characteristic information extracting unit 610 can measure the luminance change amount of the reference image and the current image from the color characteristics of the reference image and the current image. The motion compensation unit 655 can compensate for the sudden change in luminance by adding the luminance change amount to the predicted area after motion compensation. For example, the luminance variation measured by the color characteristic information extracting unit 610 may be added to an average value of pixels in the predicted area.

The input bit stream 605 corresponds to the bit stream input through the receiver 210 and the characteristic information extraction unit 220 and the decoding method determination unit 230 The color characteristic information extracting unit 610 may correspond to the color characteristic information extracting unit 610. The multimedia data decoding unit 240 includes a motion estimation unit 650, a motion compensation unit 655, an intra prediction unit 660, an inverse frequency transformation unit 640, an inverse quantization unit 630, an entropy decoding unit 620, The deblocking filtering unit 670, and the buffer 680, respectively.

The coding efficiency may be deteriorated due to a sudden change in luminance. Therefore, when decoding a bitstream encoded with the luminance change compensated at the coding end, the luminance variation is again compensated for the decoded image data after motion compensation to restore the original image .

Another embodiment of the color characteristic information extracting unit 610 may determine whether to perform inter prediction or intra prediction according to the degree of luminance change between the both images, using the extracted color characteristics of the reference image and the current image. For example, it is possible to perform intra prediction if the luminance change between the reference image and the current image is smaller than a predetermined threshold, and to perform inter prediction if the luminance change between the reference image and the current image is equal to or greater than a predetermined threshold value.

FIG. 7 shows the luminance variation between consecutive frames measured using color characteristics according to the first embodiment of the present invention.

When a sudden change in brightness occurs such as a flash light, a DC value changes between the original image and the predicted image. The abrupt change of the DC value is also undesirable in terms of coding efficiency since it induces intra prediction instead of inter prediction.

A color layout descriptor can be used when a change in luminance between the current area 760 of the current image 750 is to be obtained using the reference area 710 of the reference image 700. [ A color layout descriptor (CLD) indicates a frequency-converted value of a representative value for each of the Y, Cr, and Cb color components for each of 64 sub-images of one image. Therefore, by using the amount of change (± DELTA _CLD ) between the inverse frequency-converted values of the color layout descriptors of the reference area 710 and the current image 750, the relationship shown in the following equation (1) can be derived.

± Δ _CLD = (average pixel value of the reference area) - (average pixel value of the current area)

? _CLD may correspond to the amount of change in the luminance of the reference area 710 and the current area 760. Accordingly, the color characteristic information detector 510 or the color characteristic descriptor extractor 610 calculates the amount of change (± DELTA _CLD ) between the inverse frequency converted values of the color layout descriptors of the reference area 710 and the current image 750 And ± _ΔΔCLD can be compensated for as the luminance change amount in the motion compensated current area.

8 shows a color histogram used as a color characteristic according to the first embodiment of the present invention.

The histogram bin (horizontal axis) of the color histogram 800 represents intensity per color. The first histogram 810, the second histogram 820, and the third histogram 830 are color histograms of the first image, the second image, and the third image, which are consecutive three images.

The first histogram 810 and the third histogram 830 exhibit substantially similar intensity and distribution while the second histogram 820 is similar to the first histogram 810 and the third histogram 830 in the rightmost histogram bin The cumulative distribution is overwhelmingly high.

(A first image), suddenly a flashlight is imaged and a sudden change in brightness occurs (a second image), and when the flashlight disappears, the image is returned to normal illumination (a third image) The first histogram 810, the second histogram 820, and the third histogram 830 may be output.

Therefore, by analyzing the difference between the color histograms 810, 820, and 830, an image in which a sudden change in luminance of the images occurs can be detected, and the image level can be grasped.

Figure 9 shows a color layout used as color characteristics in accordance with a first embodiment of the present invention.

A color layout is generated by dividing the original image 900 into 64 sub-images such as the sub-image 905 and obtaining an average value for each color component for each sub-image. A binary code generated by performing 8x8 discrete cosine transform on each of the Y component, Cb component, and Cr component of the sub image 905 and assigning weights according to the zigzag scanning order to the transformed coefficients, It is a descriptor. The color layout descriptor can be sent to the decoding end and can be used for sketch-based retrieval.

The color layout 910 of the current image includes mean values 912 of the Y component for each sub-image of the current image 910, mean values 914 of the Cr component, and mean values 916 of the Cb component. The color layout 920 of the reference image also includes average values 922 of the Y component for each sub-image of the current image 920, average values 924 of Cr components, and average values 926 of Cb components .

In the first embodiment of the present invention, by ± △ _CLD of equation (1) the difference value of the color layout 920 of a color layout 910 and the reference picture for the current picture, the current to be used as a luminance variation between the image and the reference image, . Accordingly, the motion compensation unit 525 or the motion compensation unit 655 according to the first embodiment may store the difference between the color layout 910 of the current image and the color layout 920 of the reference image, The luminance variation can be compensated by adding the sum.

10 is a flowchart illustrating a multimedia encoding method based on color characteristics of multimedia according to the first embodiment of the present invention.

In step 1010, the multimedia data is input.

In step 1020, color information of the image data is detected as characteristic information for management or search of multimedia. The color information may be a color histogram, a color layout, or the like.

In step 1030, the compensation value of the luminance change amount after the motion compensation may be determined based on the color characteristic of the image data. The compensation value of the luminance change amount can be determined using the difference between the respective color histograms of the current image and the reference image or the difference between the respective color layouts. The compensation value of the luminance change amount is added to the motion-compensated current image, so that the rapidly changed luminance of the current image can be compensated.

In step 1040, the multimedia data may be encoded. The multimedia data may be encoded through frequency conversion, quantization, deblocking filtering, entropy encoding, and the like and output as a bit stream.

The color characteristics extracted in step 1010 are encoded with metadata related to the color layout, metadata relating to the color structure, meta data relating to the hierarchical color, and the like to search or manage the multimedia information based on the multimedia content characteristic at the decoding end Can be used. The descriptor may be output in the form of a bit stream together with the encoded multimedia data.

The PSNR of the block predicted by the multimedia encoding apparatus 100 according to the embodiment may be improved and the coefficient of the residual component may be reduced to increase the efficiency of ineffectiveness. Of course, the multimedia information can be retrieved by using the descriptor.

11 is a flowchart illustrating a multimedia decoding method based on color characteristics of multimedia according to the first embodiment of the present invention.

In step 1110, a multimedia data bitstream is received. The bit stream may be parsed and coded into encoded data of multimedia and information data of multimedia.

In step 1120, the color information of the image data may be extracted as characteristic information for management or search of multimedia. Characteristic information for multimedia management or retrieval can be extracted from a descriptor for management and retrieval of multimedia information based on multimedia content characteristics.

In step 1130, the luminance change amount compensation value after motion compensation may be determined based on the color characteristics of the image data. The color component average value of the current area and the difference value of the color component average value of the reference area can be used as the luminance change amount compensation value by using the color histogram, the color layout, and the like among the color characteristics.

In step 1140, the encoded data of the multimedia may be decoded. The encoded multimedia data may be decoded through entropy decoding, inverse quantization, inverse frequency transformation, motion estimation, motion compensation, intra prediction, deblocking filtering, and the like, and then restored into multimedia data.

Hereinafter, a second embodiment for encoding or decoding multimedia data based on texture characteristics of image data will be described in detail with reference to FIGS. 12 to 20. FIG.

12 is a block diagram of a multimedia encoding apparatus based on texture characteristics of multimedia according to a second embodiment of the present invention.

The multimedia encoding apparatus 1200 according to the second embodiment includes a texture characteristic information detector 1210, a data processing unit determiner 1212, a motion estimator 1220, a motion compensator 1225, an intra predictor 530, A frequency transformation unit 540, a quantization unit 550, an entropy coding unit 560, an inverse frequency transformation unit 570, a deblocking filtering unit 580, a buffer 590 and a texture property descriptor coding unit 1215, .

The overall encoding process of the multimedia encoding apparatus 1200 according to the second embodiment is similar to that of the first embodiment except that the overlapped data is omitted using the temporal similarity of continuous images of the input sequence 505 and the spatial similarity in an image, 1265).

The multimedia encoding apparatus 1200 according to the second embodiment may further include a texture characteristic information detection unit 1210, a data processing unit determination unit 1212 and a texture characteristic descriptor encoding unit 1215 in comparison with the conventional video encoding apparatus 300 do. The operation of the motion estimation unit 1220 and the motion compensation unit 1225 using the data processing unit determined by the data processing unit determination unit 1212 is similar to that of the motion estimation unit 320 of the conventional video encoding apparatus 300, And is distinguished from the compensation unit 325.

The texture property information detector 1210 according to the second embodiment analyzes the input sequence 505 to extract a texture component. For example, the texture component may be uniformity, smoothness, regularity, edge directionality, density, and the like.

The data processing unit determination unit 1212 can determine the size of the data processing unit for motion estimation of the image data using the texture property detected by the texture property information detection unit 1210. The data processing unit may be a rectangular block.

For example, the data processing unit determination unit 1212 can use the uniformity among the texture characteristics of the image data to determine that the data processing unit is larger as the texture of the image data is uniform. The data processing unit determination unit 1212 may determine that the data processing unit is larger as the image data becomes smoother by using the smoothness among the texture characteristics of the image data. The data processing unit determination unit 1212 may determine that the data processing unit is larger as the pattern of the image data is more regular by using the normality among the texture characteristics of the image data.

In particular, data processing units of various sizes can be classified into various groups according to their sizes. A group may include a data processing unit having a size within a predetermined range. When a predetermined group is mapped according to the texture characteristic of the video data, the data processing unit determination unit 1212 performs error rate optimization using the data processing units in the group and generates a data processing unit for generating the lowest error rate as an optimal data processing unit You can decide.

Therefore, the portion where the information change is large based on the texture component is determined so that the data processing unit is small, and the portion where the information change is small can be determined so that the data processing unit is large.

The motion estimation unit 1220 and the motion compensation unit 1225 can perform motion estimation and motion compensation using the data processing unit determined by the data processing unit determination unit 1212, respectively.

If the texture characteristic detected by the texture characteristic information detector 1210 according to the second embodiment is an edge histogram, the texture characteristic descriptor encoding unit 1215 according to the second embodiment uses the edge histogram information to calculate the edge histogram Data can be encoded. For example, the metadata about the edge histogram may be an edge historam descriptor in the MPEG-7 standard compression standard environment.

Or, if the texture characteristic detected by the texture characteristic information detecting unit 1210 according to the second embodiment is edge directionality, regularity, and coarseness, the texture characteristic descriptor encoding unit 1215 according to the second embodiment may determine Information can be used to encode metadata for texture browsing. For example, the metadata for texture browsing may be a texture browsing descriptor in the MPEG-7 standard compression specification environment.

Alternatively, when the texture characteristic information detected by the texture characteristic information detector 1210 according to the second embodiment is uniform, the texture characteristic descriptor coding unit 1215 according to the second embodiment uses the uniformity information, The metadata can be encoded. For example, the metadata about texture uniformity may be a homogeous texture descriptor in the MPEG-7 standard compression standard environment.

The metadata about the edge histogram, the metadata for texture browsing, and the metadata about the texture uniformity correspond to descriptors for information management and retrieval of multimedia contents.

The texture property descriptor encoded by the texture property descriptor encoding unit 1215 may be included in the bitstream 1265 as encoded multimedia data. Or may be output as a bitstream different from the encoded multimedia data.

The input sequence 505 corresponds to the image input through the input unit 110 and the characteristic information detection unit 120 and the texture characteristic information detection unit 1210 correspond to each other And the encoding method determination unit 130 and the data processing unit determination unit 1212 may correspond to each other. The multimedia data encoding unit 140 includes a motion estimation unit 1220, a motion compensation unit 1225, an intra prediction unit 530, a frequency transformation unit 540, a quantization unit 550, an entropy encoding unit 560, The frequency transforming unit 570, the deblocking filtering unit 580, and the buffer 590, respectively.

Since motion estimation or motion compensation for a current image is performed using a predetermined data processing unit based on texture characteristics without attempting to perform rate distortion optimization (RDO) on all kinds of data processing units, Can be reduced.

FIG. 13 is a block diagram of a multimedia decoding apparatus based on texture characteristics of a multimedia according to a second embodiment of the present invention.

The multimedia decoding apparatus 1300 according to the second embodiment includes a texture characteristic information extracting unit 1310, a data processing unit determining unit 1312, an entropy decoding unit 620, an inverse quantization unit 630, A motion estimation unit 1350, a motion compensation unit 1355, an intra prediction unit 660, a deblocking filtering unit 670, and a buffer 680.

The overall decoding process of the multimedia decoding apparatus 1300 according to the second embodiment is for generating a reconstructed image using the encoded multimedia data of the input bitstream 605 and all the information about the multimedia data.

The multimedia decoding apparatus 1300 according to the second embodiment further includes a texture characteristic information extracting unit 1310 and a data processing unit determining unit 1312 in comparison with the conventional video decoding apparatus 400. [ The operation of the motion estimation unit 1350 and the motion compensation unit 1355 using the data processing unit determined by the data processing unit determination unit 1312 is the same as that of the conventional video decoding apparatus 400 using the data processing unit by the error rate optimization The motion estimation unit 450 and the motion compensation unit 455 of FIG.

The texture property information extracting unit 1310 according to the second embodiment can extract texture property information using the texture property descriptor classified from the input bit stream 1305. [ For example, if the texture property descriptor is any of the metadata relating to the edge histogram, the metadata for texture browsing, and the metadata about the texture uniformity, the edge histogram, edge directionality, regularity, density, Can be extracted.

For example, in the MEPG-7 standard compression standard environment, the metadata about the edge histogram, the metadata for texture browsing, and the metadata about the texture uniformity may be each an edge histogram descriptor, a texture browsing descriptor, and an even texture descriptor.

The data processing unit determination unit 1312 can determine the size of the data processing unit for motion estimation of the image data using the texture property extracted by the texture property information extraction unit 1310. [ For example, by using uniformity, smoothness, regularity, and the like among the texture characteristics, the uniformity of the texture of the image data, the smoother the image data, or the more regular the pattern, the larger the data processing unit can be determined. Therefore, the portion where the information change is large based on the texture component is determined so that the data processing unit is small, and the portion where the information change is small can be determined so that the data processing unit is large.

The motion estimation unit 1350 and the motion compensation unit 1355 can perform motion estimation and motion compensation using the data processing unit determined by the data processing unit determination unit 1312. [

In comparison with the multimedia decoding apparatus 200 according to the embodiment, the input bitstream 1305 corresponds to the bitstream input through the receiver 210, and the characteristic information extraction unit 220 and the texture characteristic information extraction unit 1310 correspond to each other, and the decoding method determination unit 230 and the data processing unit determination unit 1312 can correspond to each other. The multimedia data decoding unit 240 includes a motion estimation unit 1350, a motion compensation unit 1355, an intra prediction unit 660, an inverse frequency transformation unit 640, an inverse quantization unit 630, an entropy decoding unit 620, The deblocking filtering unit 670, and the buffer 680, respectively.

There is a need for a method and apparatus for performing motion estimation or motion compensation on a current image using a predetermined data processing unit based on a texture characteristic without having to attempt to optimize an error rate for all kinds of data processing units at an encoding end, The multimedia data can be decoded and restored.

Fig. 14 shows the types of estimation modes used in the conventional video coding method.

In a conventional video coding method such as H.264, a 16 × 16 block 1400 for intra prediction, a 16 × 16 block 1405 for a skip mode, a 16 × 16 block 1410 for inter prediction, An inter 16 × 8 block 1415, an inter 8 × 16 block 1420, an inter 8 × 8 block 1425, and the like can be used. (For convenience of explanation, an M × N block for intraprediction is an 'intra M × N block', an M × N block for inter prediction is an 'inter M × N block', and an M × N Block is referred to as a " skipped MxN block "). The frequency conversion for a macroblock can be performed in units of 8x8 or 4x4 blocks.

Each of the macroblocks includes a skip 8x8 sub-block 1430, an inter 8x8 sub-block 1435, an inter 8x4 sub-block 1440, an inter 4x8 sub-block 1445, 4 sub-block 1450. The sub- The frequency conversion for the subblocks may be performed in units of 4x4 blocks.

The conventional video coding method uses the blocks 1400, 1405, 1410, 1415, 1420, 1425, 1430, 1435, 1440, 1445, 1450 shown in FIG. 14 to determine a block for motion estimation, After the optimization is attempted, the block with the lowest error rate is determined.

In general, a texture is complex and has a lot of detail, or an area where the boundary line of the object is located is selected as a small block size, and a smooth and edgeless area is selected as a large block size.

However, in the conventional video coding method, since the error rate optimization should be attempted for blocks of various sizes in all the prediction modes, the coding amount of operation is increased, and generally, additional overhead increases in order to express many kinds of block sizes .

15 shows types and groups of estimation modes available in the second embodiment of the present invention.

The multimedia encoding apparatus 1200 or the multimedia decoding apparatus 1300 according to the second embodiment introduces a larger data processing unit in addition to 16 × 16, 8 × 8, and 4 × 4.

For example, the multimedia encoding apparatus 1200 according to the second embodiment includes an intra 16x16 block 1505, a skip 16x16 block 1510, an inter 16x16 block 1515, Block 815, inter 8 × 16 block 1530, inter 8 × 8 block 1535, skip 8 × 8 sub block 1540, inter 8 × 8 sub block 1545, inter 8 × 4 sub block 1550 The inter 32 × 32 block 1480 and the inter 32 × 16 block 1485 as well as the inter 4 × 8 sub block 1555 and the inter 4 × 4 sub block 1560 as well as the inter 32 × 32 block 1475, The inter 16 × 32 block 1490, and the inter 16 × 16 block 1495 to perform motion estimation.

The frequency conversion unit for the skip 32x32 block 1475, the inter 32x32 block 1480, the inter 32x16 block 1485, the inter 16x32 block 1490, and the inter 16x16 block 1495 is 16x16 blocks, 8x8 blocks, and 4x4 blocks.

In the second embodiment, groups of data processing units can be classified to limit the group to be subjected to error rate optimization according to the texture characteristic. For example, an intra 16x16 block 1505, a skipped 16x16 block 1510, and an inter 16x16 block 1515 are included in the A group 1400. [ An inter 8 × 8 block 1525, an inter 8 × 16 block 1530, an inter 8 × 8 block 1535, a skip 8 × 8 sub block 1540, an inter 8 × 8 sub block 1545, 4 sub-block 1550, inter 4 × 8 sub-block 1555, and inter 4 × 4 sub-block 1560 are included in the B group 1420. In addition, the skip 32x32 block 1475, the inter 32x32 block 1480, the inter 32x16 block 1485, the inter 16x32 block 1490, and the inter 16x16 block 1495 are group C 1470).

The data processing unit determination units 1212 and 1312 according to the second embodiment increase the size of the data processing unit in the order of the B group 1420, the A group 1400, and the C group 1470. [

FIG. 16 illustrates a method of determining a data processing unit using a texture according to a second embodiment of the present invention.

When determining the data processing unit among the groups of the data processing units, the B group 1420, the A group 1400, and the C group 1470 shown in FIG. 15, the analysis of the texture components must be performed.

That is, the texture property detection unit 1210 analyzes the texture of the slice, and the texture property extraction unit 1310 analyzes the texture property descriptor for the slice, so that the texture information can be detected. For example, texture components can be defined as uniformity, normality, and stochasticity.

The data processing unit determination units 1212 and 1312 can determine an object of error rate optimization for the current slice as a large data processing unit when the texture for the current slice is defined as 'uniform'. For example, the optimal data processing unit for the current slice can be determined by attempting to optimize the error rate with data processing units in the A group 1400 and the C group 1470.

The data processing unit determination units 1212 and 1312 can determine an error rate optimization target for the current slice as a small data processing unit when the texture for the current slice is defined as 'irregular' or 'irregular'. For example, it is possible to determine an optimal data processing unit for a current slice by attempting to optimize the error rate with data processing units in the B group 1420 and the A group 1400.

Fig. 17 shows the kind of edge used as the texture characteristic according to the second embodiment of the present invention. Fig.

Of the texture properties, the types of edges can be distinguished by direction. For example, the directional orientation of the edge used in the edge histogram descriptor or texture browse descriptor may be a vertical edge 1710, a horizontal edge 1720, a 45 ° edge 1730, a 135 ° edge 1740, (1750). ≪ / RTI > Accordingly, the texture property detecting unit 1210 or the texture property extracting unit 1310 of the second embodiment can select the edge of the image data as one of the five-directional edges 1710, 1720, 1730, 1740, and 1750.

18 shows an edge histogram used as a texture characteristic according to a second embodiment of the present invention.

The edge histogram analyzes the edge components of the image region and determines the vertical direction edge 1710, the horizontal direction edge 1720, the 45 属 direction edge 1730, the 135 属 direction edge 1740 and the non-directional edge 1750, Defines the spatial distribution of the edges of the branches. A variety of histograms of semi-global or global patterns can be generated.

For example, the edge histogram 1820 represents the spatial distribution of the edges of the sub-image 1810 in the original image 1800. Thus, the five types of edges 1710, 1720, 1730, 1740, and 1750 of the sub-image 1810 have a vertical edge ratio 1821, a horizontal edge ratio 1823, a 45 ° edge ratio 1825 ), A 135 ° edge ratio (1827), and a non-directional edge ratio (1829).

Since the original image 1800 is divided into 16 sub-images and five edges are measured for each sub-image, 80 pieces of edge information can be extracted. Thus, the edge histogram descriptor for the current image contains 80 edge information, and the length of the histogram descriptor is 240 bits. If the spatial distribution of a predetermined edge is large due to the edge histogram, the corresponding region may be classified as a detail region, and if the spatial distribution of edges is small as a whole, the region may be classified as a smooth region .

In addition, the texture browsing descriptor describes the texture features that the image contains by quantifying the texture's regularity, directionality, and density in consideration of human visual characteristics. If the first value of the texture browse descriptor for the current area is large, it can be classified as a region having a more regular texture.

The uniform texture descriptor divides the frequency channel of the image into 30 channels using a Gabor filter and describes the uniform texture characteristics of the image using energy and energy standard deviation of each channel. If the energy of the equal texture component for the current area is large and the energy standard deviation is small, it can be classified into an equal area.

Therefore, the texture characteristic can be analyzed from the texture characteristic descriptor of the present invention, and the syntax indicating the data processing unit for motion estimation can be defined according to the degree of texture.

FIG. 19 is a flowchart illustrating a multimedia encoding method based on texture characteristics of multimedia according to a second embodiment of the present invention.

In step 1910, multimedia data is input.

In step 1920, the texture characteristic of the image data is detected as characteristic information for multimedia management or search. Texture properties can be defined by the directionality, density, smoothness, regularity, regularity, etc. of edges.

In step 1930, the size of the data processing unit for inter prediction may be determined based on the texture characteristic of the image data. In particular, an optimal data processing unit can be determined by performing error rate optimization only on the data processing units in a group that is classified by groups of data processing units and mapped. A data processing unit for intra prediction and skip mode as well as inter prediction may be determined.

In operation 1940, motion estimation and motion compensation are performed on the image data using the optimal data processing unit determined based on the texture characteristic. Encoding of image data is performed through intra prediction, frequency conversion, quantization, deblocking filtering, entropy encoding, and the like.

In the multimedia encoding apparatus 1200 and the multimedia encoding method according to the second embodiment, an optimal data processing unit for motion estimation may be determined using a texture property descriptor that provides a search and summary function of multimedia content information. Since the types of data processing units to be subjected to the error rate optimization (RDO) are limited, the syntax size for representing the data processing unit can be reduced, and the computation burden for optimizing the error rate can also be reduced.

FIG. 20 shows a flowchart of a multimedia decoding method based on texture properties of multimedia according to a second embodiment of the present invention.

In step 2010, a multimedia data bitstream is received. The bit stream may be parsed and coded into encoded data of multimedia and information data of multimedia.

In step 2020, the texture information of the image data may be extracted as the characteristic information for multimedia management or search. Characteristic information for multimedia management or retrieval can be extracted from a descriptor for management and retrieval of multimedia information based on multimedia content characteristics.

In step 2030, the size of the data processing unit for motion estimation may be determined based on the texture characteristic of the image data. In particular, data processing units for inter prediction can be classified into several groups according to their sizes. Different groups are mapped according to the texture level and the error rate optimization can be performed using only the data processing units in the group mapped to the texture level of the current image data. The data processing unit having the minimum error rate among the data processing units in the group can be determined as the optimum data processing unit.

Motion compensation, entropy decoding, inverse quantization, inverse frequency conversion, intra prediction, deblocking filtering, and the like using the optimal data processing unit and restored into multimedia data in step 2040.

The multimedia decoding apparatus 1300 or the multimedia decoding method according to the second embodiment reduces the computation burden of the error rate optimization for searching the optimum data processing unit using the descriptors available for information retrieval or summarization of the image content, The syntax size indicating the data processing unit can be reduced.

Hereinafter, a third embodiment of encoding or decoding multimedia data based on texture characteristics of image data will be described in detail with reference to FIGS. 21 to 29. FIG.

FIG. 21 is a block diagram of a multimedia encoding apparatus based on texture characteristics of a multimedia according to a third embodiment of the present invention.

The multimedia encoding apparatus 2100 according to the third embodiment includes a texture characteristic information detection unit 2110, an intra mode determination unit 2112, a motion estimation unit 520, a motion compensation unit 525, an intra prediction unit 2130, A frequency transformation unit 540, a quantization unit 550, an entropy coding unit 560, an inverse frequency transformation unit 570, a deblocking filtering unit 580, a buffer 590 and a texture property descriptor coding unit 2115 .

The overall encoding process of the multimedia encoding apparatus 2100 according to the third embodiment is similar to that of the first embodiment except that the overlapped data is omitted using the temporal similarity of successive images of the input sequence 505 and the spatial similarity within an image, 2165).

The multimedia encoding apparatus 2100 according to the third embodiment further includes a texture characteristic information detection unit 2110, an intra mode determination unit 2112 and a texture characteristic descriptor encoding unit 2115 in comparison with the conventional video encoding apparatus 300 . The operation of the intra prediction unit 2130 using the data processing unit determined by the intra mode determination unit 2112 is distinguished from the intra prediction unit 330 of the conventional video encoding device 300. [

The texture property information detector 2110 according to the third embodiment analyzes the input sequence 505 to extract a texture component. For example, the texture component may be uniformity, smoothness, regularity, edge directionality, density, and the like.

The intra mode determination unit 2112 can determine the size of the data processing unit for motion estimation of the image data using the texture property detected by the texture property information detection unit 2110. [ The data processing unit may be a rectangular block.

For example, the intra mode determination unit 2112 can determine the type and direction of the intra prediction mode that can be performed on the current image data, based on the distribution of the edge direction among the texture characteristics of the image data.

In particular, the priority can be determined according to the type and direction of the intra prediction mode that can be performed. The intra mode determination unit 2112 can generate an intra prediction mode table in which priority is assigned in order of principal edge directions based on the spatial distribution of edges in five directions.

The intra prediction unit 2130 can perform intra prediction using the intra prediction mode determined by the intra mode determination unit 2112. [

When the texture characteristic detected by the texture characteristic information detector 2110 according to the third embodiment is an edge histogram, the texture characteristic descriptor coding unit 2115 according to the third embodiment uses the edge histogram information to calculate the edge histogram Data can be encoded. Alternatively, when the texture characteristic detected by the texture characteristic information detector 2110 according to the third embodiment is the edge direction, the texture characteristic descriptor coding unit 2115 according to the third embodiment uses texture information for texture browsing Metadata or metadata about texture uniformity can be encoded.

For example, in the MEPG-7 standard compression standard environment, the metadata about the edge histogram, the metadata for texture browsing, and the metadata about the texture uniformity may each be an edge histogram descriptor, a texture browsing descriptor, and an even texture descriptor.

The metadata about the edge histogram, the metadata for texture browsing, and the metadata about texture uniformity correspond to descriptors for information management and retrieval of multimedia contents, respectively.

The texture property descriptor encoded by the texture property descriptor encoding unit 2115 may be included in the bitstream 2165 as encoded multimedia data. Or may be output as a bitstream different from the encoded multimedia data.

The input sequence 505 corresponds to an image input through the input unit 110 and the characteristic information detection unit 120 and the texture characteristic information detection unit 2110 correspond to each other And the encoding mode determination unit 130 and the intra mode determination unit 2112 may correspond to each other. The multimedia data encoding unit 140 includes a motion estimation unit 520, a motion compensation unit 525, an intra prediction unit 2130, a frequency conversion unit 540, a quantization unit 550, an entropy encoding unit 560, The frequency transforming unit 570, the deblocking filtering unit 580, and the buffer 590, respectively.

It is possible to reduce the amount of coding operations because intraprediction is performed on the current image using a predetermined intra prediction mode based on the texture characteristic without having to try intra prediction for all edge directions.

FIG. 22 is a block diagram of a multimedia decoding apparatus based on texture characteristics of multimedia according to a third embodiment of the present invention.

The multimedia decoding apparatus 2200 according to the third embodiment includes a texture characteristic information extracting unit 2210, an intra mode determining unit 2212, an entropy decoding unit 620, an inverse quantization unit 630, an inverse frequency transform unit 640 A motion estimation unit 650, a motion compensation unit 655, an intra prediction unit 660, a deblocking filtering unit 670, and a buffer 680.

The overall decoding process of the multimedia decoding apparatus 2200 according to the third embodiment is for generating a reconstructed image using the encoded multimedia data of the input bitstream 2205 and all the information about the multimedia data.

The multimedia decoding apparatus 2200 according to the third embodiment further includes a texture characteristic information extracting unit 2210 and an intra mode determining unit 2212 in comparison with the conventional video decoding apparatus 400. The operation of the intra prediction unit 2260 using the intra prediction mode determined by the intra mode determination unit 2212 can be distinguished from the intra prediction unit 460 of the conventional video decoding apparatus 400. [

The texture property information extracting unit 2210 may extract texture property information using the texture property descriptor classified from the input bitstream 2205. [ For example, if the texture property descriptor is any one of metadata relating to the edge histogram, metadata for texture browsing, and metadata about texture uniformity, an edge histogram, edge directionality, and the like can be extracted as texture properties.

For example, in the MPEG-7 standard compression standard environment, the metadata about the edge histogram, the metadata for texture browsing, and the metadata about the texture uniformity may be each an edge histogram descriptor, a texture browsing descriptor, and an even texture descriptor.

The intra mode determination unit 2212 can determine the type and direction of the intra prediction mode for intraprediction of the image data using the texture property extracted by the texture property information extraction unit 2210. In particular, the priority can be determined according to the type and direction of the intra prediction mode that can be performed. The intra mode determination unit 2212 can generate an intra prediction mode table in which priority is assigned in order of principal edge directions based on the spatial distribution of edges in five directions.

The intra prediction unit 2260 can perform intra prediction on the image data using the intra prediction mode determined by the intra mode determination unit 2212. [

The input bit stream 2205 corresponds to the bit stream input through the receiver 210 and the characteristic information extracting unit 220 and the texture characteristic information extracting unit 220 2210 correspond to each other, and the decoding method determining unit 230 and the intra mode determining unit 2212 can correspond to each other. The multimedia data decoding unit 240 includes a motion estimation unit 650, a motion compensation unit 655, an intra prediction unit 660, an inverse frequency transformation unit 640, an inverse quantization unit 630, an entropy decoding unit 620, The deblocking filtering unit 670, and the buffer 680, respectively.

The intra prediction is performed on the current image using a predetermined intra prediction mode based on the texture characteristic without performing intra prediction according to all kinds and directions of the intra prediction mode and the multimedia data is decoded with respect to the encoded bit stream Can be restored. Therefore, it is not necessary to perform intra-prediction according to all types and directions of the intra-prediction mode. Therefore, the burden on the calculation amount for intra-prediction can be reduced, and since the descriptor for the information search function is used without separately detecting the content characteristic, There is no need to provide a separate bit for the property.

23 shows the relationship between the original image, the sub-image, and the image block.

The original image 2300 is divided into 16 sub-images. (n, m) denotes a sub-image of the n-th row and the m-th column. The encoding of the original image 2300 may be performed along a scan sequence 2350 for the sub-images. Also, the sub-image 2310 is divided into blocks, such as the image block 2320.

The edge analysis for the original image 2300 is for detecting the edge characteristic for each sub-image, and the edge characteristic of the sub-image can be defined by the direction and intensity of the edge for each block in the sub-image.

24 shows the semantics of the edge histogram descriptor of the sub-image.

The semantics of the edge histogram descriptor for the original image 2300 represents the intensity of the edge by edge direction for each sub-image. Here, the histogram bin 'Local_Edge [n]' represents the edge intensity of the nth bin. n is an index indicating edges in five directions for every 16 sub-images, and is an integer from 0 to 79. That is, a total of 80 histogram bins are defined for the original image 2300.

'Local_Edge [n]' is the intensity of the five edges of the sub-picture according to the scan sequence 2350 for the original image 2300 in order. Local_Edge [1] ',' Local_Edge [2] ',' Local_Edge [3] ', and' Local_Edge [4] 'are examples of sub- Represents the intensity of the vertical direction, horizontal direction, 45 ° direction, 135 ° direction, and non-directional edge of the sub image at (0, 0) position, respectively.

The edge histogram descriptor can be represented by a total of 240 bits since the intensity of the edge is assigned 3 bits for every 80 histogram bins.

25 shows a table of the intra prediction mode of the conventional video coding method.

The intra prediction mode table of the conventional video coding scheme allocates a prediction mode number for every intra prediction direction. 1, 2, 3 and 4 for the vertical direction, horizontal direction, DC (direct current), left lower direction, right lower direction, vertical right direction, horizontal lower direction, , 5, 6, 7 and 8 are assigned.

The type of the intra-prediction mode depends on whether the DC value of the corresponding region is used for prediction, and the direction of the intra-prediction mode indicates the direction in which the surrounding reference region is located.

26 shows the direction of the intra prediction mode of the conventional video coding method.

The intra prediction can predict the pixel value of the current area using the pixel value of the surrounding area in the intra prediction direction corresponding to the prediction mode number. That is, depending on the type and direction of the intra prediction mode, a peripheral area in the vertical direction (0), a peripheral area in the horizontal direction (1), a peripheral area in the direct current (DC) One of the peripheral region of the lower end direction 4, the peripheral region of the vertical right direction 5, the peripheral region of the horizontal lower end direction 6, the peripheral region of the vertical left direction 7 and the peripheral region of the horizontal upper direction 8 The current area can be predicted.

FIG. 27 shows a table of intra-prediction modes reconstructed according to the third embodiment of the present invention.

The intra mode determination units 2112 and 2212 according to the third embodiment can determine an intra prediction mode that can be performed based on the texture component of the current image data. For example, the type of intra prediction direction or intra prediction mode that can be performed based on edge directionality among texture components can be determined.

The intra mode determination units 2112 and 2212 according to the third embodiment can reconstruct the intra prediction mode table using the intra prediction direction or the type of the intra prediction mode that can be performed. For example, it is possible to detect at least one major edge direction using the texture characteristic of the current image data, and select a type of the corresponding intra prediction mode and an intra prediction mode in which only the intra prediction direction can be performed. Accordingly, it is possible to reduce the amount of computation to perform intra prediction for each intra prediction direction and type.

In addition, the intra mode determination units 2112 and 2212 according to the third embodiment may include only intra prediction modes that can be performed in the intra prediction mode table. The higher the priority of the intra prediction mode or the type of the intra prediction mode table, the higher the probability of being adopted as the optimal intra prediction mode. Accordingly, the intra mode determination units 2112 and 2212 according to the third embodiment allocate the intra prediction numbers or the intra prediction numbers corresponding to the types to the lower number (higher precedence number) as the distribution direction is more edge direction The priority of the intra prediction mode table can be adjusted.

As a result of analyzing the edge histogram of the current area, the distribution of the vertical direction, horizontal direction, 45 ° direction, 135 ° direction and non-directional edge is 30%, 10%, 0% , 0%, and 60%, respectively. Accordingly, when the intra-prediction mode table is reconstructed, DC, which is the intra-prediction direction corresponding to the non-directional edge, is assigned the smallest intra prediction number 0 with the highest priority. The vertical direction and horizontal direction intraprediction directions are respectively selected for the vertical direction and the horizontal direction, which are widely distributed in the current area in the next order, and the intra prediction numbers are assigned to the intra prediction numbers 1 and 2, respectively.

FIG. 28 shows a flowchart of a multimedia encoding method based on texture characteristics of multimedia according to a third embodiment of the present invention.

In step 2810, multimedia data is input.

In step 2820, the texture characteristic of the image data is detected as characteristic information for multimedia management or search. Texture properties can be defined by edge orientation, edge histogram, and the like.

In step 2830, the intra prediction direction for intra prediction may be determined based on the texture characteristics of the image data. In particular, it is possible to include only the type and direction of the intra prediction mode that can be performed in the intra prediction mode table, and to adjust the priority between the types and directions of the intra prediction modes that can be performed.

In step 2840, intra prediction is performed on the image data using the optimal intra prediction mode determined based on the texture property. Encoding of image data is performed through motion estimation, motion compensation, frequency conversion, quantization, deblocking filtering, entropy encoding, and the like.

In the multimedia encoding apparatus 2100 and the multimedia encoding method according to the third embodiment, a direction and a type of an optimal intra prediction mode for intra prediction are determined using a texture property descriptor that provides a search and summary function of multimedia content information . Since the number of intra prediction modes to be intrapredicted is limited in order to determine the optimal intra prediction mode, the syntax size for indicating the data processing unit can be reduced and the calculation burden can also be reduced.

FIG. 29 shows a flowchart of a multimedia decoding method based on the texture characteristic of multimedia according to the third embodiment of the present invention.

In step 2910, a multimedia data bitstream is received. The bit stream may be parsed and coded into encoded data of multimedia and information data of multimedia.

In step 2920, texture information of the image data may be extracted as characteristic information for management or search of multimedia. Characteristic information for multimedia management or retrieval can be extracted from a descriptor for management and retrieval of multimedia information based on multimedia content characteristics.

In step 2930, the direction and type of intra prediction for intra prediction may be determined based on the texture characteristics of the image data. In particular, only the type and direction of the intra prediction mode that can be performed in the intra prediction mode table are included, and the priority between the types and directions of the intra prediction modes that can be performed can be changed.

In step 2940, it can be decoded through intra prediction using the optimal intra prediction mode, motion estimation, motion compensation, entropy decoding, inverse quantization, inverse frequency conversion, deblocking filtering, and the like, and restored into multimedia data.

The multimedia decoding apparatus 2200 or the multimedia decoding method according to the third embodiment reduces the computation burden of the intra prediction for searching for the optimal intra prediction mode using the descriptors available for information retrieval or summarization of the image content, The syntax size indicating all possible intra prediction modes can be reduced.

Hereinafter, a fourth embodiment of encoding or decoding multimedia data based on the temporal characteristics of sound data will be described in detail with reference to FIGS. 30 to 35. FIG.

FIG. 30 shows a block diagram of a multimedia encoding apparatus based on the fastness characteristic of multimedia according to a fourth embodiment of the present invention.

The multimedia encoding apparatus 3000 according to the fourth embodiment includes a fastness characteristic detector 3010, a window length determiner 3020, an acoustic encoder 3030, and a fast characteristic descriptor encoder 3040.

The overall encoding process of the multimedia encoding apparatus 3000 according to the fourth embodiment is to generate the encoded bit stream 3095 by omitting redundant data using the temporal similarity of continuous signals of the input signal 3005 to be.

The fastness characteristic detector 3010 according to the fourth embodiment analyzes the input information 3005 to extract a fast component. For example, the speed component may be a tempo or the like. Tempo is a term used in structured audio among MPEG audio, and represents a proportional variable indicating a relationship between score time and absolute time. A larger tempo means faster, and 120 bits per minute means twice as fast as 60 bits.

The window length determiner 3020 can determine a data processing unit for frequency conversion using the fastness characteristic detected by the fastness characteristic detector 3010. [ The data processing unit may include a frame, a window, and the like, but a window is used for convenience of explanation.

In addition, the window length determiner 3020 can determine the length or the weight of the window in consideration of the fastness characteristic. For example, the window length determination unit 3020 may determine that the window length is short if the tempo of the current acoustic data is fast, and may determine that the window length is long if the tempo is slow.

If the speed information extracted by the speed characteristic detector 3010 is not valid information, the window length determiner 3020 can determine a window of a fixed length and type. For example, when the input signal 3005 is a natural sound signal, since constant speed information can not be extracted, a natural sound signal can be encoded using a fixed window.

The audio encoding unit 3030 can perform frequency conversion of the audio data using the window determined by the window length determination unit 3020. [ The frequency-converted sound data is encoded through quantization and the like. For example, in the MPEG-7 standard compression environment, the metadata about the audio tempo may be an audio tempo descriptor.

In the case where the fastness characteristic detected by the fastness characteristic detector 3010 according to the fourth embodiment is a tempo, the fastness characteristic descriptor coding unit 3040 according to the fourth embodiment uses the tempo information to determine Metadata, semantic description information, side information, and the like.

The fast feature descriptor encoded by the fast feature descriptor encoding unit 3040 may be included in the bit stream 3095 as encoded multimedia data. Or may be output as a bitstream different from the encoded multimedia data.

The input signal 3005 and the signals input to the input unit 110 correspond to each other and the characteristic information detector 120 and the fastness characteristic detector 3010 correspond to each other And the coding mode determination unit 130 and the window length determination unit 3020 may correspond to each other. The multimedia data encoding unit 140 may correspond to the audio encoding unit 3030.

Therefore, the multimedia encoding apparatus 3000 according to the fourth embodiment determines the window length to be used for frequency conversion for encoding the sound data by using the extracted fastness characteristics for information management or search of the sound data, It is possible to encode the acoustic data so that more detailed information can be recorded with a smaller number of bits in consideration of the fastness property.

In addition, a separate process is not required to detect the fastness property of the sound data, but effective data encoding is possible because the detected information is used to generate a descriptor for retrieving the content information.

31 shows a block diagram of a multimedia decoding apparatus based on the fast paced characteristic of multimedia according to a fourth embodiment of the present invention.

The multimedia decoding apparatus 3100 according to the fourth embodiment includes a speed characteristic extracting unit 3110, a window length determining unit 3120, a sound decoding unit 3130 and an audio decoding unit 3130.

The overall decoding process of the multimedia decoding apparatus 3100 according to the fourth embodiment is to generate a reconstructed sound 3195 using the encoded information of the input bitstream 3105 and all the information about the sound data.

The fastness characteristic extraction unit 3110 according to an embodiment can extract the fastness characteristic information using the fastness characteristic descriptor classified from the input bitstream 3105. [ For example, the tempo information and the like can be extracted as the temporal characteristics if the temporal characteristic descriptor is any one of metadata related to the audio tempo, semantic attribute information, and side information. The metadata about the audio tempo may be an audio tempo descriptor in the MPEG-7 standard compression standard environment.

The window length determination unit 3120 can determine a window for frequency conversion using the speed characteristics extracted by the speed information extraction unit 2210. The window length determination unit 3120 can determine the length of the window, the shape of the window, and the like. The window length means the number of coefficients included in the window. The window type may be a symmetric window, an asymmetric window, or the like.

The sound decoding unit 3130 can decode the input bit stream 3105 and generate the restored sound 3195 while performing inverse frequency conversion using the window determined by the window length determining unit 3120. [

In comparison with the multimedia decoding apparatus 200 according to the embodiment, the input bitstream 3105 corresponds to the bitstream input through the receiver 210, and the characteristic information extractor 220 and the speed characteristic information extractor 3110 correspond to each other, and the decoding method determining unit 230 and the window length determining unit 3120 can correspond to each other. The sound decoding unit 3130 and the multimedia data decoding unit 240 may correspond to each other.

Since the window for frequency conversion is determined considering the fastness of the sound data, it is possible to effectively recover the sound data. By extracting the content characteristic from the descriptor for information retrieval instead of extracting the separate attribute information, can do.

32 shows a window table used in the conventional audio encoding method.

Since a similar pattern is repeated in the acoustic signal, it is advantageous to perform a predetermined signal processing by converting the acoustic signal into the frequency domain as compared with the operation in the time domain. To convert a sound signal into a frequency domain, the data is divided into certain units, which are called frames or windows. Since the length of the frame or window determines the resolution of the time domain or the frequency domain, the optimal frame or window length should be selected in consideration of the characteristics of the input signal in encoding / decoding efficiency.

The table shown in FIG. 32 shows a window type of AAC (Advanced Audio Coding) which is one of typical audio codecs. There are two types of window lengths, such as window lengths including 1024 coefficients, such as windows 3210, 3230, and 3240, and window lengths including 128 coefficients, such as window 3220. [

In the window type, the symmetric window includes a window 3210 having a window length of 'LONG_WINDOW' including 1024 coefficients and a window 3220 having a window length of 'SHORT_WINDOW' including 128 coefficients. In the asymmetric window, there is a long 'LONG_START_WINDOW' 3230 in which the window introduction portion is long and a 'LONG_STOP_WINDOW' 3240 in which the window end portion is long.

For a steady-state signal, a window 3210 of 'LONG_WINDOW' may be applied to have a higher frequency resolution, and in the case of a signal with a rapid change or a sudden change such as an impulse signal, Quot; SHORT_WINDOW '< / RTI > so that the change over time can be better expressed.

When the window length is long like the window 3210, since a signal is displayed using a large number of basis upon frequency conversion, a fine signal change in the frequency domain can be expressed. However, in the case of a window with a long window length, since it can not express a temporal change in the same window, a rapid change signal in the window can not be appropriately represented, resulting in distortion such as a pre-echo phenomenon.

When the window length is short like the window 3220, the change over time can be effectively expressed. However, in the case of applying a window having a short window length to a stable signal, the coding efficiency may be lowered because it does not properly reflect the similarity between windows and expresses a signal repeatedly on several windows.

33 shows a relationship in which the length of the window is adjusted based on the tempo information of sound, according to the fourth embodiment of the present invention.

The window length determiners 3020 and 3120 according to the fourth embodiment determine the window length based on the speed characteristics. In consideration of the tempo information or the beats per minute (BPM) information, the window length determiners 3020 and 3120 generate a large number of transition periods in the same interval of the sound data having the fast tempo, A window of a short length is selected. In addition, the window length determiners 3020 and 3120 select a window of a long length for frequency conversion of sound data since sound data of slow tempo occurs relatively infrequently in the same section.

For example, as shown in the diagram of FIG. 33, the tempo increases toward largo, larghetto, adagio, andante, moderato, allegro and presto As the BPM increases and the window length increases, the window length can be determined to decrease stepwise.

FIG. 34 shows a flowchart of a multimedia coding method based on the pacing characteristic of multimedia according to the fourth embodiment of the present invention.

In step 3410, multimedia data is input.

In step 3420, the fastness characteristic of the sound data is detected as characteristic information for multimedia management or search. The speed characteristic can be defined as a tempo, a BPM, or the like.

In step 3430, the window length for frequency conversion may be determined based on the fastness characteristics of the acoustic data. The window shape as well as the window length may be determined. A window having a relatively short length can be determined for fast sound data and a window having a relatively long length can be determined for slow sound data.

In step 3440, frequency conversion is performed on the acoustic data using the window determined based on the fastness characteristic. Encoding of sound data is performed through frequency conversion, quantization and the like.

In the multimedia encoding apparatus 3000 and the multimedia encoding method according to the fourth embodiment, a window length for frequency conversion can be determined using a fast feature descriptor that provides a search and summary function of multimedia content information. More accurate and efficient encoding becomes possible by selecting the window considering the speed of sound data.

35 shows a flowchart of a multimedia decoding method based on the pacing characteristic of multimedia according to the fourth embodiment of the present invention.

In step 3510, a multimedia data bitstream is received. The bit stream may be parsed and coded into encoded data of multimedia and information data of multimedia.

In step 3520, speed information of the sound data may be extracted as characteristic information for management or search of multimedia. Characteristic information for multimedia management or retrieval can be extracted from a descriptor for management and retrieval of multimedia information based on multimedia content characteristics.

In step 3530, the window length for frequency conversion may be determined based on the fastness characteristics of the acoustic data. The window length and shape may be determined. The faster the sound data is, the shorter the window is determined, and the longer the sound data is, the longer the window can be determined.

In step 3540, the data may be decoded through frequency conversion using an optimal length window, inverse quantization, and the like to be reconstructed as acoustic data.

The multimedia decoding apparatus 3100 or the multimedia decoding method according to the fourth embodiment can optimize the amount of frequency conversion computation by searching for an optimal length window using a descriptor available for information retrieval or summarization of the audio content, The signal change in the window can be represented more accurately.

FIG. 36 shows a flowchart of a multimedia encoding method based on content characteristics of multimedia according to an embodiment of the present invention.

In step 3610, multimedia data is input. The multimedia data may include image data, sound data, and the like.

In step 3620, the inputted multimedia data is analyzed, and characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content is detected. The predetermined characteristic of the multimedia content may include a color characteristic of the image data, a texture characteristic of the image data, a speed characteristic of the sound data, and the like. For example, the color characteristics of the image data may include a color layout of the image, a color histogram, and the like. The texture properties of the image data may include uniformity, smoothness, regularity, edge directionality, and density of the image texture. For example, the temporal characteristics of the sound data may include tempo information of sound, and the like.

In step 3630, the encoding scheme based on the characteristics of the multimedia is determined using the characteristic information for multimedia management or search. For example, the compensation value for the luminance change amount can be determined based on the color characteristic of the image data. The size and estimation mode of the data processing unit used in the inter prediction can be determined according to the texture characteristic of the image data. In addition, the type and direction of intra prediction that can be used can be determined according to the texture characteristic of the video data. The length of the window for frequency conversion can be determined according to the speed characteristics of the sound data.

In step 3640, the multimedia data is encoded according to a coding scheme based on the characteristics of the multimedia. The encoded multimedia data may be output in the form of a bit stream. The multimedia data can be encoded by performing operations such as motion estimation, motion compensation, intra prediction, frequency conversion, quantization, and entropy encoding.

At least one of motion estimation, motion compensation, intra prediction, frequency conversion, quantization, and entropy encoding may be performed according to a coding scheme determined in consideration of multimedia content characteristics. For example, if the compensation value of the luminance change amount is determined using the color characteristic, the luminance change amount can be compensated for the image data after motion compensation. In addition, inter prediction or intra prediction can be performed based on the inter prediction mode or the intra prediction mode determined using the texture characteristic. Also, the frequency conversion can be performed using the window length determined using the acoustic velocity characteristic.

The multimedia encoding method according to an exemplary embodiment may encode characteristic information for multimedia management or search into a multimedia content descriptor. For example, the color characteristic of the image data may be encoded into at least one of metadata relating to the color layout, metadata relating to the color structure, and metadata relating to the hierarchical color. The texture properties of the image data may be encoded with at least one of metadata relating to edge histograms, metadata for texture browsing, and metadata about texture uniformity. The speed characteristics of the sound data can be encoded into at least one of metadata relating to the audio tempo, semantic attribute information, and side information.

37 is a flowchart illustrating a multimedia decoding method based on content characteristics of multimedia according to an exemplary embodiment of the present invention.

In step 3710, the multimedia data bitstream is received and parsed to be categorized into information about multimedia encoded data and multimedia. Multimedia can include all kinds of data such as video and sound. The information on the multimedia may include metadata, a content property descriptor, and the like.

At step 3720, characteristic information for management or retrieval of multimedia is extracted from the encoded data of multimedia and the information about multimedia. Characteristic information for multimedia management or retrieval can be extracted from descriptors for management and retrieval based on the content characteristics of the multimedia.

For example, the color characteristics of the image data can be extracted from at least one of metadata relating to the color layout, metadata relating to the color structure, and metadata relating to the hierarchical color. The texture properties of the image data can be extracted from at least one of metadata relating to the edge histogram, metadata for texture browsing, and metadata about texture uniformity. The speed characteristics of the sound data can be extracted from at least one of metadata relating to the audio tempo, semantic attribute information, and side information.

The color characteristics of the image data may include a color layout of the image, a color histogram, and the like. The texture properties of the image data may include uniformity, smoothness, regularity, edge directionality, and density of the image texture. The speed characteristics of the sound data may include tempo information of sound, and the like.

In step 3730, the decoding method based on the characteristics of the multimedia is determined using the characteristic information for multimedia management or search. For example, the compensation value for the luminance change amount can be determined based on the color characteristic of the image data. The size and estimation mode of the data processing unit used in the inter prediction can be determined according to the texture characteristic of the image data. In addition, the type and direction of intra prediction that can be used can be determined according to the texture characteristic of the video data. The length of the window for frequency conversion can be determined according to the speed characteristics of the sound data.

In step 3740, the encoded data of the multimedia is decoded. According to the decoding method based on the characteristics of the multimedia, the encoded data of the multimedia is decoded. Decoding of multimedia data involves operations such as motion estimation, motion compensation, intra prediction, inverse frequency conversion, inverse quantization and entropy decoding. Multimedia data may be decoded to restore the multimedia content.

The multimedia decoding method according to an exemplary embodiment may perform at least one of motion estimation, motion compensation, intra prediction, inverse frequency conversion, inverse quantization, and entropy decoding while considering multimedia content characteristics. For example, if the compensation value of the luminance change amount is determined using the color characteristic, the luminance change amount can be compensated for the image data after motion compensation. In addition, inter prediction or intra prediction can be performed based on the inter prediction mode or the intra prediction mode determined using the texture characteristic. In addition, the inverse frequency conversion can be performed using the determined window length using the acoustic velocity characteristic.

The above-described embodiments of the present invention can be embodied in a general-purpose digital computer that can be embodied as a program that can be executed by a computer and operates the program using a computer-readable recording medium. The computer readable recording medium may be a magnetic storage medium such as a ROM, a floppy disk, a hard disk, etc., an optical reading medium such as a CD-ROM or a DVD and a carrier wave such as the Internet Lt; / RTI > transmission).

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a block diagram of a multimedia encoding apparatus based on content characteristics of multimedia according to an embodiment of the present invention.

FIG. 2 illustrates a block diagram of a multimedia decoding apparatus based on content characteristics of multimedia according to an embodiment of the present invention.

3 shows a block diagram of a conventional video coding apparatus.

4 shows a block diagram of a conventional video decoding apparatus.

FIG. 5 is a block diagram of a multimedia encoding apparatus based on the color characteristics of multimedia according to the first embodiment of the present invention.

6 is a block diagram of a multimedia decoding apparatus based on color characteristics of multimedia according to a first embodiment of the present invention.

FIG. 7 shows the luminance variation between consecutive frames measured using color characteristics according to the first embodiment of the present invention.

8 shows a color histogram used as a color characteristic according to the first embodiment of the present invention.

Figure 9 shows a color layout used as color characteristics in accordance with a first embodiment of the present invention.

10 is a flowchart illustrating a multimedia encoding method based on color characteristics of multimedia according to the first embodiment of the present invention.

11 is a flowchart illustrating a multimedia decoding method based on color characteristics of multimedia according to the first embodiment of the present invention.

12 is a block diagram of a multimedia encoding apparatus based on texture characteristics of multimedia according to a second embodiment of the present invention.

FIG. 13 is a block diagram of a multimedia decoding apparatus based on texture characteristics of a multimedia according to a second embodiment of the present invention.

Fig. 14 shows the types of estimation modes used in the conventional video coding method.

15 shows types and groups of estimation modes available in the second embodiment of the present invention.

FIG. 16 illustrates a method of determining a data processing unit using a texture according to a second embodiment of the present invention.

Fig. 17 shows the kind of edge used as the texture characteristic according to the second embodiment of the present invention. Fig.

18 shows an edge histogram used as a texture characteristic according to a second embodiment of the present invention.

FIG. 19 is a flowchart illustrating a multimedia encoding method based on texture characteristics of multimedia according to a second embodiment of the present invention.

FIG. 20 shows a flowchart of a multimedia decoding method based on texture properties of multimedia according to a second embodiment of the present invention.

FIG. 21 is a block diagram of a multimedia encoding apparatus based on texture characteristics of a multimedia according to a third embodiment of the present invention.

FIG. 22 is a block diagram of a multimedia decoding apparatus based on texture characteristics of multimedia according to a third embodiment of the present invention.

23 shows the relationship between the original image, the sub-image, and the image block.

24 shows the semantics of the edge histogram descriptor of the sub-image.

25 shows a table of the intra prediction mode of the conventional video coding method.

26 shows the direction of the intra prediction mode of the conventional video coding method.

FIG. 27 shows a table of intra-prediction modes reconstructed according to the third embodiment of the present invention.

FIG. 28 shows a flowchart of a multimedia encoding method based on texture characteristics of multimedia according to a third embodiment of the present invention.

FIG. 29 shows a flowchart of a multimedia decoding method based on the texture characteristic of multimedia according to the third embodiment of the present invention.

FIG. 30 shows a block diagram of a multimedia encoding apparatus based on the fastness characteristic of multimedia according to a fourth embodiment of the present invention.

31 shows a block diagram of a multimedia decoding apparatus based on the fast paced characteristic of multimedia according to a fourth embodiment of the present invention.

32 shows a window table used in a conventional audio coding method.

33 shows a relationship in which the length of the window is adjusted based on the tempo information of sound, according to the fourth embodiment of the present invention.

FIG. 34 shows a flowchart of a multimedia coding method based on the pacing characteristic of multimedia according to the fourth embodiment of the present invention.

35 shows a flowchart of a multimedia decoding method based on the pacing characteristic of multimedia according to the fourth embodiment of the present invention.

FIG. 36 shows a flowchart of a multimedia encoding method based on content characteristics of multimedia according to an embodiment of the present invention.

37 is a flowchart illustrating a multimedia decoding method based on content characteristics of multimedia according to an exemplary embodiment of the present invention.

Claims (46)

A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, Wherein the color characteristic of the image data includes at least one of a color layout of an image and a cumulative distribution of each color bin. 2. The method of claim 1, Encoding the multimedia data according to a coding scheme based on the characteristics of the multimedia; And And generating a bitstream including the encoded multimedia data based on the content characteristic of the multimedia. 3. The method of claim 2, The multimedia encoding method further includes encoding the characteristic information for management or searching of the multimedia into a descriptor for managing or searching for multimedia based on the multimedia content, Wherein the bitstream generation step generates a bitstream including a descriptor for managing or searching multimedia encoded based on the encoded multimedia data and the multimedia content. delete delete A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The encoding method determination step may include: And measuring a change amount between a pixel value of the current image data and a pixel value of the reference image data using the color characteristic of the image data. 7. The method of claim 6, Further comprising compensating a pixel value of the current image data using a change amount between a pixel value of the current image data and a pixel value of the reference image data. 8. The method of claim 7, Further comprising encoding the current image data by compensating a pixel value of the current image data using a change amount between the pixel values after motion compensation is performed on the current image data, Multimedia encoding method based on content characteristics. A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, Wherein the texture property of the image data includes at least one of homogeneity, smoothness, regularity, edge direction, and density of the image texture. . A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The encoding method determination step may include: And determining a size of a data processing unit for motion estimation of current image data using texture characteristics of the image data. 11. The method of claim 10, Based on at least one of uniformity, smoothness and regularity among the texture characteristics of the image data, the size of the data processing unit is determined to be larger as the texture change of the image data is smaller. Multimedia encoding method. 11. The method of claim 10, Further comprising the step of performing motion estimation or motion compensation on the current image data using a data processing unit whose size is determined with respect to the image data. The multimedia encoding method according to claim 1, . 10. The method of claim 9, And determining an intra prediction mode that can be performed on the current image data using the texture characteristic of the image data. 14. The method of claim 13, And determining at least one of a type and a priority of an intra prediction mode that can be performed on the current image data by using an edge direction of texture characteristics of the image data. Multimedia coding method based on characteristics. 14. The method of claim 13, Further comprising performing motion estimation on the current image data using an intra prediction mode determined for the current image data. ≪ RTI ID = 0.0 > [10] < / RTI > A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; And And determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The encoding method determination step may include: And determining a length of a data processing unit for frequency transformation of current sound data using the fastness characteristic of the sound data. 17. The method of claim 16, Wherein the determining step further determines that the length of the data processing unit is shorter as the current sound data becomes faster based on the tempo information among the speed characteristics of the sound data. 18. The method of claim 17, And performing frequency conversion on the current sound data using a data processing unit whose length is determined for the sound data. A method of encoding multimedia, Receiving multimedia data; Analyzing the multimedia data to detect characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; Determining a coding scheme based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia; And A descriptor for managing or searching multimedia based on the multimedia content when the predetermined characteristic of the multimedia content is a color characteristic of the image data, the descriptor may include at least one of a metadata related to a color layout of the image data, ) And metadata on the scalable color Encoding the at least one of the metadata, A descriptor for managing or searching multimedia based on the multimedia content when the predetermined characteristic of the multimedia content is a texture characteristic of the image data, the metadata related to the edge histogram of the image data, the metadata for browsing and the metadata about texture homogeneity of texture, When the predetermined characteristic of the multimedia content is the fastness characteristic of the sound data, metadata related to the audio tempo, semantic description information, and side information The method according to claim 1, further comprising the step of encoding at least one of the multimedia contents. In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, Wherein the color characteristic of the image data includes at least one of a color layout of an image and a cumulative distribution by a bin color bin. 21. The method of claim 20, wherein the multimedia decoding method further comprises: Decoding the coded data of the multimedia according to a decoding method based on the characteristics of the multimedia; And And reconstructing the decoded multimedia data based on the decoded multimedia data. 21. The method according to claim 20, Parsing the bitstream to extract a descriptor for multimedia management or management based on the multimedia content; And And extracting the characteristic information from the descriptor. ≪ RTI ID = 0.0 > [10] < / RTI > delete delete In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The decoding method determination step may include: And measuring a change amount between the pixel value of the current image data and the reference image data using the color characteristic of the image data. The multimedia decoding method of claim 25, Performing motion compensation on the inverse frequency-converted current image data; And Compensating the pixel value of the motion-compensated current image data using the variation between the pixel value of the current image data and the pixel value of the reference data. . In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, Wherein the texture property of the image data includes at least one of uniformity, smoothness, regularity, edge directionality, and density of the image texture. In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The decoding method determination step may include: And determining a size of a data processing unit for motion estimation of the current image data using the texture property of the image data. 30. The method of claim 28, Wherein the size of the data processing unit is determined such that the smaller the texture change of the current image data is, based on at least one of texture uniformity, smoothness, and regularity among the texture characteristics of the image data, Based multimedia decoding method. 29. The method of claim 28, And performing motion estimation or motion compensation on the current image data using a data processing unit whose size is determined for the image data. In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The decoding method determination step may include: And determining an intra prediction mode that can be performed on the current image data using the texture property of the image data. 32. The method of claim 31, And determining at least one of a type and a priority of an intra prediction mode that can be performed on the current image data based on the directionality of an edge among the texture characteristics of the current image data. . 32. The method of claim 31, Further comprising performing motion estimation on the current image data using an intra prediction mode determined for the current image data. 27. The method of claim 26, The multimedia decoding method includes: Further comprising decoding the current image data by compensating a pixel value of the current image data using a change amount between the pixel values after motion compensation is performed on the current image data, A multimedia decoding method based on content characteristics. In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The decoding method determination step may include: And determining a length of a data processing unit for inverse frequency conversion of current sound data using the fastness characteristic of the sound data. The decoding method of claim 35, Wherein the determining step further determines that the length of the data processing unit is shorter as the current sound data is faster, based on the tempo information among the speed characteristics of the sound data. The multimedia decoding method of claim 35, And performing inverse frequency conversion on the current sound data using a data processing unit whose length is determined for the sound data. The multimedia decoding method according to claim 1, In a method for decoding multimedia, Receiving multimedia data bitstreams and parsing the bitstreams to classify encoded multimedia data and information about the multimedia; Extracting characteristic information for management or searching of the multimedia from the multimedia information; And And determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia, Wherein the characteristic information is at least one of a color characteristic of the image data, a texture characteristic of the image data, and a speed characteristic of the acoustic data, The characteristic information extracting step includes: Parsing the bitstream to extract metadata from the descriptor about color layout, metadata about color structure, metadata about hierarchical color, metadata about edge histogram, metadata for texture browsing, metadata about texture uniformity Extracting at least one of metadata related to the audio tempo, semantic attribute information, and side information; And Extracting a color characteristic of the image data from the extracted descriptor if the extracted descriptor is at least one of metadata relating to the color layout, metadata relating to the color structure, and metadata relating to the hierarchical color, Extracting a texture property of the image data from the extracted descriptor if the extracted descriptor is at least one of metadata relating to the edge histogram, metadata for texture browsing, and metadata related to the texture uniformity, If the extracted descriptor is at least one of metadata related to the audio tempo, the semantic attribute information, and the side information, extracting a speed characteristic of the sound data from the extracted descriptor To the content characteristics of Multimedia decoding method. An apparatus for encoding multimedia, An input unit for receiving multimedia data; A characteristic information detector for analyzing the multimedia data and detecting characteristic information for managing or searching multimedia based on a predetermined characteristic of the multimedia content; A coding mode determining unit for determining a coding mode based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia; A multimedia data encoding unit for encoding the multimedia data according to a coding scheme based on the characteristics of the multimedia; And And a descriptor coding unit for coding the characteristic information for management or searching of the multimedia into a descriptor for management or search of multimedia based on the multimedia content, Wherein the characteristic information includes at least one of a color characteristic of a video data, a texture characteristic, and a speed characteristic of audio data, and the descriptor for management or search of multimedia based on the multimedia content includes a meta At least one of the data, the metadata about the color structure and the metadata about the hierarchical color, the metadata about the edge histogram, the metadata for texture browsing, the metadata about the texture uniformity, and the speed characteristics of the acoustic data of the acoustic data And a multimedia encoding unit for decoding the multimedia content based on the content characteristic of the multimedia. delete delete A multimedia decoding apparatus comprising: A receiving unit for receiving the multimedia data bit stream and parsing the bit stream to classify the encoded data of the multimedia and the information about the multimedia; A characteristic information extracting unit for extracting characteristic information for managing or searching multimedia from information about the multimedia; A decoding method determining unit for determining a decoding method based on the characteristics of the multimedia using characteristic information for management or searching of the multimedia; A multimedia data decoding unit decoding the encoded data of the multimedia according to a decoding method based on the characteristics of the multimedia; And And a reconstruction unit for reconstructing the decoded multimedia data, Wherein the characteristic information extracting unit extracts a descriptor for management or searching of the multimedia by parsing the bit stream, extracts the characteristic information from the descriptor, Wherein the characteristic information includes at least one of a color characteristic and a texture characteristic of the image data, and a fast characteristic of the acoustic data, The descriptor for managing or searching multimedia based on the multimedia content may include at least one of metadata related to color layout of the image data, metadata relating to a color structure and metadata relating to a hierarchical color, metadata relating to an edge histogram, The metadata for the texture uniformity, and the temporal characteristics of the acoustic data of the acoustic data. The multimedia decoding apparatus according to claim 1, delete delete A computer-readable recording medium having embodied thereon a program for implementing a multimedia encoding method based on the content characteristic of the multimedia according to any one of claims 1 to 3 and 6 to 19. 38. A computer-readable recording medium having recorded thereon a program for implementing a multimedia decoding method based on the content characteristic of multimedia according to any one of claims 20 to 22, and 25 to 38.

Images