KR101397146B1 - Method and apparatus for encoding/decoding Multi-media data - Google Patents

Abstract

The present invention relates to a method and apparatus for encoding / decoding multimedia data including images, audio, text, and the like, the decoding method comprising: receiving multimedia data; The image display apparatus according to claim 1, further comprising: an image display section in which a plurality of image data are displayed, and an image display section in which at least two image data among a plurality of image data are simultaneously displayed, Uploading data to a memory; And reproducing the at least one image data uploaded to the memory using the animation effect information included in the animation area.

According to the method and apparatus for encoding / decoding multimedia data according to the present invention, it is possible to construct a slide show with only a small amount of multimedia data. As a result, it is possible to reduce the processing time and transmission time of the multimedia data.

Description

[0001] The present invention relates to a method and apparatus for encoding / decoding multi-media data,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for encoding / decoding multimedia data including images, audio, text, and the like, and more particularly to a method and apparatus for encoding / decoding multimedia data by sequentially reproducing a plurality of media data, ≪ / RTI >

In a multimedia data processing method for integrating and encoding various types of media data such as images, audio, text, and the like, time consumed in encoding / decoding and time consumed in data transmission are increased due to a large amount of data there is a problem.

SUMMARY OF THE INVENTION The present invention provides a multimedia data structure capable of efficiently encoding / decoding multimedia data, and a method and apparatus for encoding / decoding multimedia data using the multimedia data structure.

According to an aspect of the present invention, there is provided an encoding method including: generating a data region including a plurality of media data regions; Generating a media information area including a plurality of track areas corresponding to each of the plurality of media data areas; Generating an animation area having information on an animation effect to be applied to a plurality of image data included in the data area; And generating additional information having information on an image display section in which the plurality of image data is displayed and an image switching section in which two or more image data of the plurality of image data are simultaneously displayed, A plurality of media data included in the media data are dependent on one timeline.

According to an aspect of the present invention, there is provided a decoding method including: a data area including a plurality of media data areas; A media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area having information on an animation effect to be applied to a plurality of image data included in the data area; Wherein the image display section displays the image data of the plurality of image data and the image switching section in which two or more image data of the plurality of image data are simultaneously displayed, Uploading at least one image data to be stored in the memory; And reproducing the at least one image data uploaded to the memory using the animation effect information included in the animation area.

According to an aspect of the present invention, there is provided an encoding apparatus comprising: a data area generator for generating a data area including a plurality of media data areas; A media information area generation unit for generating a media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area generation unit for generating an animation area having information on an animation effect to be applied to a plurality of image data included in the data area; And an additional information generating unit for generating additional information having information on an image display section in which the plurality of image data are displayed and an image switching section in which two or more image data of the plurality of image data are simultaneously displayed, And the plurality of media data included in the data area is dependent on one timeline.

According to an aspect of the present invention, there is provided a decoding apparatus comprising: a data area including a plurality of media data areas; A media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area having information on an animation effect to be applied to a plurality of image data included in the data area; A memory for storing image data to be displayed; Wherein the image display section displays the image data of the plurality of image data and the image switching section in which two or more image data of the plurality of image data are simultaneously displayed, A memory controller for uploading at least one image data to be stored in the memory; And a playback unit that plays back at least one image data uploaded to the memory using animation effect information included in the animation region.

According to the method and apparatus for encoding / decoding multimedia data according to the present invention, it is possible to construct a slide show with only a small amount of multimedia data. As a result, it is possible to reduce the processing time and transmission time of the multimedia data.

Hereinafter, a method and apparatus for encoding / decoding multimedia data according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 schematically shows the overall structure of multimedia data according to the present invention. As shown in FIG. 1, a multimedia data file includes a file type area, a media information area, and a data area.

The file type field indicates the format of the multimedia data file and can be expressed in the form or version of the file. For example, if the file type is "mp42 ", it can indicate that the multimedia file format is the MPEG-4 version 2 format.

The data area includes a plurality of media data areas, and each of the media data areas includes media data. In FIG. 1, the data area includes first, second, and third media data areas, but may include four or more or two or less media data areas. The media data area may include various types of media data such as image data, audio data, or text data, and the image data may be still image data or moving image data.

The media information area has information on the media data included in the data area. Referring to FIG. 1, the media information area includes a plurality of track areas corresponding to a plurality of media data areas included in the data area. That is, the media information area includes a first track area having information on the media data included in the first media data area, a second track area having information on the media data included in the second media data area, And a second track area having information on the media data included in the area.

The track area included in the media information area may have timing information for constructing a slide show by sequentially reproducing the media data included in the corresponding media data area. For example, the first track area may have information on the playback period of the media data included in the first media data area.

For example, if the media data is music data, the track area corresponding to the media data may include the music information of the music data or the composer information.

FIG. 2 illustrates a first embodiment of the structure of multimedia data according to the present invention. The media information area may include track areas and meta areas corresponding to the media data areas, respectively, And may include a media area and a track meta area.

The meta area included in the media information area at the same level as the track areas includes information about media data included in the data area. And the meta area includes identification information of the media data and location information. More preferably, the meta area may include name of media data, content type information, ID, location information, and size information.

The following describes an example of the media data information included in the meta area. The first media data area includes N JPEG image data, the second media data area includes MP3 audio data, Is included in the text data.

Also, the meta area includes a first meta area and a second meta area, and the first and second meta areas may divide information on media data. For example, the first area may include the name of the media data and the content type, and the second area may include information on the physical location and size of the media data.

The handler type of the meta area can be specified as "lsr1 ".

Referring to FIG. 3, the meta area may include an animation area having information on an animation effect to be applied to media data. The animation area may include at least one of grouping information of an animation effect, size information of media data to be animated, opacity effect information, and geometric transformation effect information. The grouping information is information indicating a combination of animation effects to be applied to the media data, and the size information is information describing a change in size of the image in the case of image data.

By applying the animation effect, it is possible to realize more natural and interesting image conversion in the slide show configuration. For example, with the animation effect, the previously displayed image may disappear and the next image to be displayed may begin to appear.

The opacity effect information means information describing a fade-in or fade-out effect of the image. In addition, the geometric transformation information may be information describing effects such as transition between images, scale transformation of images, rotation, skew, and the like.

The animation area may include information on a motion path of an image or information on a motion path of objects included in the image, and may include color change information or shape information of an image. The shape information of the image may be a rectangle, a circle, an ellipse, a line, a polyline, a polygon, or the like. In addition, the animation area may include attribute control information for changing attributes of the media data for various animation effects applicable to the media data.

As shown in FIG. 4, the meta area may be located at a file level without being included in the media information area. That is, the multimedia data file may be composed of four areas: a file type area, a meta area, a media information area, and a data area.

The animation region may be a language such as Light-weigh Application Scene Representation (LASeR), Scalable Vector Graphics (SVG), or Binary Format for Scene (BIFS) for animation effect information technology. The LASeR, SVG, or BIFS may be implemented in XML (extensible Mark-up Language) format or Binary encoded format.

The following is an embodiment in which animation information is expressed using SVG, which is a representation of opacity effect information, scale conversion information, and size conversion information.

The symbol (<) means start and the symbol (>) means end. The symbol (/) means the end of the context to be defined. For example, if <svg> through </ svg> are a bundle and <g id = "Article_image"> through </ g> are a bundle, then <image width = "200" height = "150" # sample = 1> to </ image>. In the <g id = "Article_image">, "g" represents the group, Article_iamge represents the name of the group (g), "opacity" and "transform" are the names of the animation attributes or animation effects, Represents a transformation. A symbol (sum) means to represent the animation properties together. The symbol dur indicates information on the reproduction interval of the image data, and the symbol infinite is not defined.

When the image data is regarded as one sample, the size of the image to be implemented by the image data (# sampleID = 1) having the identification number (ID) of '1' is 200 in width and 150 in height to be. Then, the transparency of the image data (# sampleID = 1) is changed from full transparency (value = 0.00) to linear (value = 1.00) linearly (calcMode = ) Repeatedly (repeatDur = "indefinite"). Further, the size change ("transform", "scale") of the image data (#sampleID = 1) is set to half the size of the image (value = y; x = 1.00; 0.50) in the horizontal direction from the original image size (RepeatDur = "indefinite") for 8 seconds (dur = "8.0") in a linear (calcMode- "linear") manner. Further, it is described that the two animation effects are performed together (additive = "sum").

The image data consisting of one or more samples is divided into chunks, and the samples in each chunk can be arranged in chronological order. At this time, each sample included in the chunk has its own unique identification number (ID), and the unique identification number (ID) of each sample can be assigned from '1'. In this case, the image data (# sampleID = 1) having a unique identification number (ID) of '1' represents its sample, not the data itself.

Referring to FIG. 2, at least one of the first, second, and third track areas includes a media area including timing information for sequentially reproducing media data included in the data area to construct a slide show, Lt; RTI ID = 0.0 &gt; meta area. &Lt; / RTI &gt; In addition, the timing information may be composed of another area other than the track area or a separate file from the multimedia data file having the structure shown in Fig.

Preferably, the timing information is information on an interval or a reproduction order in which the media data is reproduced on the time line, and all the media data included in the data area are dependent on one time line. That is, the timing information of all the media data included in the data area is preferably displayed on one time line.

Since each of the media data has different timing information, the playback period of each of the media data may not coincide.

In addition, the media area may include a first area having information on the playback period of the media data, a second area having the location information of the media data, and a third area having the size information of the media data. The media data to be reproduced can be retrieved using the position and size information included in the second and third areas.

The timing information of the media area may be expressed using a language such as a Light-weigh Application Scene Representation (LASeR), a Scalable Vector Graphics (SVG), or a Binary Format for Scene (BIFS). The LASeR, SVG, or BIFS may be implemented in XML (extensible Mark-up Language) format or Binary encoded format.

Timing information for all the media data included in the data area may be included in one media area, for example, the media area of the first track area. Alternatively, each of the media areas may have timing information of corresponding media data. That is, the media area of the first track area has timing information of the first media data, the media area of the second track area has timing information of the second media data, the media area of the third track area has the timing information of the third media data As shown in Fig.

The track meta area may include information for identifying media data. That is, the attribute information of the media data may be included in the meta area under the media information area, or may be included in the track meta area under the track area. If the information is included in the meta area, it is preferable that information on all media data included in the data area is included in one meta area. If the information is included in the track meta area, information about each media data Is preferably divided into track meta areas included in the corresponding track areas. The track meta area may also have information about the animation effect.

FIG. 3 shows a second embodiment of the structure of multimedia data according to the present invention. The data area may include an image data area, an audio data area, and a text data area. In addition, the media information area may include a slide show area having information on image data, an audio track area having information on audio data, and a text track area having information on text data.

The image data included in the image data area may be still image data or moving picture data and may be in the form of Joint Picture Expert Group (JPEG), Moving Picture Expert Group (MPEG) -1, 2 or Advanced Video Coding (AVC) And may be compressed data. That is, the image data may be data of various types of video clips or photographs obtained through a device (not shown) such as a camcorder (not shown) or a portable terminal (not shown).

The audio data included in the audio data area may be music data, accompaniment data, voice data, or the like, or compressed data in the form of MP3 or AAC (Advanced Audio Coding). Also, the audio data may be a result of combining the accompaniment data and the voice data, and the accompaniment data may be data that is expressed only by the musical instrument sound except the voice of the singer in music.

Also, the text data included in the text data area may be data having a character string divided in line units, in which case each line can be treated as a sample which is a reproduction unit.

FIG. 5 shows an embodiment of a method of constructing a slide show using multimedia data, in which the image data area has six pieces of image data, the audio data area has three pieces of audio data, Quot; has four text data.

As shown in FIG. 5, the media data are dependent on one timeline, but have separate timing information, so that playback intervals may not coincide with each other. Each of the media data included in the media data area preferably has playback section information, location information, and size information.

In addition, the media area of the slide show area may have playback section information, location information, and size information of six pieces of image data, three pieces of audio data, and four pieces of text data. In another embodiment, the media area of the slide show area has the playback section information, the position information, and the size information of six pieces of image data, and the media area of the audio track area includes the playback section information, Size information, and the media area of the text track area can have playback section information, position information, and size information of four pieces of text data.

FIG. 6 shows an embodiment of a method for reproducing a plurality of image data for a slideshow configuration.

Referring to FIG. 6, n images may be sequentially displayed for a slide show, and an interval in which the slide show is displayed includes an image display period (indicated by a solid arrow) and an image display period The above images can be displayed at the same time and can be divided into image switching sections (indicated by dotted arrows) in which switching of displayed images is performed.

In FIG. 6, tm1 denotes a time at which the mth image starts to be displayed, tm2 denotes a time at which the mth image starts to disappear, and tm3 denotes a time at which the mth image completely disappears. In addition, dm1 denotes a section length from the time when the mth image starts to be displayed until the time when the mth image starts to disappear, and dm2 denotes a range from the time when the mth image starts to disappear to the time when the mth image disappears, +1) th image is displayed at the same time.

Referring to FIG. 6, an image display period in which image 1 is displayed for d11 from t11 to t12, and image switching is performed in which image 1 and image 2 are simultaneously displayed for switching an image to be displayed for d12 from t12 to t13 Section. The image 1 that was displayed at the time t12 starts to disappear and the image 2 starts to be displayed and at the time t13, the image 1 disappears completely and only the image 2 can be displayed.

Then, image 2 is displayed during the image display period from t13 to t22, and image 2 and image 3 are simultaneously displayed during the image transition period from t22 to t23, and the display image is switched.

The animation effect described above may be applied to the two or more images to simultaneously display two or more images during the image transition period and to implement a natural and interesting image transition.

Information on image display intervals and image switching intervals as described above such as information on display time t11, t12, ...., tn1, tn2 or information on the length of each interval d11, d12, ....., d (n-1) 2, dn1) are preferably generated as additional information.

The following Tables 1 and 2 show examples of the structure of the additional information for the image display periods and the image switching periods.

As shown in Table 1 and Table 2, the additional information includes identification information for each of the image data, for example, information about a name of an image data file (File Name) or an address of a image data (Data Address) Time information for each display interval, for example, information about a display time (Timing) of image data or a length of a display interval (Duration).

For example, information on images to be displayed for each frame, for example, the number of image data displayed in the corresponding frame and the file name or data address can be obtained at each display time using the additional information as described above .

Therefore, it is preferable that the additional information includes at least one of information about a display timing (Timing) of image data and information about a duration of a display section (Duration) so that the additional information can be displayed when each of the plurality of images is displayed Do.

The decoding apparatus can recognize when each of the plurality of images included in the slide show is displayed using the additional information and efficiently reproduce the slide show by using the identified information for image reproduction using the animation effect .

The additional information may be included in any one of a plurality of areas included in the multimedia data described with reference to FIGS. 1 to 4. In such a case, the additional information may be included in the media information area. In addition to the timing information, . &Lt; / RTI &gt;

In addition, the additional information may be configured as a file other than the multimedia data file having the structure as shown in FIG. 1 to FIG.

FIG. 7 is a block diagram of an encoding apparatus according to an embodiment of the present invention. The encoding apparatus includes a file type region generation unit 100, a media information region generation unit 110, An additional information generating unit 130, and an output unit 140. The operation of the encoding apparatus shown in Fig. 6 will be described in connection with a flowchart showing an embodiment of the encoding method according to the present invention shown in Fig.

The file type area generation unit 100 generates a file type area indicating a format of the multimedia data file in step 300. The media type area generation unit 110 generates information on the media data included in the data area, A media information area including information on an animation effect to be applied to the data is generated (step 310). The additional information generating unit 130 generates additional information including information about an image display period in which image data is displayed and an image switching period in which two or more image data are simultaneously displayed in operation 320. The data area creating unit 120 creates a data area including a plurality of media data areas (step 330). The output unit 140 composes the generated file type area, the media information area, and the data area into one file and outputs the encoded multimedia data (operation 340). As described above, the additional information may constitute one file together with the file type area, the media information area, and the data area, or may be output in the form of a file different from the file.

7 is only an embodiment of the operation of the encoding apparatus according to the present invention, and thus the order of generation of the regions may be changed, or two or more regions in parallel It may be generated at the same time.

The detailed description of the file type area, the media information area, the data area, and the additional information is the same as described above and will not be described here.

FIG. 8 is a block diagram of an embodiment of a decoding apparatus according to the present invention. The decoding apparatus includes a data input unit 200, an animation effect information extraction unit 210, a media data extraction unit 220, An additional information extracting unit 230, a memory 240, a memory control unit 250, and a reproducing unit 260.

The data input unit 200 receives the multimedia data having the structure as described above (operation 400). Preferably, the multimedia data includes additional information about an image display section and an image changeover section of a plurality of image data for slideshow configuration.

The animation effect information extracting unit 210 extracts animation effect information by parsing the animation area in the multimedia data.

The media data extracting unit 220 extracts media data including a plurality of image data from the data area. The media data extracting unit 220 may extract the media data from the data area using the size information and the location information of the media data included in the media information area.

The additional information extracting unit 230 extracts additional information about the image display period and the image conversion period of the plurality of image data from the input multimedia data.

The memory controller 250 uploads the image data to be displayed to the memory 240 at predetermined time intervals, for example, on a frame basis, using the extracted additional information. The number of images to be displayed on a frame-by-frame basis may be one or two or more.

The playback unit 260 applies the animation effect to the image data uploaded to the memory 240 using the extracted animation effect information to reproduce the slide show.

As described above, the image data included in the image data area can be reproduced using the animation effect information, and the audio data and the text data can be reproduced using the timing information included in the media information area.

FIG. 10 is a flowchart illustrating an embodiment of a decoding method according to the present invention, and shows a method of reproducing a slide show composed of m frames.

Referring to FIG. 10, the memory controller 250 extracts image data to be displayed in the first frame of the slide show using the additional information, and uploads the image data to the memory 240 (steps 400, 410).

The playback unit 260 plays back the first frame by applying an animation effect to the image data uploaded to the memory 240 (operation 420).

After confirming that the playback of all the m frames has been completed (step 430), steps 410 and 420 are repeatedly performed for each frame until the playback of the entire slide show is completed.

As described above, only the image data to be displayed through the animation effect using the additional information is uploaded to the memory 240 and reproduced. Thus, efficient use of the memory is possible, and thus, even in a decoding apparatus having only a small- A slide show including images can be played back. In addition, it is possible to realize natural and interesting image conversion animation by using small size memory.

The encoding / decoding apparatus according to the present invention is provided in a multimedia broadcasting transmitting / receiving apparatus such as a digital audio broadcasting (DAB) and a digital multimedia broadcasting (DMB), and can be used for encoding / decoding an audio signal or a voice signal . Also, the multimedia broadcasting transmitting / receiving apparatus may include a mobile communication terminal. In addition, the multimedia data may be included in a broadcasting signal.

The encoding / decoding method according to the present invention may be implemented as a program for execution in a computer and stored in a computer-readable recording medium. The multimedia data having the data structure according to the present invention may be read by a computer. Lt; RTI ID = 0.0 &gt; media. &Lt; / RTI &gt; The computer-readable recording medium includes all kinds of storage devices in which data that can be read by a computer system is stored. Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may be implemented in the form of a carrier wave (for example, transmission via the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. Functional programs, codes, and code segments for implementing the user tracking method can be easily inferred by programmers of the art to which the present invention pertains. In addition, the bit stream generated by the above-described encoding method can be stored in the computer-readable recording medium or transmitted using a wired / wireless communication network.

The decoding apparatus according to the present invention may be provided in a reproducing apparatus such as a mobile communication terminal, a multimedia player such as a portable multimedia player (PMP), and a personal digital assistant (PDA). The above-described decoding apparatus may be embodied in hardware and incorporated in a reproducing apparatus, or may be embodied in a reproducing apparatus in software as described above.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

1 is a diagram schematically showing the overall structure of multimedia data according to the present invention.

2 is a diagram showing a first embodiment of the structure of multimedia data.

3 is a diagram showing a second embodiment of the structure of the multimedia data.

4 is a diagram showing a third embodiment of the structure of the multimedia data.

5 is a diagram showing an embodiment of a method of constructing a slide show using multimedia data.

6 is a diagram illustrating an embodiment of a method of reproducing a plurality of image data.

7 is a block diagram showing an embodiment of a configuration of an encoding apparatus according to the present invention.

8 is a block diagram showing an embodiment of a configuration of a decoding apparatus according to the present invention.

9 is a flowchart illustrating an encoding method according to an embodiment of the present invention.

10 is a flowchart illustrating an embodiment of a decoding method according to the present invention.

Claims (17)

Creating a data area including a plurality of media data areas; Generating a media information area including a plurality of track areas corresponding to each of the plurality of media data areas; Generating an animation area having information on an animation effect to be applied to a plurality of image data included in the data area; And Generating additional information having information on an image display section in which the plurality of image data is displayed and an image switching section in which two or more image data of the plurality of image data are simultaneously displayed, Wherein the plurality of media data included in the data area is dependent on one timeline, The data including the data area, the media information area and the animation area constitute one file, Wherein the additional information is generated as a separate file from the file. The method according to claim 1, Wherein the media information area is included in the media information area. delete A data area including a plurality of media data areas; A media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area having information on an animation effect to be applied to a plurality of image data included in the data area; Wherein the image display section displays the image data of the plurality of image data and the image switching section in which two or more image data of the plurality of image data are simultaneously displayed, Uploading at least one image data to be stored in the memory; And And applying the animation effect to at least one image data uploaded to the memory using the animation effect information included in the animation area, Wherein the additional information is input as a separate file from a file including the data area, the media information area, and the animation area. 5. The method of claim 4, Wherein the step of uploading the image data to the memory is performed on a frame basis. 5. The method of claim 4, Wherein the animation region is described as one of LASeR, SVG, and BIFS. The method according to claim 6, Wherein one of the LASeR, SVG, and BIFS is implemented as an XML foramt or a binary encoded format. 5. The method according to claim 4, And the media information area is included in the media information area. delete 5. The method according to claim 4, And identification information of the plurality of pieces of data and time information of display start time and end time of each of the plurality of image data. 5. The method of claim 4, Wherein the multimedia data is included in a digital broadcast signal. delete 5. The method of claim 4, Wherein the multimedia data is input using digital media. A data area generation unit for generating a data area including a plurality of media data areas; A media information area generation unit for generating a media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area generation unit for generating an animation area having information on an animation effect to be applied to a plurality of image data included in the data area; And And an additional information generating unit for generating additional information having information on an image display period in which the plurality of image data are displayed and an image switching interval in which two or more image data of the plurality of image data are simultaneously displayed, Wherein the plurality of media data included in the data area is dependent on one timeline, The data including the data area, the media information area and the animation area constitute one file, Wherein the additional information is generated as a separate file from the file. A data area including a plurality of media data areas; A media information area including a plurality of track areas corresponding to each of the plurality of media data areas; An animation area having information on an animation effect to be applied to a plurality of image data included in the data area; A memory for storing image data to be displayed; Wherein the image display section displays the image data of the plurality of image data and the image switching section in which two or more image data of the plurality of image data are simultaneously displayed, A memory controller for uploading at least one image data to be stored in the memory; And And a playback unit that plays back at least one image data uploaded to the memory using animation effect information included in the animation region, The additional information And a file included in the media information area or different from a file including the data area, the media information area, and the animation area. 16. The method of claim 15, Wherein the animation area is described by one of LASeR, SVG, and BIFS. delete

Images