KR20100018987A - Method for contraction information of multimedia digital contents - Google Patents

Abstract

PURPOSE: A contraction information of multimedia digital content is provided to quickly and accurately select desired content from search list. CONSTITUTION: A video clip/low quality audio extracting unit(165) extracts and stores various information including video and audio or only video. The video clip/low quality audio extracting unit extracts color based on extraction information. A video bit rate classified encoding unit(190) generates video contraction information having various qualities by encoding extracted information again using codex information.

Description

Contraction information of multimedia digital contents and how to use them {Method for contraction information of multimedia digital contents}

The present invention relates to multimedia digital content reduction information and a method of using the same. More specifically, the present invention relates to a method of effectively condensing the contents of digital contents of a multimedia environment such as photos, music, and videos in a limited computing power and network bandwidth environment, and efficiently using the condensed information.

In general, an image viewer or a file browser of a personal computer has a function of displaying a small size image in order to preview an image file, which is called thumbnail image browsing.

Therefore, when the user executes the thumbnail image browsing, the original image is generated by decoding the compressed image (* .JPG, * .TIF, etc.), and the thumbnail image is generated again from the original image and displayed on the screen.

Such thumbnail image browsing is usefully used on an electronic album that stores and displays an image taken by a camera, and a user can easily find a desired image through the displayed thumbnail image.

In the case of IPTV and Cable TV, which have a large number of channels recently, how to provide the user with information about each channel is one of the big problems. Using the program schedule is one way to tell, but it doesn't tell you the current progress or content.

Among them, several channels are displayed in the form of thumbnail images on one screen, but this method only obtains instantaneous information. Simultaneous playback of all channels on one screen cannot be achieved due to the performance limitations of the playback device.

In other words, there is a need for a method that can show the contents of various channels over time without exceeding the limits of the playback apparatus.

In particular, in the case of Internet search, if a video is searched by a specific title, thumbnail images and descriptions appear on the screen as search results. The images and descriptions that appear in the search results are mostly written by the user who uploaded the contents. In such cases, the images and descriptions may be different from the actual video.

The images and explanations that are different from the actual images are caused by mistakes or intentional errors of the user who uploaded the images.

This error causes a serious problem in the reliability of the search results. Showing all the images will solve the problem, but this is not possible due to network bandwidth and computing power limitations.

In addition, in the case of CCTV where a large number of cameras are installed, multiple channels can be viewed simultaneously by increasing the number of physical displays in the installed location, but in remote areas, multiple channels cannot be viewed on one screen at a time due to network speed limitations. In general, four or up to eight channels of live video are not supported.

This network speed problem also affects sound. Most CCTV cameras do not support audio. This is because the MJPEG format, which can compress and send only the screen in a short time due to the problem of network bandwidth and CPU power, which is insufficient for screen transmission, is used.

Currently, high-performance codecs such as H.264 are used, but most of them transmit only video information except audio due to limitations in network bandwidth and CPU power.

Also, in the case of music search, similar to video search, images such as performers or singers, album covers, etc. are used as representative images, and lyrics or feelings, descriptions of the performers or singers are intended to express the feeling of the music.

This type of explanation is because audio is difficult to express visually like a video. If you have a lot of search results, you have to read the description or listen to a clip of a particular piece of music to find out if it is the music you are looking for. Therefore, in the case of music, not only the auditory judgment but also a method of visually determining like a video is needed.

In addition, digital photography has become popular due to the rapid spread of digital cameras, and the number and capacity of digital photographic contents possessed by individuals can not be ignored due to the popularization of equipment and high-capacity memory capable of taking high-resolution photos.

Searching for high-resolution photographs, hundreds of megabytes of scanned images, and these high-resolution photographs and images that take up tens of megabytes is a single thumbnail image, and the searcher must look at the thumbnail image to see if it's what they want.

However, small thumbnail images can sometimes make it difficult to grasp the contents of a photo. For example, if you represent a group photo as a small thumbnail image, you don't know who appeared in the photo.If you look at the original to see it, you need to manipulate the mouse (click / click) to communicate the user. And screen movement.

Accordingly, there is a need to present various reduction methods for digital contents in a multimedia environment, and these methods may be established as new solutions to problems in the field of searching and selecting digital contents and contents expression.

Accordingly, the present invention is to solve the above problems, the present invention indicates what is the best reduction information in the state of not exceeding the limit of network bandwidth, the limit of computing power, the limit of limited performance of various devices, Its purpose is to present how information can be used alone or in combination.

According to the present invention as a technical idea for achieving the above object, (a) inputting a digital content consisting of a video; (b) extracting and storing f (n) thumbnail images of the entire video at various sizes and resolutions at intervals of f (s) seconds by analyzing the type and content of the input digital content type; (c) extracting and storing information of various qualities including the video and the audio of the extracted thumbnail image or the audio is removed, and extracting the variation amount color and the representative color based on the extraction information; (d) re-encoding the extracted information at a lower bit rate using the codec information specified in the basic value and environment information to generate video reduction information of various qualities; And (e) displaying a thumbnail image according to the network of the user terminal and the performance of the terminal when the video reduction information is searched on the web and the channel is selected.

In addition, according to the present invention (a) inputting digital content consisting of audio; (b) generating audio contraction information by analyzing the type and content of the input digital content type; And (c) visually displaying the change amount of the audio with a low bandwidth by using the audio change amount video and the change color of the generated audio contraction information.

In addition, according to the present invention (a) a step of inputting a multimedia digital content consisting of a photo; (b) generating N thumbnail images by analyzing types and types of the input multimedia digital content types; (c) extracting original and reduced image reduction information of the thumbnail image and dividing the original and reduced image reduction information into horizontal f (x) and vertical f (y) blocks to store the reduced information; And (d) selecting the thumbnail image and the divided image having a suitable resolution according to the network speed, the screen resolution, and the characteristics of the display device of the reduced information, and sequentially or designating the thumbnail image or the divided image according to the intended display method. And displaying at intervals of f (s) seconds in the intended order.

As described above, according to the present invention, 1) the user can quickly and accurately select the desired content from the search result list by showing content contents through various forms of abbreviation without exceeding the performance limit in the case of Internet search. ) In case of CCTV, various types of methods such as video clip and audio changing color table, audio sampling method, and audio separation extraction method are presented.3) For music search, visualization video and visualization video clip, visualization through color, representative color, quality It presents various methods such as adjustment. 4) In case of digital photo, it presents the method of making thumbnail image of various resolution / size, dividing original and reduced image, etc. Effective as a new solution to the problem Can.

Hereinafter, with reference to the accompanying drawings, the configuration and operation of the embodiment of the present invention will be described in detail.

1 is a block diagram of a system for extracting abbreviated information of multimedia digital content according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a process of extracting abbreviated information of a video and its result according to an embodiment of the present invention. 3 is a diagram illustrating a process of extracting abbreviated information of audio and its result according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating a process of extracting abbreviated information of a video and its result according to an embodiment of the present invention.

As shown in FIG. 1, the multimedia content reduction information extraction system according to the present invention includes a multimedia digital content 100 such as a video, audio, photo, and the like; The digital content 100 may be input to refer to the image change amount palette 105, the volume / tempo change amount palette 110, the default value, and the environment information 115 to refer to the content type / content analyzer 150, photo resizing, and the like. Split image generator 155, thumbnail image extractor 160, video clip / low quality audio extractor 165, video / audio clip generator 170, audio visualization video generator 175, audio variation detection unit Digital content reduction information extraction means 140 for extracting digital content reduction information from the image change amount detecting unit 185 and the video bit rate encoding unit 190; Storage means 195 for storing the reduced information, such as thumbnail image, video clip / low quality image, video clip, audio visualization video, audio representative color, change color, image representative color extracted from the digital content reduction information extraction means 140 );

In this case, the digital content reduction information extraction means 140 includes a content input unit 145 for receiving the multimedia digital content 100; A content type / content analyzer 150 analyzing the type and form of the content when the multimedia digital content 100 is received; A photo resizing and split image generating unit 155 for generating n thumbnail images of horizontal, vertical, and size from an original photograph when the input content is a horizontal, vertical photo image; A thumbnail image extracting unit 160 for extracting a plurality of thumbnail images of the entire image from the original video at various sizes and resolutions at intervals of f (n) seconds, and storing the extracted images in the storage unit 195; A video clip / low quality audio extractor 165 extracting information of various quality video clips, low quality audio, and low quality audio clips from which audio is removed and storing the contents in the storage means 195; A video / audio clip generator 170 for generating a plurality of movie clips of various qualities and low quality movie clips containing images and audio and storing the same in a storage means 195; An audio visualization video generation unit 175 for generating an audio variation video or a part of the audio and storing the audio change in the storage means 195; An audio change amount detecting unit 180 generating a color and a representative color of the change amount by referring to the volume / tempo change amount palette 110 and storing the color and representative color in the storage means 195; An image change amount detecting unit 185 for extracting a change amount color and a representative color and storing the change amount color and the representative color in the storage means 195; And a video bit rate encoding unit 190 for generating video reduction information of various qualities by re-encoding the extracted information using a code value specified in a basic value and environment information at a lower bit rate than the original video.

Here, a description is given of a given function value used in the system for extracting abbreviated information of digital contents according to the present invention.

The resulting value of the function f () of the present invention depends on the given parameter and the content characteristic.

First, in the case of a video, thumbnail images of various resolutions / sizes are generated at intervals of f (n) seconds as abbreviated information. Video clips without audio, such as Flash clips, audio clips created with f (q) quality for f (d) seconds (k> d) at f (k) second intervals, Movie clips containing f (aq) quality audio at f (u) seconds f (aq) quality and f (vq) quality and resolution for f (l) seconds (u> l). 105) and volume / tempo change color palette (110 in FIG. 1) based on the color and representative color of the video and audio changes, and several videos with audio and video quality of f (aq) and f (vq). To use them.

In this case, the abbreviated information may be used alone or may be used in combination. And the quality of the information used for mixing is determined by network bandwidth and computing power.

In the case of audio, it can also be abbreviated information as audio of several levels, audio clips for f (u) seconds at f (l) second intervals, abbreviations for the first, middle, and last parts, visualization videos and clips of audio variation, Color representations and representative colors for audio variation according to volume / tempo variation palette are used.

In this case, the abbreviated information may be used alone or may be used in combination. And the quality of the information used for mixing is determined by network bandwidth and computing power.

In addition, the reduced information of the photo is used as the reduced size of various sizes and f (n) divided images of the original and the reduced version. The thumbnail and the split image may be used independently or may be mixed.

Next, based on the multimedia content reduction information extraction system according to the present invention, the video reduction information extraction process, the audio reduction information extraction process, and the photo reduction information extraction process will be described in more detail.

As shown in FIG. 2, the video abbreviation extracting process of the present invention comprises the steps of: inputting digital content consisting of a video; Analyzing the type and content of the input digital content type to extract and store f (n) thumbnail images of the entire video at various sizes and resolutions at intervals of f (s) seconds; Extracting and storing information of various qualities including the video and audio of the extracted thumbnail image or the audio is removed, and extracting the variation amount color and the representative color based on the extraction information; Generating the video reduction information of various qualities by re-encoding the extracted information using the codec information specified in the basic value and environment information at a lower bit rate than the original video; And displaying a thumbnail image according to the network of the user terminal and the performance of the terminal when the web reduction information is searched on the web and the channel is selected.

In more detail, if the content entered through the content input unit 145 is analyzed by the content type / content analyzer 150 as a video, the digital content reduction information extracting means of FIG. 1 through the thumbnail image extractor 160. Extracting thumbnail images of the entire image at various sizes and resolutions at intervals of f (n) seconds, such as thumbnail image extraction (210, 213, 216) of 140, and the extracted images are stored in the storage means 195. Are stored in.

The video clip / low quality audio extractor 165 extracts various quality video clip 220, low quality audio 230, and low quality audio clip 240 information from which audio is removed and stores the contents thereof. 195).

In addition, the video / audio clip generation unit 170 generates various quality movie clips 250 and low quality movie quality 255 including video and audio.

At this time, the movie clip / low-quality movie clip (250) (255) is a f (s) seconds interval f (d) seconds (S> D) to play the video and audio of lower quality than the original movie (200) f (n) is generated.

Next, the variation amount color 260 and the representative color 270 are extracted through the image variation amount detection unit 185.

At this time, referring to the image change amount palette 105 and the volume / tempo change amount palette 110 of FIG. 1, color information corresponding to the change amount is extracted to generate a color.

The representative color 270 has two types of audio representative colors and image representative colors representing the entire video, and through this color, a person can recognize a sudden change in the activity and volume of the image through the colors.

Finally, the video is re-encoded using the codec information specified in the basic value and the environment information 115 at a lower bit rate than the original video 200, so that various quality Low Quality 1 (280) and Low Quality n 290 are generated.

At this time, the video abbreviation information shows the thumbnail image extraction 210 or the video clip 220, or the video clip 220 and the low quality audio extraction 230 according to the user's network and equipment performance when searching the web and selecting a channel. It will be mixed and played.

By doing so, many images can be displayed on the screen at the user terminal, and at the same time, the user feels like playing and the overall contents can be known. In addition, the movie clip 250 can be viewed by fade in / out (Fade In / Out). Here, the user terminal represents a computer device, such as a personal computer, a notebook.

In addition, as the representative color 270, the feeling of the image can be determined intuitively / visually, and the change amount color 260 can be used to represent the change amount of the video and audio in a low network band over time.

When a user selects a video, the original video 200 or a low quality video, that is, Low Quality 1 (280) or Low Quality n (290), can be played according to the network condition and the performance of the computer or the playback device.

As shown in FIG. 3, the audio abbreviation extraction process of the present invention comprises the steps of: inputting digital content consisting of audio; Generating audio contraction information by analyzing the type and content of the input digital content type; And visually displaying the change amount of the audio with a low bandwidth by using the audio change amount video and the change color of the generated audio reduction information.

In more detail, if the content input through the content input unit 145 is analyzed by the content type / content analysis unit 150 as audio, the low quality audio extraction unit of the video clip / low quality audio extraction unit 165 is low quality. The audio data of the 310 and 320 is generated.

In addition, an audio clip 330 and a highlight 340 are generated through the video / audio clip generator 170.

Next, the audio visualization video generating unit 175 generates the audio variation video 350 or a part of the audio change.

Finally, the color change 360 and the representative color 370 of the change amount are generated through the audio change amount detecting unit 180 with reference to the volume / tempo change amount palette 110.

Here, the audio variation video 350 and the variation color 360 of the generated audio contraction information allow the user to visually see the variation of the audio with a low network bandwidth, and display the characteristics of the audio at a time through the representative color 370. I can figure it out.

In addition, low-quality audio, audio clips 330, highlights 340, and the like, can deliver sufficient content even with low network bandwidth and computing power.

As shown in FIG. 4, the process of extracting photo abbreviation information of the present invention comprises the steps of: inputting multimedia digital content consisting of photos; Generating N thumbnail images by analyzing the type and shape of the input multimedia digital content type; Extracting original and reduced image reduction information of the thumbnail image and separating the original and reduced image reduction information into horizontal f (x) and vertical f (y) blocks and storing the reduced information; And selecting the thumbnail image and the segmented image having the appropriate resolution according to the network speed, the screen resolution, and the characteristics of the display device of the user terminal, and sequentially or designating the thumbnail image or the segmented image according to the intended display method. And displaying at intervals of f (s) seconds.

In more detail, when the multimedia digital content 100 of a picture is input through the content input unit 145 of the digital content abbreviation extracting unit 140, the content type / content analysis unit 150 and the type of content may be used. Analyze the form.

As a result of the analysis, when the input content is a picture having a width (W) and a height (H), the picture resizing and split image generation unit 155 has the width (W1, W…, Wn), and the length (H1, H…, Hn). ) Thumbnail 1, thumbnail… of n thumbnail images (see reference numeral 400 in Fig. 4) of size (Wn <W… <W1 <W, Hn <H… <H1 <H). , Thumbnail n).

Then, the original and the reduced image are divided into horizontal f (x) and vertical f (y) blocks as shown in FIG. 4 such as the original image segmentation 410 and the reduced image segmentation 420 of the photo reduction information extraction shown in FIG. Is stored in the storage means 195.

Here, the values of f (x) and f (y) may be changed by the size of the picture, the characteristics of the device to be displayed, and the definition of the user.

The generated picture reduction information selects a thumbnail image and a segmented image having an appropriate resolution according to a user's network speed, screen resolution, and display device characteristics when displaying Internet search results and a list of pictures, and according to an intended display method. The thumbnail image or the divided image 430 is displayed at intervals of f (i) seconds in a sequential or designated / intentional order.

Here, according to the display intention, a method of using a thumbnail image and a split image together to show a thumbnail image first and then a split image may be used.

By using this method, the user can view an image of higher resolution than the thumbnail image without any control. Even when expressing the image selected by the user's selection, the optimal image can be viewed quickly by selecting and transmitting the most suitable image for the limited circumstances of the user environment, device, and network.

1 is a block diagram of a system for extracting abbreviated information of multimedia digital contents according to the present invention.

2 is a diagram illustrating a process of extracting abbreviated information of a video and its result according to an embodiment of the present invention.

3 is a diagram illustrating a process of extracting abbreviated information of audio and its result according to an embodiment of the present invention.

4 is a diagram illustrating a process of extracting abbreviated information of a video and its result according to an embodiment of the present invention.

Description of the Related Art

100: multimedia digital content

105: image variation amount palette

110: volume / tempo change color palette

115: basic value and environmental information

140: digital content reduction information extraction means

145: content injection interface

150: content type / content analysis unit

155: Photo Resizing and Split Image Generator

160: thumbnail image extraction unit

165: video clip / low quality audio conversion unit

170: video / audio clip generator

175: audio visualization and video generation unit

180: audio variation detection unit

185: image change detection unit

190: Encoding unit for each video bit rate

195: storage means

Claims (11)

(a) inputting digital content consisting of a video; (b) extracting and storing f (n) thumbnail images of the entire video at various sizes and resolutions at intervals of f (s) seconds by analyzing the type and content of the input digital content type; (c) extracting and storing information of various qualities including the video and the audio of the extracted thumbnail image or the audio is removed, and extracting the variation amount color and the representative color based on the extraction information; (d) re-encoding the extracted information using a codec information specified in a default value and environment information at a lower bit rate than the original video to generate video reduction information of various qualities; And and (e) displaying thumbnail images according to the network of the user terminal and the performance of the terminal when the video reduction information is searched on the web and the channel is selected. The multimedia digital according to claim 1, wherein when the audio of the thumbnail image extracted in step (c) is removed to include only a video, information of one selected from video clip, low quality audio, and low quality audio clip is stored. Content abbreviation information and how to use it. The method of claim 1, wherein a movie clip is generated when the video and audio of the thumbnail image extracted in the step (c) are included, and the movie clip is f (d) seconds (S> D at intervals of f (s) seconds). Multimedia digital content contraction information, and a method of using the same, characterized in that f (n) is generated with lower quality than original video. The method according to claim 1, wherein the change color in the step (c) is extracted by referring to an image change amount palette and a volume / tempo change amount palette. The method according to claim 1, wherein in the step (e), the video clip or the video clip and low quality audio are mixed and played according to the network of the user terminal and the performance of the terminal. (a) inputting digital content consisting of audio; (b) generating audio contraction information by analyzing the type and content of the input digital content type; And and (c) visually displaying the amount of change of the audio with a low bandwidth by using the audio amount of change video and the amount of change color among the generated audio reduction information. 7. The method of claim 6, wherein the type and content of the digital content type input in step (b) are analyzed to generate audio abbreviation information of an audio clip and a highlight. The method according to claim 6, wherein the type and content of the digital content type input in the step (b) are analyzed to generate audio abbreviation information of the change color and the representative color with reference to the volume / tempo change amount table. Multimedia digital content abbreviation information and how to use it. (a) inputting multimedia digital content consisting of photographs; (b) generating N thumbnail images by analyzing types and types of the input multimedia digital content types; (c) extracting original and reduced image reduction information of the thumbnail image and dividing the original and reduced image reduction information into horizontal f (x) and vertical f (y) blocks to store the reduced information; And (d) selecting the thumbnail image and the segmented image having a suitable resolution according to the network speed, the screen resolution, and the characteristics of the display device of the user terminal, and sequentially or designating the thumbnail image or the segmented image according to the intended display method. And displaying at intervals of f (s) seconds in a predetermined order. 10. The method of claim 9, wherein in the step (c), the horizontal f (x) and the vertical f (y) are changed according to the size of the picture, the characteristics of the device to be displayed, and the definition of the user. Abbreviated information and how to use it. 10. The method of claim 9, wherein a method of displaying a thumbnail image first and then displaying a segmented image is selectively used according to the display method intended in the step (d).

Images