KR20080084303A - Technology which is storing easily, quickly and accurately only wanted part from the movie and audio files - Google Patents

Abstract

A method for creating U-contents by easily, quickly and accurately extracting only a desired portion of a multimedia file is provided to allow a user to put in an annotation and index function by properly utilizing a portion extracting method and a contents editing method. If a subtitle is not provided by a service provider, a subtitle and a reproduction time point are extracted according to a speaker independent type continuous voice recognizing method(101). If a subtitle is provided by the service provider, a reproduction time point is extracted from the subtitle(102). A reproduced image and a corresponding subtitle image are displayed in synchronization for a user who wants to extract a desired portion(103,104). The user marks or selects a corresponding portion by underlining it or with a color or an image, and stores a major portion of video and a voice file through a storage command(105-107). Personal information of the user is ciphered for the purpose of a DRM and recorded in a binary form(108).

Description

Technology that is storing easily, quickly and accurately only wanted part from the movie and audio files}

1 is a block diagram illustrating a method of extracting a portion of a video or audio in which a speech point derivation factor (eg, a text or a caption) is not provided;

2 is a UI example of a program capable of partial extraction while simultaneously watching an image and a subtitle;

3 is a block diagram illustrating a method of extracting a portion of a moving picture or a voice provided with a firing point derivation factor (eg, a character or a caption);

4 is a configuration diagram showing an example of one of the editing method and use case according to the purpose of using the partial extraction data through the process of FIGS. 1 and 2;

FIG. 5 is a block diagram of screen editing while simultaneously viewing the partially extracted data output specification design guide and screen editing contents for each output medium; FIG.

The present invention provides a method for quickly and easily finding a specific part (a part desired by a user, a main scene, an essential part in an e-learning course) in a video and an audio file, and extracting the extracted part by displaying the color or underline. It is about.

In general, there are Microsoft's Window Media Player and Macromedia's Flash Player, which are commonly used to play video and audio. The multimedia file player provides tools such as a move bar that can change the location of the currently playing file. The purpose and use of the invention are fundamentally different from those of the present invention for viewing.

In addition, even if you can find the part you want, you can not easily see the front and back in a short time in e-learning courses or web services that have a one-sided characteristic, and you can not set the subject and area of the main part quickly and accurately. Since there is no function to extract the parts, it is fundamentally different from the present invention.

The object of the present invention devised to solve the above problems of the prior art is to easily display the desired part or main part (the main point) while watching or watching the video (voice) and the subtitle at the same time by subtitle (text) the voice in the video and the audio file. It provides users with a convenient way to quickly and accurately find and save and extract the designated part automatically by color or underlining (marking) between the start and end positions on the subtitle. It is to provide.

Partial extraction and storage method according to the present invention for achieving the above object comprises the steps of extracting the subtitles through the speaker-independent continuous speech recognition from the video or voice not provided subtitles;

Storing the body point and the longitudinal point of each sentence or vocabulary in the extracted subtitle in a temporary storage;

Arranging two windows side by side to display the playback screen of the video or audio file and the extracted subtitles at the same time;

When the user drags or clicks the playback position on the playback screen by using the subtitles and playback time stored in the temporary storage, moves the scroll bar on the subtitle display screen, or clicks a specific part of the subtitle display screen to specify the position. Synchronizing the play screen and the subtitle display screen together;

Marking a sentence that the user wants to extract by changing colors or underlining the text and clicking a save button;

Querying (queries) a sentence of the marked text in a temporary repository and reading a regeneration body point and a seed point;

And extracting and storing a desired portion from a video and an audio file by using the extracted extraction target time information.

In addition, the extraction and storage method according to another embodiment of the present invention comprises the steps of storing the subtitles provided in the temporary storage as the subtitles are provided in advance by the video and audio file provider;

Storing the sentence or vocabulary, the start and end time of playback of each word after speech recognition in a temporary storage through the lexical ordering feature of the main vocabulary of the stored subtitles;

Subsequently, the processing may include extracting and storing a desired portion from a video and an audio file by performing the same procedure after arranging the above-described playback screen and a screen displaying the extracted subtitles in parallel. do.

In addition, in order to more effectively implement the object of the present invention, in order to allow the user to find a desired position more quickly, a scene within the corresponding playback point on the caption display screen may be extracted as a still image and displayed next to the caption.

In addition, due to the characteristics of the present invention, a copyright problem may occur because a user can separately store a video on my computer. Therefore, the user's personal information stored at the time of generating the partial extraction file is encrypted and recorded in the extraction file in binary form. Characterized in that the step including the step of implementing the DRM function.

In the present invention, the embodied point is generally expressed as time information, but the present invention can be embodied as any information capable of recognizing a position in a video or voice, such as a frame number capable of expressing the position of a video.

Finally, the users of the present invention (product manufacturing and users, etc.) have the convenience of automatic editing and processing in order to increase the convenience of learning by storage media or learning according to a predetermined intention of data or information of a desired part for each user. Providing a means to enhance

Hereinafter, a partial extraction and storage method according to an embodiment of the present invention will be described in detail with reference to the accompanying table.

After removing the subtitles 112 by removing the non-voice sound such as background music or noise from the video or audio file 111 played on the Internet or the user's computer, and extracting the subtitle 112. The temporary storage 121 is stored. The generated subtitles have some errors in speech recognition, so even if the recognition rate is low, there is no problem for the user to underline or find the desired location, so there is no problem in finding and storing the desired parts.

After the speech recognition process by the sentence start time and end time of the sentence or every time point of the beginning and end of the vocabulary (113), etc. matched to the subtitle 112 file structured to extract the playback time to store in the temporary storage 121 And structuring step 102. The start point and the end point are defined as carcass and carcass points in the present invention, respectively.

In step 102, simultaneous processing may be performed when voice recognition is performed in step 101.

In step 103 of providing a UI as shown in FIG. 2 to quickly and easily find a desired part of the user, the playback screen 210 and the subtitle screen 220 are arranged side by side so as not to overlap, and a temporary storage ( The subtitle 112 stored in 121 is displayed together with the time information 113, and an important scene is extracted into the still image 223 at each time point and arranged as shown in the drawing.

Now, in the playback screen, subtitle screen synchronization step (104) to help the user quickly find the desired position can be implemented more effectively by using the patent registration technology (201-two window split), 210 and 220 windows do not overlap In this case, when the user drags or rolls over the play bar 211 (moves the mouse on the play rock) in order to change an image playback time on the play screen 210, the scroll bar 221 is displayed on the subtitle screen 220. It automatically moves to the subtitle position as it moves automatically. On the contrary, when the subtitle of the specific position is clicked on the subtitle screen 220 or the position of the currently viewed subtitle is changed using the scroll bar 221, the video play position is moved on the play screen 210 and the play bar 211 is displayed. The display position of is also changed automatically.

When the user finds a point in time to extract, the user may designate a portion to be extracted through the step 105 of marking the extraction object through a similar operation such as a mouse drag or a sentence click on the subtitle screen 220. In this case, the extracted region may be the whole sentence or the middle of the sentence, and the extraction range may also specify one region or a plurality of regions. After marking is completed, the user clicks the underlined part storage button 222.

Next, as a result of querying (finding) the subtitle 112 stored in the temporary storage 121 using the text information of the marked extraction time, the start time and the end time information 113 are read and the extraction time information is stored. Step 114 is performed.

In the extracting part 107 corresponding to the stored one or the plurality of extraction time information 114, the partial extraction image or the voice 115 is created by storing only the voice or storing the voice + video together and It can be stored or created on the server and used for various services.

The partial files of the video and audio produced by the present invention are randomly generated by a plurality of users, and may cause copyright problems when copying and distributing files other than the purpose of use by an individual alone. Therefore, as an optional step, the personal information of the file creator obtained by using the information of the logged-in user in the case of the use on the Internet site or the program registration in the case of the use in My Computer (personal information similar to the name, social security number, etc.) And forcibly encrypting 116 to record 108 in a binary form in the partially extracted video and audio file. This information can then be taken as a follow-up when the file is illegally distributed and can be used as a countermeasure to prevent illegal activities by warning users of such contents.

The example described so far has described an example in which captions are not provided in the video or voice 111 of the service provider. However, if subtitles are provided by the service provider, more accurate and complete video and audio extraction and storage can be obtained (Fig. 3).

Since the subtitles of the video or audio file 311 provided by the service provider and the subtitles 312 are stored in the temporary storage 321, step 301, and then the subtitles are reproduced in the sentence-based playback point extraction step 302. When speech recognition is performed using the arrangement order information of the vocabulary, the playback time point by sentence or vocabulary can be more precisely structured to obtain and store the playback time point 314.

After step 303 of displaying the playback screen 210 and the subtitle screen 220, extracting and storing the partially extracted video or audio 316 (307) and DRM encrypted recording step 308 1 is the same as step (103) to step (108) of Figure 1, and a description thereof will be omitted.

As an example in which the present invention is used, the partial design data 316, which has undergone all the above-described processes, is subjected to specification design 401 for each purpose of use, and screen editing 402 is performed according to the output form. The contents are stored in the screen output contents storage medium 405 for each purpose. At this time, if the index function 403 according to the output device designation is given to the screen output contents at the same time, and the storage and calling function 404 is provided to the electronic devices for each use of the users, a plurality of users can utilize the various uses 406. In an example in which the present invention is used, in 402 (output form and screen editing) of FIG. 4, using Patent Registration No. 076314 (501) provides even more convenience. With the output screen configuration and the processing edit window 502, the right screen can design and develop the output while simultaneously viewing the edit specification and the reference 503 according to the output type, which can greatly reduce production time and effort.

As described above, according to the present invention, the user can preview the subtitles and partial images in the video and the voice so that the user can easily and quickly and accurately find the desired part by simply underlining the necessary part in the subtitles and automatically extracting and saving them by clicking the save button. There is.

In addition, by using the partial extraction method and the content editing method according to the present invention, there is also an effect that can be added to the annotation and indexing function.

Therefore, a service provider who wants to utilize the present invention can provide a user with a function of u-learning that can be utilized in an information device that can be extracted and carried only in the case of e-learning and carried in e-learning, and various video and sound service services. Companies, such as news, movies, music, dramas, such as all images and voices can be seen quickly only the desired portion, it is effective to provide the convenience to extract.

Therefore, especially in the field of e & u learning, not only the high efficiency of learning but also the general public will greatly contribute to the activation of the use of various services including multimedia such as video and voice.

Claims (8)

Storing the caption and playback time of the subtitles and the vocabulary of the caption with caption information through voice recognition in a video or audio file and storing the caption information in a temporary storage; Marking (selecting) a portion of the caption to be extracted; Video or audio extraction method comprising the step of automatically extracting and storing the playback time information with the selected text information and the partial extraction and storage from the video or audio file The method of claim 1, wherein when the playback time of the corresponding part is structured through voice recognition, the separation time and the playback time are extracted in consideration of the characteristics of the change of the screen, the voice or the sound, as well as the characteristics of the vocabulary, words, and sentences. Featured video or voice extraction method The method of claim 1, wherein when a subtitle corresponding to the video or audio file is provided in advance, a playback time is extracted by recognizing a voice or sound pattern using the provided subtitle. The method of claim 1, wherein the playback screen and the subtitle display screen are simultaneously displayed when the video or audio file is played. When the playback position of the playback screen is changed, the display position of the subtitle screen is automatically moved to the corresponding position. The video or audio extraction method characterized in that the playback position of the playback screen is automatically moved to the corresponding position and synchronized The video or audio extraction method of claim 1, wherein the main scene for each view point is extracted as a still picture and displayed on a playback screen so that a user can find a desired position easily and quickly and accurately. The method of claim 1, wherein the user's personal information is encrypted for the purpose of copyright protection when the partial extraction and storage of the video or audio file are simultaneously recorded in the video and audio file generated in binary form. The method of extracting video or audio, comprising the step of presenting the method of editing the output form and specification according to the use media of the user and the output medium by using the extracted data (video or audio) according to claim 1. The processing edit according to claim 7, wherein only the desired contents are extracted from the video or audio file, and the output form and specification are stored according to a predetermined intention, and the convenience of automatic editing is different for each medium (eg MP3, PMP, PDA, etc.). Video or audio extraction method characterized by providing a means

Images