KR101389730B1 - Method to create split position accordance with subjects for the video file - Google Patents

Abstract

The present invention relates to a method for creating split positions, which is capable of splitting sections according to subjects of video content by analyzing a sound source and a video of a video file. The method of the present invention comprises the steps of: (A) extracting a sound source from a file to be analyzed; (B) dividing the sound source into set time intervals and extracting words included in the sound source; (C) calculating an accumulated value of the extracted words and analyzing distribution thereof; and (D) creating split positions of the file to be analyzed according to a rate of change in the accumulated value of the accumulated words. According to the present invention, reproduction sections can be split by matching keywords extracted from a sound source and video of a video file, with respect to the video file including various subjects, thereby making clear a subject of each split video. [Reference numerals] (AA) Start; (BB) No; (CC) Yes; (DD) End; (S110) Execute a file to be analyzed; (S210) Extract a sound source; (S220) Extract words; (S230) Correct the words; (S240) Accumulate and store the extracted words; (S250) Analyze distribution of representative words; (S310) Extract a video; (S320) Recognize and extract a text; (S330) Degree of a change in an image >= Setting value; (S340) Store a video position and an important word; (S410) Split a sector

Description

{METHOD TO CREATE SPLIT POSITION ACCORDANCE WITH SUBJECTS FOR THE VIDEO FILE}

The present invention relates to a method for generating a split position for each subject of a video file, and more particularly, to a method for generating a split position for analyzing a sound source and an image of a video file and dividing a section for each subject of a video content.

Recently, due to the development of Internet technology, services using online are expanding through large-scale file transmission or real-time streaming of high-definition video. In addition, the type of video data is also enormous, and the desire to select and provide video data customized to a specific portion desired by the user is increasing from the user's point of view.

Typically, video lecture service has the advantage that learners can learn effectively regardless of time and place because they do not need to find and take lectures directly. However, video lectures are generally provided on the Internet. Produced in about 50 minutes or more, the concentration of the learner is lowered, there is a problem that the learner can not quickly / accurately search the content to take.

In addition, with the development of portable digital devices, as the number of learners who take video lectures for a short time while moving, the necessity of short video lectures divided by subjects is increasing.

Meanwhile, in order to satisfy the needs of the learners, Korean Patent No. 10-0997599 discloses a technology for dividing a video into a plurality of scenes and generating metadata for each of the divided scenes. Specifically, as shown in FIG. 1, a scene divider for generating section information metadata of a scene indicating a start position and an end position for each divided scene, based on the section information metadata of the scene from the scene divider. And a scene change detector for automatically detecting a scene change point through a pixel difference between frames of the video, a histogram difference of color or luminance, and the like.

However, when the caption included in the divided scene is automatically assigned through the conventional technology and a subject (text) is automatically assigned, the accuracy of the subject matched with the divided scene may be lowered. In this case, although the user can improve the accuracy by analyzing the divided scene directly and give a theme, there is a problem that the user loses time in checking all the divided scenes.

(0001) Republic of Korea Patent Registration 10-0997599

The present invention has been made to solve the above-mentioned problems, and the subject division position of the video file which can divide the playback section by extracting / matching the keywords and the degree of change from the sound source and the video constituting the video file, respectively. We want to provide a way to create it.

According to a feature of the present invention for achieving the above object, the method for generating a divided position for each subject of a video file according to the present invention comprises the steps of: (A) extracting a sound source from the analysis target file; (B) extracting words included in the sound source by dividing the sound source by a predetermined time interval; (C) calculating a cumulative value of the extracted words and analyzing the distribution thereof; And (D) generating a split position of the analysis target file according to the cumulative value change rate of the accumulated words.

In this case, step (B) may be performed by correcting an error recognition through shaping the extracted words according to pronunciation variations.

And the present invention (a) extracting the sound source from the analysis target file; (b) extracting words included in the sound source by dividing the sound source by a predetermined time interval; (c) calculating a cumulative value of the extracted words and analyzing a distribution thereof; (d) extracting an image from the analysis target file; (e) recognizing the image and extracting text; (f) comparing image variation and setting values of the image; (g) storing the image position and storing the text extracted from the section before the image position when the image change degree of the image is equal to or larger than a set value as a result of the comparison in the step (f); And (h) generating a split position of the analysis target file based on the distribution change of the extracted word of step (c) and the stored image position and text of step (g). And a method of generating a split position.

In this case, the distribution analysis of the extracted words in the step (c) may be to set the representative word according to the extraction frequency of the extracted words, and to analyze the distribution of the number of extraction for each play time of the representative word.

The image change degree may be a numerical value representing a change rate of the reference image in the image, and the set value may be a reference value for determining whether to change the content of the image according to the image change degree.

On the other hand, the text storage of step (g) may be to select and store the key word selected based on the number of repetitions within the section of the extracted text.

The selection of the division position of the (h) step may include: (h1) calculating a change position of the representative word according to a time-dependent change of the representative word extracted from the sound source; (h2) comparing the representative word with a key word extracted from the image; And (h3) when the representative word and the main word coincide with each other, setting the storage position of the image as a split position and setting a tag displaying a subject of a corresponding section as the representative word. have.

In this case, when the representative word and the main word do not match, the step (h2) may be performed by changing the representative word into any one of the words extracted by the step (c).

When the representative word and the main word do not coincide with each other, the main word may be changed to any one of words among the texts extracted by the step (g) to perform the step (h2).

As described above, the method for generating a divided position for each subject of a video file according to the present invention can expect the following effects.

According to the present invention, since a playback section is divided by matching keywords extracted from a sound source and an image of the video file including various subjects, there is an advantage of clarifying a subject for each divided image.

In addition, the present invention can clarify the subject for each divided image, there is an advantage that the user can be provided in a timely manner a video on the subject to watch.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing the configuration of an editing apparatus for detecting a scene change point of a moving picture according to the prior art.
2 is a flowchart illustrating a specific embodiment of a method for generating a divided position for each subject of a video file according to the present invention.
3 is an exemplary view showing a cumulative result of words extracted from a sound source of a video file according to the present invention.
Figure 4 is an exemplary diagram showing the distribution of the representative word in accordance with the playback section of the video file according to the present invention.
Figure 5 is an exemplary diagram showing the image position and the main word in a graph according to the image change degree for the image of the video file according to the present invention.
FIG. 6 is a diagram illustrating overlapping FIGS. 4 and 5 to divide a video file according to a subject according to the present invention. FIG.
7 is a conceptual diagram illustrating sector division of a file according to a specific embodiment of the method for generating a segmentation location for each subject of a video file according to the present invention.

The present invention relates to a method of generating a split position for dividing a video file into sectors according to a subject. The video includes audio video interleaved (AVI), moving picture experts group (MPEG), advanced streaming format (ASF), and WMV according to a format. A file configured in a format such as (Window Media Video) may be applied. In addition, the file may include lectures, news, advertisements, etc. according to the purpose of production, but in the exemplary embodiment of the present invention, a lecture file having a distinctive composition is described as an example for convenience of description.

In the present invention, the sound source and image extraction, the word extraction, the text extraction, the sector division generation, etc. of the file are performed by a dedicated program. The dedicated program is installed in hardware, and various control means and storage of the hardware are provided. It can operate by a means.

Hereinafter, with reference to the accompanying drawings, a specific embodiment of the method for generating a divided position for each subject of a video file as described above will be described.

2 is a flowchart illustrating a specific embodiment of a method for generating a segmented position for each subject of a video file according to the present invention, and FIG. 3 is an exemplary diagram showing a cumulative result of words extracted from a sound source of a video file according to the present invention. 4 is an exemplary diagram showing the distribution of the representative words according to the playback section of the video file according to the present invention, Figure 5 is a video position and the main word according to the image change degree for the image of the video file according to the present invention 6 is an exemplary diagram illustrated in a graph, and FIG. 6 is an exemplary diagram overlapping FIGS. 4 and 5 to divide a video file according to the present invention according to a subject, and FIG. According to a specific embodiment of the method of generating a partition position, a conceptual diagram of sector division of a file is provided.

First, as shown in FIG. 2, the method for generating a divided position for each subject of a video file according to the present invention begins with executing an analysis target file (S110). In this case, the file is a multimedia contents consisting of digitized sound source and video, and can be transmitted and received through a broadband communication network or a high-speed data network, and can be played on a digital device (computer, notebook, smartphone, PDA, etc.). Say that.

Next, the controller extracts a sound source from the analysis target file (S210). At this time, the sound source is extracted by a dedicated program and stored in the storage unit.

Next, a word is extracted from the sound source extracted in step 210 (S220). In this case, the controller extracts a word from the sound source at a set time interval and stores the word in the storage.

Here, the word extraction may be performed by a variety of methods known in the art, for example, by considering the similarity between the pattern registered in the database and the input pattern by considering the speech as a kind of pattern, The speech may be modeled, and a unique model may be assigned to each target word or phoneme to determine and extract from which speech model the input voice has the highest probability.

Next, the correction for the extracted word is performed (S230). The correction is to format the extracted words according to various pronunciation variations appearing in the dialogue voice. That is, the information on the pronunciation variations may be constructed as a database, and through this, correction of consonants and vowels may be used to formalize extracted words, and a language model database may be used to match the most likely words. You can also format the extracted words by doing.

 Next, the cumulative value of the corrected extracted words is stored (S240). In this case, the cumulative value represents the frequency of occurrence of the extracted word as a numerical value. In the embodiment of the present invention, the cumulative value is calculated at a time interval set for time series analysis of the extracted word.

Herein, referring to FIG. 3, the results of operation 240 are divided into 30s intervals of the playback period from the start point of the file to the end point of the file. If the cumulative values are stored, the cumulative values for the words extracted from the playback section '0 to 30s' of the file are' A-6 (Extracted word-cumulative value), B-2, C-1, D-0, E- 0, F-1,... 'Is stored, and the cumulative values for words extracted from the playback section' 30s ~ 60s' are 'A-3, B-1, C-0, D-0, E-2, F-1, ...'. 'Is stored. In this way, the cumulative value of the extracted words for each playback section can be stored.

Next, the distribution analysis for the representative word is performed (S250). In this case, the representative word implies the subject of the contents of the file, the cumulative value of the extracted words may be sorted in ascending order, and the representative word may be set through a relative ranking, and the controller may reflect the absolute value of the cumulative value. Extracted words whose cumulative value is greater than or equal to an absolute value may be set as representative words.

Here, the distribution analysis shows a time series change of the cumulative distribution of the representative word, through which a graph with a horizontal axis as a time and a vertical axis as a cumulative value can be generated.

For example, referring to FIG. 4, a graph generated through the distribution analysis is described. The set representative words are 'A', 'G', and 'X', and the cumulative values for each representative word in FIG. As shown in the figure, the distribution analysis result shows that the cumulative value of the representative word 'A' is high in the first half, the cumulative value of the representative word 'G' is high in the middle, and the cumulative value of the representative word 'X' is high in the second half. Appears. Through this, it is possible to grasp the subject of any particular section of the playback section of the file.

Next, the controller extracts an image from the analysis target file (S310). At this time, the image is extracted by a dedicated program and stored in the storage unit.

The text is recognized and extracted from the extracted image (S320). In this case, the controller stores the extracted text as a storage unit. Here, the recognition and extraction of the text may be performed by various known techniques, and for example, the edge pixel frequency in the vertical or horizontal direction and the fundamental frequency by Fourier transform through a characteristic spatial frequency distinguished from other parts. It may be extracted using a feature of, or may be extracted using a coefficient value calculated by a feature that contrasts with the background and stays at the same position of the image for a predetermined time or more.

Next, the image gradient and the set value of the image is compared (S330). That is, in the case where the lecturer lectures through the writing, the lecture image may be divided by changing the image occupancy rate of the blackboard according to the amount of writing. In this case, when the image of the person affects the image occupancy of the blackboard, the image occupancy of the blackboard may be calculated through a known technique in which a value obtained by recognizing a human image is reflected in the image occupancy.

In this case, the degree of change of the image represents the ratio of the image set by the controller to the image frame as a numerical value. In the case of the lecture image, the image set by the controller becomes a blackboard.

Here, the set value represents a numerical value for determining the position of the change point for changing the content of the image according to the degree of change of image, and may be a case where all the writings of the blackboard are deleted from the lecture image.

In this case, when the image change degree of the image is equal to or greater than a predetermined value as a result of the comparison in step 330, the image position and the key word are stored (S340). Here, the video position indicates a point where the image is changed in the playback section of the file, and serves as a criterion for section division by topic of the video file.

In addition, the key word indicates the main content of each section, and the text having the highest cumulative value among the stored texts may be set as the key word, or the text extracted at a specific location may be set as the key word. have.

For example, referring to FIG. 5, when a specific variable n is assigned to a point at which the degree of change of the image set by the controller is equal to or greater than the set value, the file is terminated from the starting point (n = 0) of the file. It is divided into four sections up to the point END, n = 4, and a value is assigned to the specific variable n to indicate the start and end positions of each section.

In this case, n = 4 means that the video image has been converted four times within the playing range of the video. For example, it may mean that the writing content is erased four times and the new writing content is filled.

The key word in the first section (n = 0 to n = 1) is 'A', the key word in the second section (n = 1 to n = 2) is 'G', and the third section ( The key word in n = 2 to n = 3 is 'G' and the key word in the fourth section (n = 3 to END) is 'X'.

On the other hand, when the comparison result of the step 330, the image change degree of the image is less than the set value, the step 320 is repeated.

Next, the control unit generates sector division for the file (S410). In this case, the sector division may be performed by overlapping the results of the steps 250 and 340.

In this case, the overlapping will be described with reference to FIG. 6, wherein the sectors are classified according to the subject of each section of the video file, and the key words of the section divided by the degree of change of the video and the extracted words for each play time of the sound source. The sector may be divided by matching the set representative word according to the cumulative value of.

That is, basically, the video files are classified according to the change of the representative word analyzed in operation 250, and the correct division position is set according to the image position calculated in operation 340. FIG. In addition, the checking of the split position may confirm whether the representative word and the main word of the section match.

Meanwhile, when the main word of each divided section is different from the representative word extracted from the sound source, the representative word or summary from the words or texts extracted through the steps 210 to 250 or the steps 310 to 340. The sector may be partitioned by resetting and matching words.

As illustrated in FIG. 7, a flag for identifying a position at which a subject lecture is changed is located in a playback section of the sector-divided file. In this case, the learner may select / replay the flag to selectively or repeatedly take the lecture to be taken, and the learner may be provided with the lecture file of a low capacity divided into several pieces according to the subject.

The rights of the present invention are not limited to the embodiments described above, but are defined by the claims, and various modifications and variations can be made by those skilled in the art within the scope of the claims. It is self-evident.

The present invention relates to a method of generating a split position for dividing a section for each subject of a video content by analyzing a sound source and an image of a video file. Since the playback section is segmented by matching the keywords extracted from the image, the subject can be clarified for each segmented image.

Claims (9)

(a) extracting a sound source from the analysis target file;
(b) extracting words included in the sound source by dividing the sound source by a predetermined time interval;
(c) calculating a cumulative value of the extracted words and analyzing a distribution thereof;
(d) extracting an image from the analysis target file;
(e) recognizing the image and extracting text;
(f) comparing image variation and setting values of the image;
(g) storing the image position and storing the text extracted from the section before the image position when the image change degree of the image is equal to or larger than a set value as a result of the comparison in the step (f); And
(h) generating a split position of the analysis target file based on the distribution change of the extracted word of step (c) and the stored image position and text of step (g). To create split locations by topic.
The method of claim 1,
Distribution analysis of the extracted word of step (c),
And a representative word is set according to the frequency of extraction of the extracted words, and the distribution of the number of times of extraction of the representative words for each play time is analyzed.
3. The method according to claim 1 or 2,
The image degree of change is a numerical value of the rate of change of the reference image in the image,
The setting value is a reference value for determining whether to change the contents of the image according to the image change degree.
The method of claim 3, wherein
The text storage of step (g),
And selecting and storing the key word selected based on the number of repetitions in the section among the extracted texts.
5. The method of claim 4,
Selection of the division position of the (h) step,
calculating a change position of the representative word according to a time-dependent change of the representative word extracted from the sound source;
(h2) comparing the representative word with a key word extracted from the image; And
(h3) if the representative word coincides with the key word, setting the storage position of the image as a split position and setting a tag displaying a subject of the corresponding section as the representative word. How to create a split location for each video file.
6. The method of claim 5,
(h3) If the representative word and the main word do not match, the moving image file is characterized by performing the step (h2) by changing the representative word to any one of the words extracted by the step (c). To create split locations by topic.
6. The method of claim 5,
(h3) when the representative word and the main word do not match, the step (h2) is performed by changing the main word to any one of words among the texts extracted by the step (g). How to create a split location for each video file. delete delete

Images