KR102281266B1 - System and Method for Extracting Keyword and Ranking in Video Subtitle - Google Patents

Abstract

The present invention relates to a system and method for extracting and ranking subtitle keywords in an image that enhances user search efficiency by analyzing keyword frequency in images and big data processing-based image analysis. Content input unit; Cloud editing server that supports extraction and ranking of subtitle keywords in the video; Keyword extraction and ranking calculation that extracts the frequency of words in the video to construct a ranking table with clip information for each section and provides it to users government; and, the keyword extraction and ranking calculating unit includes: an audio extraction module for extracting audio in the video, a caption extraction module for extracting captions in the video, and audio and subtitles extracted through the audio extraction module and the caption extraction module A keyword extraction module that extracts keywords using It includes an image analysis processing DB that stores the results of analyzing audio and subtitles in all images.

Description

System and Method for Extracting Keyword and Ranking in Video Subtitle}

The present invention relates to an image analysis and keyword analysis system, and more particularly, to a system and method for extracting and ranking caption keywords in an image that enhances user search efficiency by analyzing keyword frequency in images and analyzing big data processing-based images.

Recently, various video service services such as YouTube, Wave, and Netflix are being provided to users.

Many users are watching videos, broadcasts, and movies produced by CP (Content Provider) through the above service. These image providing services generally support a user's image search through the title of the image and various keywords that describe the image.

As described above, in the prior art, when a user searches for a desired video on a video streaming site, a search is performed based on information (title, description, hashtag) previously input by the user who uploaded the video.

In the process of performing an image search, a search result of a simple keyword search is provided through the introduction of an image without providing a separate frame index.

If the length of the video is long and the content to be found is a part of the entire content, the user watches the video by paying a separate search cost after such a search result.

The extra cost refers to the time element that occurs in the process of searching for a scene desired by the user through functions such as section search and playback speed control in order to find the necessary video information, and the user must perform additional work to find the desired video information. means to do

Therefore, there is a demand for the development of a new technology to increase the user search efficiency through keyword extraction and keyword frequency analysis in the image.

Republic of Korea Patent Publication No. 10-2019-0053587 Republic of Korea Patent Publication No. 10-2009-0075593 Republic of Korea Patent Publication No. 10-2011-00144403

The present invention is to solve the problems of the video search technology of the prior art, and to provide a system and method for extracting subtitle keywords and ranking in the video that enhances the user search efficiency by analyzing the frequency of keywords in the video and analyzing the big data processing-based video. There is a purpose.

It is an object of the present invention to provide a system and method for extracting subtitle keywords and ranking in an image so that a ranking system can be built based on information that occurs a lot in an image to provide accurate image information to a user.

It is an object of the present invention to provide a system and method for extracting subtitle keywords and ranking within an image so that an efficient image search can be performed by increasing the accuracy of a user's search result through keyword analysis and image analysis of an image.

The present invention extracts the frequency of words appearing in an image to configure a ranking table with clip information for each section, provides it to the user to support search, learns clip information through machine learning, and provides a learning image to the user to provide similar An object of the present invention is to provide a system and method for extracting subtitle keywords and ranking within a video to support video search.

Other objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, the system for extracting and ranking subtitle keywords in an image according to the present invention is a content input unit for extracting subtitle keywords in an image and inputting content for ranking calculation; Support for extracting and ranking subtitle keywords in an image a cloud editing server that extracts the frequency of words in the image, constructs a ranking table with clip information for each section, and provides a keyword extraction and ranking calculation unit for providing it to the user; including, wherein the keyword extraction and ranking calculation unit comprises: A voice extraction module for extracting audio from an image, a subtitle extraction module for extracting subtitles within an image, a keyword extraction module for extracting keywords using the audio and subtitles extracted through the audio extraction module and the subtitle extraction module; A keyword ranking module that supports searches by extracting the frequency of words that appear and composing a ranking table with clip information for each section and providing it to users, and storing the results of analyzing audio and subtitles in all videos produced and saved It is characterized in that it includes an image analysis processing DB.

Here, the keyword extraction and ranking unit is a configuration for image analysis, and a learning image extraction module for extracting a learning image based on the keyword analyzed in the subtitle keyword extraction and ranking calculation system in the image, and an image for learning with the extracted image It is characterized in that it comprises a learning module, an image analysis module for performing image analysis using the image learning result, and an analysis result providing module for providing the result analyzed by the image analysis module.

And the learning image extraction module is characterized in that, in order to support image analysis tailored to the user's search pattern, the image search target is an image in which there is no sound that is difficult to categorize by keyword, or an image with subtitles attached to a frame or only title and tag classification do it with

And the keyword extraction module performs morpheme analysis to extract keywords, and morpheme analysis includes a composition that analyzes sentences based on spaces, a composition that processes a search with words, and a composition that processes adjectives by grouping them together with the extracted words. characterized in that

And for keyword frequency analysis in the keyword ranking module, it is characterized by using Spark among big data platforms for iterative calculation of deep learning required for user clustering and real-time processing of uploaded images.

And the keyword ranking module includes a frequency analysis module for keyword frequency analysis, and the frequency analysis module extracts key keywords from the video subtitle file and stores the extracted keyword data in the storage unit ( Storage), a keyword analysis unit that performs keyword frequency analysis based on the keyword data stored in the storage unit, and ranks images in the order of the highest frequency for a specific keyword. characterized by including.

In addition, the keyword analysis unit (Keyword analysis) is built in a Spark-based cluster, and it is characterized by parallel processing of keyword frequency analysis work and ranking work on multiple servers.

And in order to increase the accuracy of the image analysis result stored in the image analysis processing DB, Tensorflow is used to analyze the information of the user who watched the video, and the information of gender, age, field of interest, department, and user log file It is characterized in that it includes a configuration that divides the user into several groups based on it.

And for clustering for video user analysis using TensorFlow, a user extractor that extracts information input and selected by a user when a user joins a video platform and log data recorded while the user watches an image, and a user extractor A user who reads the properties extracted from the DataFrame into a DataFrame and converts them into a matrix array format to quantify each value if the value of the properties is a string, and to quantify so that values that are related to each other are selected near and distant values are selected It is characterized in that it further comprises a digitization unit.

In accordance with the present invention for achieving another object, there is provided a method for extracting caption keywords in an image and calculating a ranking. A content input step for extracting caption keywords in a video and inputting content for ranking calculation; a voice extraction step of extracting a voice in the video and a video A caption extraction step of extracting subtitles; A keyword extraction step of extracting keywords using the audio and subtitles extracted through the audio extraction step and the caption extraction step; A ranking table with clip information for each section by extracting the frequency of words in the video and a keyword ranking step to support a search by configuring it and providing it to a user; storing the results of analyzing the audio and subtitles in all produced and stored videos; characterized in that it comprises a.

Here, as a frequency analysis process for constructing a keyword frequency analysis big data processing function in the keyword ranking step, a step of reading one line of the subtitle keyword extraction file for keyword frequency analysis, determining whether the keyword extraction file has been read, or Grouping by keyword, counting the number of keywords for each group, storing the analyzed result file in HDFS, analyzing the frequency of video keywords, and performing this process until all keyword extraction files are read characterized.

And in the keyword ranking step, as a keyword ranking process for constructing a keyword frequency analysis big data processing function, the step of reading the result file for keyword ranking, the step of judging whether all result files have been read, and all the result files If it is not read, the process of making the result file into a data frame, filtering with keywords as the ranking criterion, and creating one data frame using union operation until all result files are read; If the file has been read, it is characterized in that it includes the step of performing the image keyword ranking operation by performing the process of storing the sorted and ranked result files in a descending order in HDFS.

As described above, the system and method for extracting caption keywords in an image and calculating a ranking according to the present invention have the following effects.

First, it improves user search efficiency by analyzing the frequency of keywords in the video and analyzing big data processing-based images.

Second, a ranking system is built based on the information generated a lot in the video so that accurate video information can be provided to the user.

Third, through keyword analysis and image analysis of images, the accuracy of the user's search results is increased to enable efficient image search.

Fourth, by extracting the frequency of words in the video, a ranking table is constructed along with clip information for each section, and it is provided to users to support search, and clip information is learned through machine learning and learning images are provided to users for similar videos. to support you in your search.

1A and 1B are block diagrams of a system for extracting subtitle keywords and calculating rankings in an image according to the present invention;
2A and 2B are diagrams showing morpheme analysis of Korean and English according to the relationship
3A and 3B are block diagrams showing a keyword analysis result table, a keyword analysis result, and a frequency counting result;
4 is a system configuration diagram for analyzing the frequency of keywords in an image in a cluster environment.
5 is an input data file (CSV format) configuration diagram
6 is a system configuration diagram for clustering for video user analysis using TensorFlow.
7 is an overall configuration diagram of a subtitle keyword extraction and ranking process according to the present invention.
8A and 8B are flowcharts showing a frequency analysis process and a keyword ranking process for constructing a keyword frequency analysis big data processing function

Hereinafter, a preferred embodiment of the system and method for extracting caption keywords in an image and calculating a ranking according to the present invention will be described in detail as follows.

Features and advantages of the system and method for extracting caption keywords and ranking in a video according to the present invention will become apparent through the detailed description of each embodiment below.

1A and 1B are block diagrams of a system for extracting caption keywords in an image and calculating a ranking according to the present invention.

The present invention improves user search efficiency by analyzing keyword frequency in video and big data processing-based video analysis, and provides accurate video information to users by building a ranking system based on information generated a lot in video, and provides keyword analysis and video This is to enable efficient image search by increasing the accuracy of the user's search results through image analysis.

In particular, the present invention extracts the frequency of words appearing in an image to configure a ranking table with clip information for each section and provides it to the user to support the search, learns clip information through machine learning, and provides the learning image to the user This was done to support the search for similar images.

To this end, the present invention may include a configuration that enables a user to search for the contents of an image and shortens a search time by allowing a user of a video streaming site to check a desired clip image.

The present invention includes a function to remove unnecessary words when working on a dictionary for extracting key keywords (remove the proposition in Korean), and the frequently searched keywords are ranked according to the frequency of appearance, and the keyword content is determined according to the measured ranking. It may include a configuration in which the included points are displayed together with the thumbnail so that when the corresponding thumbnail is clicked, the image is played from that part.

Figure 1a is for keyword analysis in the subtitle keyword extraction and ranking system in the video according to the present invention, the content input unit 100, the cloud editing server 200 that supports for extracting and ranking the subtitle keyword in the video, keywords It includes an extraction and ranking unit 300 .

Here, the keyword extraction and ranking calculating unit 300 includes an audio extraction module 31 for extracting audio from an image, a caption extraction module 32 for extracting a caption within an image, an audio extraction module 31 and a caption extraction module A keyword extraction module 33 that extracts keywords using the audio and subtitles extracted through 32, and a ranking table with clip information for each section by extracting the frequency of words in the video and providing it to the user It includes a keyword ranking module 34 for supporting search, and an image analysis processing DB 35 for storing audio and subtitle analysis results in all images produced and stored.

This keyword analysis is a big data image analysis technique that analyzes audio information and subtitle files of an image to level the most frequently occurring keywords and utilize them for image analysis.

As the core information of the video as well as the video information is generally shown through the lecturer's voice or subtitle information, analysis keywords (voice, subtitles) are composed.

And Figure 1b shows a configuration of extracting a learning image based on the keyword analyzed in the subtitle keyword extraction and ranking system in the image according to the present invention, and performing image analysis with the extracted image.

As a configuration for image analysis, a learning image extraction module 36 for extracting a learning image based on the keyword analyzed in the subtitle keyword extraction and ranking system in the image, and a video learning module 37 for learning with the extracted image, , an image analysis module 38 that performs image analysis using the image learning result, and an analysis result providing module 39 that provides the result analyzed by the image analysis module 38 .

Image analysis can be applied at a stage where image data of a certain level or higher is established, and learning is performed by extracting a learning image based on keyword information of the image analysis processing DB constructed in the initial stage.

After that, video image learning is performed using deep learning and images are searched through similar image search techniques.

The image search target is an image for which keyword classification is difficult, that is, an image with no audio and subtitles attached to a frame or an image with only title and tag classification.

The reason is to optimize the resource cost of the system by supporting image analysis tailored to the user's search pattern, since a significant resource load occurs on the analysis system when all image patterns are extracted.

In order to provide efficient image analysis and search results, the system and method for extracting and ranking caption keywords in an image according to the present invention include a configuration of extracting words and morphemes for keyword analysis.

2A and 2B are diagrams illustrating morpheme analysis of Hangul and English according to the relationship.

A morpheme is defined as the smallest unit with meaning.

If you analyze the morphemes, you can easily understand the content you want to convey in the entire sentence and context.

In the case of English words, it is easy to extract each word, but in the case of Korean, it is difficult to accurately extract and analyze its meaning due to various factors such as adverbs, investigations, adjectives, and irony used in context.

The present invention proposes a keyword extraction algorithm for efficiently performing extraction and semantic analysis for keyword analysis by solving such a problem.

As the criteria for analyzing morphemes, there are series relation and integration relation. The series relation is vertical, which determines whether another morpheme can be replaced in its place, and the integration relation is horizontal, which considers whether the morpheme can be combined with other morphemes.

Among them, the series relationship plays a more important role in the morpheme analysis, and only the series relationship can give morpheme qualifications.

By analyzing the morphemes, it is possible to accurately determine the meaning to be conveyed on the current screen by understanding the subtitles of the image, and by providing this to the user, it becomes an element that can provide the currently necessary clip section rather than the entire unwanted image.

This can be seen as an area that can be utilized in the image analysis field as it can efficiently lower the resource cost and API cost for performing image processing through image analysis.

3A and 3B are block diagrams showing a keyword analysis result table, a keyword analysis result, and a frequency counting result.

The present invention is an algorithm used for morpheme analysis, and in addition to the configuration of analyzing sentences based on spaces, the investigation is processed with words, and in the case of adjectives, it includes a configuration of processing by grouping together with the extracted words.

The result of analyzing and extracting information of a specific sentence using this is the same as in FIG. 3A , and the result of morphological analysis and calculating the frequency are the same as in FIG. 3B .

In order to increase the efficiency of keyword frequency analysis, the present invention includes a configuration for analyzing keyword frequency in an image in a cluster environment as follows.

4 is a system configuration diagram for analyzing the frequency of keywords in an image in a cluster environment.

Apache Hadoop using MapReduce is used a lot to process large amounts of data. In order to search for images that increase in real time from the recently increased image providing sites, Apache Spark, which processes data in memory, is more useful than Apache Hadoop, which is a batch processing method. have an advantage

In the present invention, keyword frequency analysis of images is performed in a cluster environment built with Apache Spark.

When searching for a video on a video platform, it is usually searched because of the video's name and description. In addition, the video platform algorithm calculates the relevance of the word searched by the user to the image to increase the search result.

It is clear why video platforms want high-fidelity search results.

This is to provide high convenience to users. As the video platform has a lot of content, the user needs to select and provide the video they want, so that the user feels convenience, so high-accuracy search results are inevitably important.

In the system and method for extracting caption keywords in an image and calculating a ranking according to the present invention, a keyword-centered search is used to increase the accuracy of a search result.

Keyword-centered search refers to a search that finds an image with keywords that are the core of an image. In other words, if a keyword or a word with a meaning is regarded as a key word in an image, and the user enters a keyword, the images in which the keyword is frequently mentioned are displayed as a search result.

The more a specific keyword is mentioned in a video, the more likely that specific keyword is a key keyword in the video, and the more mentioned videos are shown as a search result.

In order to implement keyword-centered search, it is necessary to analyze the frequency of keywords for each image on the image platform.

Keyword frequency analysis refers to the operation of calculating how many times each keyword is mentioned in the video. By analyzing the frequency of keywords, you can find out which keywords appeared most in each video, and by collecting the analysis results, you can finally establish a ranking of which keywords appeared in which videos the most.

This ranking information becomes the result of keyword-centered search.

With the advantage of keyword-based search, the existing title-based image search method makes it difficult to find an image if the user forgets the image title. However, as long as the user remembers only the keywords of the video, the video can be found through a keyword-centered search.

In other words, the advantage of keyword-centered search is to increase search results by providing users with a different search option.

Keyword-centric search introduces a big data platform because it requires analysis of keyword frequency for many images, and Spark is used among big data platforms for iterative computation of deep learning required for user clustering and real-time processing of uploaded images.

4 shows the configuration of the frequency analysis module, which is a detailed configuration of the keyword ranking module 34.

The frequency analysis module includes a keyword extraction unit and a keyword analysis unit.

The keyword extraction unit is responsible for extracting key keywords from the video subtitle file, and stores the extracted keyword data in the storage of the keyword analysis unit.

This module corresponds to the first work to provide keyword-driven search.

The keyword analysis unit performs a keyword frequency analysis operation based on the keyword data stored in the storage unit and ranks the images in the order of the highest frequency for a specific keyword.

This module corresponds to the second task to provide keyword-driven search.

If you look at the right side of FIG. 4, there are tables A video and B video. Each table shows the frequency results of all keywords from each video.

The task of creating such a table is the keyword frequency analysis task.

The ranking operation collects the table resulting from the keyword frequency analysis operation to indicate which image has the highest frequency for a specific keyword.

Keyword of FIG. 4 : "Thread" The rank table is the result table of the ranking operation.

This table is the result of a keyword-driven search.

Since the keyword analysis unit is built on a Spark-based cluster, keyword frequency analysis and ranking tasks are processed in parallel on multiple servers.

By building a cluster, it is possible to perform keyword frequency analysis and ranking from continuously increasing keyword data at a cost that is realistically possible.

A cluster consists of one Client, Master node and several Worker nodes. Client node performs distributed processing operation instructions, Master node performs cluster resource management, and Worker node handles actual operations.

If you create a Spark Application that describes a distributed processing job and distribute it from the client node to the master node, the job is processed in parallel on three worker nodes, and worker nodes can be added or reduced at any time.

In addition, you can secure a lot of storage space by using HDFS as Storage.

5 is a configuration diagram of an Input Data file (CSV format).

It is a keyword file in CSV format that contains all keywords extracted from the video, and the video name and keywords extracted from the video are separated by a comma in each line.

The CSV file is used as input data for Spark Application. It reads the file stored in HDFS, calculates the frequency of keywords for each image, sorts in descending order, and saves it as a CSV file.

In addition, the system and method for extracting caption keywords and ranking in an image according to the present invention further include the following configuration in order to increase the accuracy of image analysis through video user analysis.

The present invention analyzes the information of the user who watched the video using Tensorflow, and divides the user into several groups based on the information of gender, age, field of interest, department, and user log file (watching time). do.

The user's information is analyzed and clustered so that the user can watch the desired video without a search function.

First, quantifying gender and areas of interest is dealt with first, and then, among various clustering methods, a method of generating a group by analyzing quantified information using the K-means clustering algorithm is explained.

In K-means, gender, age, and field of interest data are located in a three-dimensional space, and the number of center points is designated to determine the location of the center point n where the sum of distances between points is the minimum. It is a clustering algorithm that groups points close to this central point based on the central point, and shows relatively efficient results compared to other clustering.

6 is a system configuration diagram for clustering for video user analysis using TensorFlow.

First, the user extracting unit extracts information input and selected by a user when a user joins a video platform, and log data recorded while the user watches an image.

And among the extracted user information, as shown in Table 1, the fields of interest, age group, gender, and department (series) attributes suitable for numerical work are used.

Among the user log records, the average video viewing record is extracted and added as an attribute.

As the next operation, the properties are read into a DataFrame from the user digitization unit and converted into a matrix array format. If value among properties is a string, each value must be digitized.

At this time, the values that are related to each other are digitized so that they are selected near and distant values are selected.

Table 2 shows an example of quantifying user interests.

Table 3 shows another example of quantifying user interests.

If there are several subjects in the field of interest, related subjects such as math and physics, English and a second foreign language should be designated as close numbers, and subjects with less relevance should be numbered farther away.

Since there are two types of gender, 0 and 1 are used as the standard, and the departments are divided into the College of Engineering and the College of Arts and Sports, and then the departments that are closely related to each other are selected one by one and selected so that they end at the center of the two universities, that is, each university. do.

At this time, the age is set on a scale from 1 to 100, so the number of interest fields and departmental attributes is also assigned based on 100.

After that, the clustering unit uses this numerical information to represent users as a single point in space and performs K-Means clustering with N center points.

Hereinafter, the method of extracting caption keywords and calculating the ranking in the video according to the present invention will be described in detail.

7 is an overall configuration diagram showing the subtitle keyword extraction and ranking process according to the present invention, and FIGS. 8A and 8B are flowcharts showing the frequency analysis process and keyword ranking process for constructing a keyword frequency analysis big data processing function.

The method for extracting caption keywords in an image and calculating a ranking according to the present invention is largely divided into a content input step for extracting caption keywords in an image and content input for ranking calculation, an audio extraction step of extracting audio from the video, and extracting a caption in the video a caption extraction step, a keyword extraction step of extracting keywords using the audio and subtitles extracted through the audio extraction step and the caption extraction step, and a ranking table with clip information for each section by extracting the frequency of words in the video It comprises a keyword ranking step that supports a search by configuring it and providing it to a user, and a step of storing the analysis result of audio and subtitles in all videos produced and stored.

And Figure 8a shows a frequency analysis process for constructing a keyword frequency analysis big data processing function in the keyword ranking step.

First, one line of the subtitle keyword extraction file is read for keyword frequency analysis.

It is determined whether the extracted keyword files have been read, or grouped by keyword, counted the number of keywords for each group, and the analyzed result file is stored in HDFS to analyze the video keyword frequency, and this process reads all the keyword extraction files. do it until it comes

8B shows a keyword ranking process for constructing a keyword frequency analysis big data processing function in the keyword ranking step.

First, the result file is read for keyword ranking.

It is judged whether all result files have been read.

If all result files have not been read, the process of making the result file into a data frame, filtering with keywords as the ranking criterion, and creating one data frame using union operation is performed until all result files are read. .

If all result files have been read, image keyword ranking is performed by performing the process of sorting in descending order and storing the ranked result files in HDFS.

The system and method for extracting subtitle keywords and ranking in a video according to the present invention described above constructs a ranking table with clip information for each section by extracting the frequency of words appearing in the video, and provides it to the user to support the search, It learns clip information through learning and provides learning videos to users so that they can search for similar videos.

It will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention as described above.

Therefore, the specified embodiments are to be considered in an illustrative rather than a restrictive sense, the scope of the present invention is indicated in the claims rather than in the foregoing description, and all differences within the scope equivalent thereto are included in the present invention. will have to be interpreted.

100. Content Input
200. Cloud Editing Server
300. Keyword Extraction and Ranking Unit

Claims (12)

a content input unit for extracting caption keywords in the video and inputting content for ranking calculation;
a cloud editing server that supports extraction and ranking of subtitle keywords in the video;
Includes; a keyword extraction and ranking calculation unit that extracts the frequency of words in the video to construct a ranking table with clip information for each section and provides it to the user;
The keyword extraction and ranking calculating unit performs keyword extraction using a voice extraction module for extracting audio in the video, a subtitle extraction module for extracting subtitles in the video, and audio and subtitles extracted through the audio extraction module and the subtitle extraction module. a keyword extraction module that extracts the frequency of words in the video, configures a ranking table with clip information for each section, and provides it to users to support search; Includes an image analysis processing DB that stores the analysis result of subtitles,
The keyword extraction and ranking calculation unit is a configuration for image analysis, and a learning image extraction module for extracting a learning image based on the keyword analyzed in the subtitle keyword extraction and ranking calculation system in the image, and an image learning module for learning with the extracted image and an image analysis module for performing image analysis by using the image learning result, and an analysis result providing module for providing the result analyzed by the image analysis module. delete According to claim 1, wherein the training image extraction module,
In order to support image analysis tailored to the user's search pattern,
Subtitle keyword extraction and ranking system in video, characterized in that the image search target is an image that has no voice that is difficult to categorize as a keyword and whose subtitles are subordinated to a frame or an image with only title and tag classification. The method of claim 1, wherein the keyword extraction module performs morphological analysis for keyword extraction,
Morphological analysis is a system for extracting and ranking keywords in subtitles in an image, characterized in that it includes a configuration for analyzing sentences based on spaces, a configuration for processing a survey with words, and a configuration for grouping and processing adjectives together with extracted words . The method of claim 1, wherein for keyword frequency analysis in the keyword ranking module,
Subtitle keyword extraction and ranking system in video, characterized in that it uses Spark among big data platforms for iterative computation of deep learning required for user clustering and real-time processing of uploaded images. The method according to claim 5, wherein the keyword ranking module comprises a frequency analysis module for keyword frequency analysis,
The frequency analysis module includes: a keyword extraction unit for extracting key keywords from the video subtitle file and storing the extracted keyword data in a storage unit of a keyword analysis unit;
Subtitle keyword in video, characterized in that it includes a keyword analysis unit that performs a keyword frequency analysis operation based on the keyword data stored in the storage unit and a keyword analysis unit that ranks the images in the order of the highest frequency for a specific keyword. Extraction and ranking system. [Claim 7] The system of claim 6, wherein the keyword analysis unit is constructed as a Spark-based cluster, and performs the keyword frequency analysis and ranking tasks in parallel in multiple servers. The method of claim 1, wherein in order to increase the accuracy of the image analysis result stored in the image analysis processing DB,
Using Tensorflow to analyze the information of the user who watched the video, based on the gender, age, field of interest, department, and information in the user log file, the video, characterized in that it includes a configuration to divide the user into several groups Subtitle keyword extraction and ranking system. The method of claim 8, wherein for clustering for video user analysis using TensorFlow,
A user extraction unit for extracting information input and selected by a user when a user joins a video platform, and log data recorded while the user watches a video;
The properties extracted from the user extraction unit are read into a DataFrame and converted into a matrix array format. If the value of the properties is a string, each value is digitized, and the values that are related to each other are selected near and distant values are selected. Subtitle keyword extraction and ranking system in the video, characterized in that it further comprises a user digitization unit to digitize. a content input step for extracting caption keywords in the video and inputting content for ranking;
A voice extraction step of extracting a voice in the video and a subtitle extraction step of extracting a subtitle in the video;
a keyword extraction step of extracting keywords using the audio and subtitles extracted through the audio extraction step and the caption extraction step;
a keyword ranking step of extracting the frequency of words appearing in the image, constructing a ranking table with clip information for each section, and providing it to a user to support a search;
Storing the results of analyzing the audio and subtitles in all images produced and stored;
For video analysis in the keyword extraction stage and keyword ranking stage,
A learning image extraction process for extracting learning images based on keywords analyzed in the process of extracting keywords for subtitles and ranking in the video, a video learning process for learning from the extracted images, a video analysis process for performing image analysis using the image learning results , Subtitle keyword extraction and ranking method in the video, characterized in that it further comprises an analysis result providing process for providing the analyzed result in the image analysis process. 11. The method of claim 10, wherein in the keyword ranking step, a frequency analysis process for constructing a keyword frequency analysis big data processing function,
Reading one line of the subtitle keyword extraction file for keyword frequency analysis;
A step of determining whether the keyword extraction file has been read, or grouping by keyword,
Counting the number of keywords for each group, storing the analyzed result file in HDFS, analyzing the video keyword frequency, and performing this process until all keyword extraction files are read. and ranking methods. The method of claim 10, wherein in the keyword ranking step, a keyword ranking process for constructing a keyword frequency analysis big data processing function,
Reading the result file for keyword ranking;
A step of determining whether all result files have been read;
If all result files have not been read, the process of making the result file into a data frame, filtering it with keywords as the ranking criterion, and creating one data frame using union operation is performed until all result files are read. Wow,
When all the result files have been read, the process of storing the sorted result files in descending order and the ranked result files in HDFS is performed to perform the image keyword ranking operation. calculation method.

Images