KR102357313B1 - Content indexing method of electronic apparatus for setting index word based on audio data included in video content - Google Patents

Abstract

Disclosed is a content indexing method of an electronic device. The content indexing method comprises: a step of acquiring at least one piece of text from audio data included in video content; a step of setting an index word for the video content based on at least one keyword included in the acquired text; and a step of providing the video content when user input requiring a search for the set index word is received.

Description

Content indexing method of electronic device that sets index word based on audio data included in video content { CONTENT INDEXING METHOD OF ELECTRONIC APPARATUS FOR SETTING INDEX WORD BASED ON AUDIO DATA INCLUDED IN VIDEO CONTENT }

The present disclosure relates to an electronic device for setting an index word for video content, and more particularly, to an electronic device for extracting an index word by converting audio data included in video content into text.

Various services are being used to provide video content desired by a user by using an application or a web page.

In addition, a number of search services that allow users to directly search for desired video content are also provided.

In general, a search for video content is a keyword search-based service according to the title, introduction, date, genre, producer, performer, and the like of the video content.

However, in the case of Internet lectures that provide knowledge or videos that provide information, there are many cases where important information or keywords are not included in the title or introductory text, so a simple title or genre-based search has limitations.

Therefore, there is a need for an indexing service that enables users to search for specific content of video content.

Registered Patent Publication No. 10-1587926 (Comment tagging system for streaming video and method for providing the same)

The present disclosure provides an electronic device or system that enables an effective search for video content by setting an index word based on audio data included in video content.

Objects of the present disclosure are not limited to the above-mentioned purposes, and other objects and advantages of the present disclosure that are not mentioned may be understood by the following description, and will be more clearly understood by the embodiments of the present disclosure. Moreover, it will be readily apparent that the objects and advantages of the present disclosure may be realized by the means and combinations thereof indicated in the claims.

A content indexing method of an electronic device according to an embodiment of the present disclosure includes: acquiring at least one text from audio data included in video content; based on at least one keyword included in the acquired text, the video content The method may include setting an index word for content, and providing the video content when a user input requesting a search for the set index word is received.

In the obtaining of the text, the text may be obtained by converting a voice included in the audio data.

In addition, the step of setting the index word may include dividing the obtained text into a plurality of subtexts based on a sentence, paragraph, or paragraph, and dividing the video content into a plurality of time sections matching each of the plurality of subtexts. classify, select an index word for at least one sub text among the plurality of sub texts, and set the selected index word to match in a time section matching the sub text among the plurality of time sections included in the video content may be

In this case, the providing of the video content may include, when a user input for receiving the video content searched according to the set index word is received, matching the set index word among the plurality of time sections included in the video content. A time interval can be provided.

In addition, the content indexing method of the electronic device includes, when a user input requesting evaluation information of the video content searched according to the set index word is received, another user's comment on the time section matching the index word The method may further include providing a reading service for the bulletin board.

In this case, in the content indexing method of the electronic device, when the time section of the video content searched according to the set index word is provided to the user, at least one method for recording the user's comment for the time section in the bulletin board The method may further include providing a user interface of

The content indexing method of the electronic device may include: receiving a user input requesting an additional explanation while the video content is being provided; The method further includes: identifying a time segment being played at the received time period; searching for at least one other video content based on an index word matching the identified time segment; and providing the retrieved other video content. You may.

Meanwhile, the content indexing method of the electronic device includes the steps of: identifying a user's learning purpose and learning level; selecting at least one index word matching the user's learning purpose and learning level; The method may further include recommending one video content.

The content indexing method of the electronic device according to the present disclosure has an advantage in that it is possible to index by setting an appropriate index word for each video content based on an audio-based deep analysis of the video content.

The content indexing method of the electronic device according to the present disclosure can provide a learning management solution capable of managing learning content and learning time centered on a learner, since index words can be set and provided for each time section.

1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
2 is a flowchart illustrating a content indexing method of an electronic device according to an embodiment of the present disclosure;
3 is a view for explaining an operation of an electronic device for extracting an index word from audio data according to an embodiment of the present disclosure;
4 is a view for explaining an operation of an electronic device for setting an index word for each time section included in video content according to an embodiment of the present disclosure;
5 is a diagram for explaining an operation of providing, by an electronic device, a bulletin board for each of time sections in which an index word is set, according to an embodiment of the present disclosure;
6 is an algorithm for explaining an operation of an electronic device providing a user interface (UI) related to reading and writing of a bulletin board before and after providing video content, according to an embodiment of the present disclosure;
7 is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Prior to describing the present disclosure in detail, a description will be given of the description of the present specification and drawings.

First, terms used in the present specification and claims have been selected in consideration of functions in various embodiments of the present disclosure. However, these terms may vary depending on the intention or legal or technical interpretation of a person skilled in the art, and the emergence of new technology. Also, some terms are arbitrarily selected by the applicant. These terms may be interpreted in the meanings defined herein, and in the absence of specific definitions, they may be interpreted based on the general content of the present specification and common technical common sense in the art.

Also, the same reference numerals or reference numerals in each drawing attached to this specification indicate parts or components that perform substantially the same functions. For convenience of description and understanding, the same reference numbers or reference numerals are used in different embodiments. That is, even though all components having the same reference number are illustrated in a plurality of drawings, the plurality of drawings do not mean one embodiment.

In addition, in this specification and claims, terms including an ordinal number such as “first” and “second” may be used to distinguish between elements. This ordinal number is used to distinguish the same or similar elements from each other, and the meaning of the term should not be construed as limited due to the use of the ordinal number. As an example, the use order or arrangement order of components combined with such an ordinal number should not be limited by the number. If necessary, each ordinal number may be used interchangeably.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment of the present disclosure, terms such as “module”, “unit”, “part”, etc. are terms for designating a component that performs at least one function or operation, and such component is hardware or software. It may be implemented or implemented as a combination of hardware and software. In addition, a plurality of "modules", "units", "parts", etc. are integrated into at least one module or chip, except when each needs to be implemented in individual specific hardware, and thus at least one processor. can be implemented as

In addition, in an embodiment of the present disclosure, when it is said that a certain part is connected to another part, this includes not only a direct connection but also an indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 1 , the electronic device 100 may include a memory 110 and a processor 120 .

The electronic device 100 may correspond to a server device capable of communicating with various user terminals. In this case, the electronic device 100 may interwork with various user terminals through at least one web page or application, and may provide a learning management method according to various embodiments to be described later through the user terminal.

Alternatively, the electronic device 100 may be a smartphone, a notebook PC, a desktop PC, a tablet PC, a personal digital assistant (PDA), a wearable device, a TV, a set-top box, a console device, a virtual reality (VR) / augmented reality (AR) device. It may correspond to various terminal devices such as

The memory 110 is a configuration for storing an operating system (OS) for controlling overall operations of components in the electronic device 100 , at least one instruction, and data.

The memory 110 may include non-volatile memory such as ROM and flash memory, and may include volatile memory such as DRAM. Also, the memory 110 may include a hard disk, a solid state drive (SSD), or the like.

The memory 110 may temporarily or permanently store at least one video content.

The video content may be composed of a plurality of image frames and audio data. The video content may correspond to various categories of video content, such as education, sports, games, news, dramas, movies, and information provision.

As a specific example, the video content may correspond to a Massive Open Online Course (MOOC), an Open Course Ware (OCW), an Internet lecture video, or the like.

The processor 120 is a configuration for overall controlling each configuration included in the electronic device 100 , and various units such as a central processing unit (CPU), an application processor (AP), a graphic processing unit (GPU), a VPU, and an NPU. can be composed of

The processor 120 may control the electronic device 100 by executing an instruction stored in the memory 110 . Hereinafter, an operation of the processor 120 according to various embodiments of the present disclosure will be described with reference to the drawings.

2 is a flowchart illustrating a content indexing method of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 2 , the electronic device 100 may obtain at least one text from audio data included in video content ( S210 ).

Specifically, the electronic device 100 may acquire text by converting speech/voice included in audio data based on a Speech-to-Text (STT) technology. Here, the electronic device 100 may sequentially derive at least one phoneme, word, or sentence using the acoustic model and the language model, but is not limited thereto.

Also, the electronic device 100 may perform translation by analyzing the language (nationality) of voices constituting the audio data. In this case, the electronic device 100 may use at least one self-translation module/tool or an external translation service (eg, an external server).

For example, when the nationality of the user and the nationality matching the language of the text are different, the electronic device 100 may translate the language of the text into the language of the nationality of the user.

The electronic device 100 may distinguish the voices of different speakers by analyzing characteristics of the audio data, and when the voices of a plurality of different speakers are included in the audio data, separate texts may be obtained for each speaker. Here, the electronic device 100 is a linear predictive coefficient (Linear Predictive Coefficient), Cepstrum (Cepstrum), Mel Frequency Cepstral Coefficient (Mel Frequency Cepstral Coefficient, MFCC), each frequency band energy (Filter Bank Energy) analysis, CNN Various techniques for analyzing the characteristics of each user's voice included in audio data, such as (Convolutional Neural Network)-based classification and RNN (Recurrent Neural Network)-based analysis, can be used.

Meanwhile, in addition to audio data, the electronic device 100 may extract text from at least one image frame constituting video content. Specifically, the electronic device 100 may acquire the text included in the image frame by using OCR (Optical Character Recognition) technology.

Referring to FIG. 2 , the electronic device 100 may set at least one index word for video content based on a keyword included in the acquired text ( S220 ).

Specifically, the electronic device 100 may extract at least one keyword from the obtained text, and may generate an index word using the extracted keyword or at least one word related to the extracted keyword. Here, the index word may be composed of one word or may be composed of a phrase or sentence including a plurality of words.

The electronic device 100 may use various types of natural language processing technology. As an example, the electronic device 100 may extract at least one subject word from the text or derive a subject word encompassing the text by using a language model such as Generation Pre-trained Transformer 3 (GPT-3).

Also, the electronic device 100 may extract a keyword from within the summary after summarizing the text through at least one summary algorithm using a template or an artificial intelligence model.

Meanwhile, the electronic device 100 may set an index word for each of a plurality of time sections included in video content.

To this end, the electronic device 100 may classify the obtained text into a plurality of sub texts and set an index word for each sub text.

Here, the electronic device 100 may obtain a plurality of sub-texts by dividing the text by sentence, paragraph, or paragraph.

As a specific example, the electronic device 100 may extract one or more keywords that may become a subject by sequentially analyzing texts converted from audio data. In addition, the electronic device 100 may define a plurality of sub-texts by dividing the text into paragraph units based on each keyword.

In addition, the electronic device 100 may divide the video content into a plurality of time sections matching each of the plurality of sub texts. Specifically, for each sub-text, the electronic device 100 may identify a time period in which audio data corresponding to the sub-text and matching image frames are reproduced.

Here, the electronic device 100 may select an index word with respect to at least one subtext among the plurality of subtexts. Specifically, at least one keyword included in the subtext may be selected as an index word. In this case, it is also possible to select a plurality of index words for one sub-text.

In this case, the electronic device 100 may set the index word selected for the sub-text to match a time interval matching the sub-text. As a result, one or more index words may be set for each of the plurality of time sections.

When the electronic device 100 is implemented as a server, the above-described processes S210 to S220 may be performed entirely on the electronic device 100 or may be performed on a system including the electronic device 100 and the user terminal. .

For example, when video content is reproduced in the user terminal, the user terminal may obtain text by converting audio data in real time. In this case, the electronic device 100 as a server may receive a text from the user terminal, analyze the received text, and set at least one index word matching the video content.

When the electronic device 100 is implemented as a user terminal, it is also possible that all of the above-described processes S210 to S220 are performed through the electronic device 100 that is a user terminal.

For example, in a situation where the electronic device 100, which is a user terminal, receives real-time streaming of video content, the electronic device 100 may convert audio data received in real time into text, and use at least one Keywords (index words) can be extracted. In this case, the electronic device 100 may transmit information on the index word set for the video content to at least one server, and as a result, index information generated for each video content will be shared to various user terminals through the server. can

After an index word is set for one or more video contents according to the various embodiments described above, when a user input requesting a search for the set index word is requested, the electronic device 100 provides at least one video content matching the index word It can be done (S230).

When the electronic device 100 is a server, a user input (requesting a search) may be received through various user terminals capable of communicating with the electronic device 100 . When the electronic device 100 is a user terminal, a user input may be received through a user input unit (eg, a touch panel, a button, a microphone, a keypad, etc.) provided in the user terminal.

The electronic device 100 may provide various information on video content found according to the index word.

Specifically, the electronic device 100 may provide information on the title of the searched video content, (production/upload) date, play time, introductory text, number of views, paid/free status, sales amount, and the like.

Then, when a user input for receiving the searched video content is received, the electronic device 100 may provide a video content playback service.

When the electronic device 100 is a server, the electronic device 100 may transmit video content stored in the server to the user terminal or perform real-time streaming. Alternatively, the electronic device 100 may provide a streaming service by interworking with an external server in which video content is stored and a user terminal.

When the electronic device 100 is a user terminal, video content stored in the electronic device 100 may be played. Alternatively, the electronic device 100 may download video content from at least one external server or request streaming.

Meanwhile, when an index word is set for each time section of video content, the electronic device 100 may provide (play) only a time section matching the set index word among a plurality of time sections included in the video content (searched according to the index word). . In this case, the electronic device 100 may also provide information on a reproduction point (eg, several minutes and several seconds) within the video content of the time period being reproduced.

In this case, there is an advantage that only a time period suitable for the index word search can be provided to the user, and an unnecessary time period that does not fit the purpose of the index word search (eg, learning/research purpose) can be automatically skipped.

3 is a diagram for explaining an operation of an electronic device for extracting an index word from audio data according to an embodiment of the present disclosure.

Referring to FIG. 3 , the electronic device 100 may acquire text 330 by converting audio data 320 included in video content 310 corresponding to an Internet lecture. Here, the audio data 320 includes the speaker's voice.

Then, the electronic device 100 may extract the subject word of the text 330 as a keyword (“Australopithecus” 340 of FIG. 3 ) using a keyword extraction module or the like.

The keyword extraction module may analyze the meaning of each word/sentence included in the text by performing a natural language processing function, and may use an artificial intelligence model trained to select a subject word when at least one sentence/paragraph is input. .

In this case, the electronic device 100 may set the keyword “australopithecus” 340 as an index word matching the video content 310 .

FIG. 4 is a diagram for explaining an operation of setting, by an electronic device, an index word for each time section included in video content, according to an embodiment of the present disclosure.

The electronic device 100 may classify the above-described text 330 into a plurality of sub-texts according to at least one subject word and/or a paragraph.

In addition, the electronic device 100 may divide the video content 310 into a plurality of time sections 410 , 420 , and 430 matching each of a plurality of subtexts.

In this case, the electronic device 100 may set the subject word included in each sub text as an index word for each time section. As a result, referring to FIG. 4 , “Australopithecus” 411 is set as an index word for time section A 410, and “Homo erectus” 421 is set as an index word for time section B 420, and , “Homo neanderthalensis” 431 may be set as an index word for the time period C 430 .

Meanwhile, after the index word is set for each time section as in the above-described embodiment of FIG. 4 , the electronic device 100 may recommend at least one other video content in relation to the corresponding video content 310 based on the index word.

Specifically, while the corresponding video content 310 is being played (via at least one user terminal), the electronic device 100 may receive a user input requesting additional explanation.

In this case, the electronic device 100 may identify a time section being reproduced at a time when a user input requesting additional explanation is received among a plurality of time sections (eg, 410 , 420 , 430 ) included in the video content. .

Here, the electronic device 100 may additionally search for at least one other video content based on an index word matching the identified time period, and may additionally provide information on the other searched video content or play the corresponding video content. have.

Of course, if additionally searched video content also sets a separate index word for each of a plurality of time sections, the electronic device 100 may provide only a time section matching the above-described index word among the additionally searched video content.

As such, the electronic device 100 according to the above-described embodiment of the present disclosure searches for and recommends/provides suitable additional content according to the point in time when a user's input requesting additional explanation is received by using the index word set for each time section. There are advantages to being able to

Meanwhile, according to an embodiment, the electronic device 100 may provide an independent bulletin board for each index word.

As an example, when an index word is set for each of a plurality of time sections, the electronic device 100 serving as a server provides various UIs for providing video content and/or a separate bulletin board for each time section (of video content) in which one or more index words are set. (User Interface) can be provided.

In this case, the user terminals connected to the electronic device 100 as the server may read the bulletin board and write an article by using each index word, and as a result, opinions of various users may be shared.

In relation to this, FIG. 5 is a diagram for explaining an operation of providing, by an electronic device, a bulletin board for each of time sections in which an index word is set, according to an embodiment of the present disclosure. 5 , it is assumed that independent index words 411 , 421 , 431 are set for each time section 410 , 420 , 430 as shown in FIG. 4 .

Referring to FIG. 5 , the electronic device 100 may provide independent bulletin boards 510 , 520 , and 530 for each index word.

As a specific example, when a user input requesting evaluation information of the video content 310 searched according to the index word “australopithecus” 411 is received, the electronic device 100 returns to the time interval A 410 matching the index word. It is possible to provide a viewing service for the bulletin board including the comments of at least one other user.

In this case, the content of human evaluation of various viewpoints (eg, lecture style, instructor's attitude, lecture difficulty, faithfulness of content, etc.) for the explanation of "Australopithecus" provided through the video content 310, which is the content of Internet lectures. can be provided, and since such evaluation contents are difficult to be obtained only by mechanical analysis of audio data, there is an advantage in that the precision of curation for video content is improved through this.

And, when the time section A 410 of the video content 310 found according to the index word “australopithecus” 411 is provided to the user (playback), the electronic device 100 displays the user's comment in the above-mentioned bulletin board. At least one user interface (UI) for recording may be provided.

Specifically, the electronic device 100 records (ex. writes) the user's comment in the bulletin board on the premise that the corresponding time section A 410 is reproduced for a certain period of time or over a certain ratio (via the user's user terminal). A variety of UIs can be provided.

In this case, the written text may be divided into various categories such as evaluation, review, question, and request.

For example, when a question text is written, the electronic device 100 , which is a server, may provide a service that allows a lecturer or producer of the video content 310 to directly answer through a user terminal.

In addition, since texts containing various evaluations/reviews/requests can be shared with the lecturer or producer, the user's feedback on the content of the video content 310 can have a good influence on the lecturer or producer's production of additional content. there is also

6 is an algorithm for explaining an operation of an electronic device providing a user interface (UI) related to reading and writing of a bulletin board before and after providing video content, according to an embodiment of the present disclosure.

Referring to FIG. 6 , as the index word is searched ( S610 ), the electronic device 100 may provide information on video content matching the index word ( S620 ).

For example, when the electronic device 100 is a server, the electronic device 100 may provide various information such as a title, date, producer, and sales amount of the corresponding video content. In this case, the electronic device 100 may provide a payment service for the corresponding video content, and may interwork with at least one finance-related server.

Also, the electronic device 100 may provide information on an address/link of a web page through which the searched video content can be streamed.

Here, the electronic device 100 may provide information about a time section matching an index word among video contents (eg, a reproduction point of the time section).

Alternatively, the electronic device 100 may generate and provide a separate web page/address for streaming only a time section matching an index word among video contents.

Here, a user input requesting evaluation information for the corresponding video content may be received. In this case (S630 - Y), the electronic device 100 may provide a viewing service for a bulletin board in which comments of other users related to the corresponding video content are recorded (S640).

Specifically, the electronic device 100 may provide a viewing service of the bulletin board for a time period matching the searched index word among the corresponding video contents.

Meanwhile, when a user input for receiving video content is received (S650), the electronic device 100 may provide a video content playback service (S660).

For example, when the electronic device 100 is implemented as a server, the electronic device 100 may provide a streaming service or a download service for video content through a user terminal of a user.

Here, the electronic device 100 may reproduce the entire video content through the user terminal and visually inform the location of the playback point of the time section matching the index word, or play only the time section matching the index word among the video contents. have.

When the reproduction of the provided video content is completed, the electronic device 100 may provide at least one UI for recording the user's comment on the above-described bulletin board (via the user terminal), and the electronic device 100 may Depending on the input, comments corresponding to evaluation/review/question/request, etc. can be recorded on the bulletin board.

In this way, only viewing of other users' comments is allowed before the video content is actually played, and as the user's comments are received only when the video content is actually played, the reliability and accuracy of comments in the bulletin board can be guaranteed. It works.

Meanwhile, according to an embodiment related to video content for educational purposes, the electronic device 100 may recommend at least one index word according to the user's learning purpose and/or learning level.

First, the electronic device 100 may identify the user's learning purpose and/or learning level.

The purpose of learning can be divided according to the subject of education (ex. mathematics, Korean, English, etc.), the field of education (ex. engineering, web development, raising companion animals, etc.), specific educational content, etc. Targets may also be included.

The level of learning may be classified according to the user's grade (eg, lower elementary, upper elementary, middle school, second grade, high school, etc.), the user's score for at least one test, etc. not limited

As an example, the electronic device 100 as a server may receive the user's learning purpose and/or learning level through the user terminal.

As another example, the electronic device 100 as a server may provide at least one test (eg, including a plurality of solving problems) suitable for the learning purpose input through the user terminal, According to the user's learning level can be identified.

In addition, the electronic device 100 may select at least one index word matching the user's learning purpose and learning level.

As an example, the electronic device 100 may search for at least one keyword related to a learning purpose and/or a learning level. In this case, the electronic device 100 may use its own database or at least one external server that provides a search service.

Here, the electronic device 100 may select at least one keyword having the highest ratio commonly included in the plurality of documents/contents obtained according to the search result as the index word.

As a specific example, when the user's learning objective is mathematics and the learning level is a first year of secondary school, the electronic device 100 may obtain keywords such as prime factorization, integer, rational number, and coordinate plane, which frequently appear in search results, and the corresponding You can select at least one index word among keywords.

In addition, the electronic device 100 may recommend at least one video content found according to the selected index word to the user.

In this case, there is an effect that a learning plan including optimal video contents suitable for the learning purpose and learning level of the user who is the learning target can be derived.

Meanwhile, according to an embodiment, the electronic device 100 may change the index word set for the video content according to the search amount of the video content according to the index word.

Specifically, it is assumed that 'index word AC' is set for 'video content AB'.

In this case, the electronic device 100 identifies the first number of times that 'video content AB' is searched through the 'index word AC' for a certain period (eg, the last week, the last month, etc.), and during the same period (regardless of whether or not the search is performed) In this case, it is possible to simply identify the second number of times 'video content AB' has been reproduced.

Here, if the ratio of the first number divided by the second number is less than the threshold ratio, the electronic device 100 may change the index word set for 'video content AB' from 'index word AC' to at least one other index word. .

Specifically, the electronic device 100 may obtain and set another index word by using at least one keyword (a keyword different from 'index word AC') included in the text in which the audio data of 'video content AB' is converted.

As such, when the search frequency (search efficiency) according to the index word is lower than the demand for actual video content, the electronic device 100 may change the index word to another index word, and as a result, validation and update of the utility of the index word in real time This enables smooth management of the indexing environment of video content.

Meanwhile, FIG. 7 is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 7 , the electronic device 100 may further include at least one of a communication unit 130 , a user input unit 140 , and an output unit 150 in addition to the memory 110 and the processor 120 .

Referring to FIG. 7 , the processor 120 includes an STT module 121 , a keyword extraction module 122 , an index management module 123 , a search module 124 , a content reproduction module 125 , and a bulletin board management module 126 . etc can be controlled. These modules may be implemented in hardware and/or software.

The STT module 121 is a module for converting audio data of video content into text, and may use an acoustic model, a language model, a pronunciation dictionary, and the like.

The keyword extraction module 122 is a module for extracting at least one keyword from the converted text, and may use natural language processing technology, an artificial intelligence model for keyword extraction, and the like.

The index management module 123 is a module for setting index words for at least one video content and at least one time section included in the video content.

The index management module 123 may change/update/delete at least one index word set in each video content.

The search module 124 is a module for searching for at least one video content through an index word.

The search module 124 may perform a search based on a database provided in the electronic device 100 or an external device, and this database may include matching information between video content and an index word.

The content reproduction module 125 is a module for providing a video content reproduction service.

The content playback module 125 may play video content stored in the memory 110 , and may provide a streaming service of video content based on communication with at least one external server.

The bulletin board management module 126 is a module for creating and managing a bulletin board provided for each index word.

As at least one index word is set for video content, the bulletin board management module 126 may generate a bulletin board for the index word.

The bulletin board management module 126 may store various user comments written in the bulletin board in the memory 110 .

The bulletin board management module 126 may use at least one artificial intelligence model trained to derive an index word matching the bulletin board and/or a correlation of a comment to a video content (time interval).

As an example, the AI model may be trained based on text converted from audio data included in at least one video content, an index word set for the video content, and user comments on the video content. In addition, the present AI model may be trained based on comments unrelated to the corresponding video content.

In this case, the bulletin board management module 126 may input each comment in the bulletin board into the artificial intelligence model, and identify a comment irrelevant to the index word and video content of the bulletin board. For example, when the correlation index output from the artificial intelligence model to which the comment is input is less than the threshold, the bulletin board management module 126 may identify the comment as noise.

Also, the bulletin board management module 126 may delete a corresponding comment from the bulletin board.

Meanwhile, a specific module configuration may be different from that of FIG. 7 , and modules for performing various functions other than the modules shown in FIG. 7 may be additionally included.

The communication unit 130 is a configuration for the electronic device 100 to transmit/receive data to and from various external devices, and may include at least one circuit for communication.

Communication unit 130 is TCP/IP (Transmission Control Protocol/Internet Protocol), UDP (User Datagram Protocol), HTTP (Hyper Text Transfer Protocol), HTTPS (Secure Hyper Text Transfer Protocol), FTP (File Transfer Protocol), SFTP ( A communication protocol (protocol) such as Secure File Transfer Protocol) and MQTT (Message Queuing Telemetry Transport) may be used to transmit/receive various information to and from one or more external electronic devices.

To this end, the communication unit 130 may be connected to an external device based on a network implemented through wired communication and/or wireless communication. In this case, the communication unit 130 may be directly connected to an external device, or may be connected to an external electronic device through one or more external servers (eg, Internet Service Providers (ISPs)) that provide a network.

The network may be a personal area network (PAN), a local area network (LAN), a wide area network (WAN), etc. depending on the area or size, and depending on the openness of the network, an intranet, It may be an extranet or the Internet.

Wireless communication includes long-term evolution (LTE), LTE Advance (LTE-A), 5th generation (5G) mobile communication, code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), DMA (Time Division Multiple Access), WiFi (Wi-Fi), WiFi Direct, Bluetooth, NFC (near field communication), at least one of the communication methods such as Zigbee may include

Wired communication may include at least one of communication methods such as Ethernet, optical network, USB (Universal Serial Bus), and ThunderBolt.

Here, the communication unit 130 may include a network interface or a network chip according to the above-described wired/wireless communication method. Meanwhile, the communication method is not limited to the above-described example, and may include a communication method newly appearing according to the development of technology.

When the electronic device 100 is a server, the electronic device 100 may communicate with various user terminals through the communication unit 130 , and interwork with each user terminal through at least one web page or application to enable each user The content indexing method according to the above-described various embodiments may be performed.

When the electronic device 100 is a user terminal such as a smartphone, the electronic device 100 indexes contents according to the above-described various embodiments through an application/web page provided by at least one external server connected through the communication unit 130 . At least some of the methods may be performed.

The user input unit 140 is configured to receive a user command or user information.

When the electronic device 100 is implemented as a user terminal such as a smartphone or a notebook PC, the user input unit 140 may be implemented as a touch sensor, a button, a camera, a microphone, a keyboard, and the like. When the electronic device 100 is a desktop PC, the user input unit 140 may be implemented as a terminal connected to various input interfaces such as a keyboard or a mouse.

The output unit 150 is configured to output various information and provide it to the user.

When the electronic device 100 is implemented as a user terminal such as a smart phone, the output unit 150 may include a display, a speaker, an earphone/headset terminal, and the like.

On the other hand, when the electronic device 100 is implemented as a server, the electronic device 100 does not include the user input unit 140 and the output unit 150 itself, but rather the user through the user terminal including the user input unit and the output unit. It can receive command/user information and output various information.

Meanwhile, the various embodiments described above may be implemented by combining a plurality of embodiments as long as they do not conflict with each other.

Meanwhile, the various embodiments described above may be implemented in a recording medium readable by a computer or a similar device using software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described in the present disclosure are ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays) ), processors, controllers, micro-controllers, microprocessors, and other electrical units for performing other functions may be implemented using at least one.

In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the above-described software modules may perform one or more functions and operations described herein.

Meanwhile, computer instructions or a computer program for performing a processing operation in each device in the system according to various embodiments of the present disclosure described above are stored in a non-transitory computer-readable medium. can be saved. When the computer instructions or computer program stored in the non-transitory computer-readable medium are executed by the processor of the specific device, the specific device performs the processing operation in the electronic device according to the various embodiments described above.

The non-transitory computer-readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, a cache, a memory, and can be read by a device. Specific examples of the non-transitory computer-readable medium may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present disclosure have been illustrated and described, but the present disclosure is not limited to the specific embodiments described above, and is generally used in the technical field belonging to the present disclosure without departing from the gist of the present disclosure as claimed in the claims. Various modifications may be made by those having the knowledge of

100: electronic device 110: memory
120: processor

Claims (10)

A method for indexing content in an electronic device, the method comprising:
obtaining at least one text from audio data included in the video content;
setting an index word for the video content based on at least one keyword included in the obtained text; and
When a user input requesting a search for the set index word is received, providing the video content;
The step of setting the index word is,
dividing the obtained text into a plurality of sub-texts based on a sentence, paragraph, or paragraph;
dividing the video content into a plurality of time sections matching each of the plurality of subtexts;
selecting an index word for at least one subtext among the plurality of subtexts,
setting the selected index word to match in a time section matching the subtext among the plurality of time sections included in the video content;
The content indexing method of the electronic device includes:
receiving user input requesting further explanation while the video content is being presented;
identifying a time section being reproduced at a time point when the user input requesting the additional explanation is received from among the plurality of time sections included in the video content;
searching for at least one other video content based on an index word matching the identified time interval; and
The content indexing method of the electronic device further comprising; providing the other searched video content. According to claim 1,
Obtaining the text includes:
A content indexing method of an electronic device, wherein text is obtained by converting a voice included in the audio data. delete According to claim 1,
Providing the video content comprises:
When a user input for receiving the video content searched according to the set index word is received, a time section matching the set index word among the plurality of time sections included in the video content is provided. . According to claim 1,
The content indexing method of the electronic device includes:
When an input from a user requesting evaluation information of the video content searched according to the set index word is received, providing a viewing service for a bulletin board including comments of other users for the time section matching the index word; Further comprising, a method for indexing content in an electronic device. 6. The method of claim 5,
The content indexing method of the electronic device includes:
When the time section of the video content searched for according to the set index word is provided to the user, providing at least one user interface for recording the user's comment for the time section in the bulletin board; further comprising , a method for indexing content in electronic devices. delete According to claim 1,
The content indexing method of the electronic device includes:
identifying the learning purpose and learning level of the user;
selecting at least one index word matching the learning purpose and learning level of the user; and
The content indexing method of the electronic device further comprising; recommending at least one video content found according to the selected index word. In an electronic device,
a memory in which at least one instruction is stored; and
A processor that performs the content indexing method of claim 1 by executing the instruction. In the computer program stored in a computer-readable medium,
A computer program stored on a computer-readable medium, which is executed by at least one processor of an electronic device to cause the electronic device to perform the content indexing method of claim 1 .

Images