JP2022051500A - Related information provision method and system - Google Patents

Abstract

To provide a related information provision method that can provide useful information to a user by utilizing a moving image format.SOLUTION: A related information provision method according to the present invention includes the steps of playing a moving image content including subtitles in a first area of a display unit, outputting a graphic object including a part of a text constituting the subtitles to a second area of the display unit, and providing related information on the part of the text on the basis of a selection signal to the graphic object, and in the subtitles, the part of the text is highlighted and displayed so as to be distinguished from the text of the other part.SELECTED DRAWING: Figure 1

Description

The present invention relates to a method and a system for providing related information using a moving image.

As technology evolves, the use of digital devices is increasing. In particular, electronic devices (for example, smartphones, tablet PCs, etc.) have various functions such as surfing the Internet using the Internet, listening to music, and watching video contents, as well as communication functions such as telephone calls and text messages.

With the popularization of such electronic devices, the consumption of multimedia contents is rapidly increasing, and in particular, the consumption of moving images (or moving image contents) is gradually increasing.

Also, with the development of content utilizing electronic devices, users tend to prefer more visualized content such as moving images to content consisting only of text.

Therefore, along with such a trend, services related to a method of providing useful information to a user by utilizing a moving image format are being actively developed.

The present invention is for providing a related information providing method and system capable of providing useful information to a user by utilizing a moving image format.

Further, the present invention is for providing a related information providing method and system for constructing a user environment capable of providing related information regarding a content composed of moving images.

Furthermore, the present invention is intended to provide a related information providing method and system capable of more intuitively and organically providing deeper information in cooperation with a content consisting of a moving image.

In order to solve the above problems, the related information providing method according to the present invention includes a step of reproducing moving image content including subtitles in a first area of a display unit, and a graphic object including a part of text constituting the subtitles. The subtitle includes a step of outputting to a second area of the display unit and a step of providing related information about the part of the text based on the selection signal for the graphic object, and the part of the text is the other part in the subtitle. It is characterized by being highlighted and displayed so as to be distinguished from the text of.

Further, the related information providing system according to the present invention reproduces the communication unit, the display unit including the display area divided into the first area and the second area, and the moving image content including the subtitles in the first area, and the first. The two regions include a control unit that controls the display unit so as to output a graphic object including a part of the text constituting the subtitle, and the control unit includes the part based on a selection signal for the graphic object. The display unit is controlled so as to output related information regarding the text of the above, and in the subtitle, the text of the part is highlighted and displayed so as to be distinguished from the text of the other part. It is a feature.

Further, the program that can be stored in the computer-readable medium according to the present invention is executed by one or more processes in an electronic device, and constitutes the step of reproducing the moving image content including the subtitle in the first area of the display unit, the subtitle. The command comprises a step of outputting a graphic object including a part of the text to a second area of the display unit, and a step of providing related information about the part of the text based on a selection signal for the graphic object. In the subtitle, the part of the text is highlighted and displayed so as to be distinguished from the text of the other part.

As described above, in the related information providing method according to the present invention, the display area of the display unit is divided into a plurality of areas, the moving image content is related to one of the areas, and the information provided by the moving image content is related to the other area. Information can be provided. Therefore, the user can more intuitively use the information related to the information provided by the moving image content at any time while using the moving image content. As a result, the user can avoid the complexity of moving to another function to search for related information and searching for the information.

It is a conceptual diagram for demonstrating the related information provision system by this invention. It is a flowchart for demonstrating the related information provision method by this invention. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of shifting to the function in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the method of providing the related information by this invention, and the method of providing the related information in a system. It is a conceptual diagram for demonstrating the moving image content reproduction function in the related information provision method and the system by this invention. It is a conceptual diagram for demonstrating the moving image content reproduction function in the related information provision method and the system by this invention. It is a conceptual diagram for demonstrating the moving image content reproduction function in the related information provision method and the system by this invention. It is a conceptual diagram for demonstrating the moving image content reproduction function in the related information provision method and the system by this invention. It is a conceptual diagram for demonstrating the moving image content reproduction function in the related information provision method and the system by this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components will be designated by the same reference numerals regardless of the drawing numbers, and duplicate description thereof will be omitted. The component suffixes "module" and "part" used in the following description are added or mixed to facilitate the preparation of the specification, and are themselves significant or useful. I don't have it. In addition, in explaining the embodiment of the present invention, if it is determined that the specific description of the related publicly known technique makes the gist of the embodiment of the present invention unclear, the detailed description thereof will be omitted. Further, the accompanying drawings are merely to assist in understanding the embodiments of the present invention, and the accompanying drawings do not limit the technical idea of the present invention, and any modification included in the idea and the technical scope of the present invention. , Should be understood to include equivalents or alternatives.

Terms including ordinal numbers, such as "first", "second", etc., are used to describe various components, but the components are not limited by the above terms. The above terms are used only to distinguish one component from the other.

When it is mentioned that one component is "connected" or "connected" to another component, it may be directly connected or connected to another component, with yet another component in between. It should be understood that it may be acceptable. On the other hand, when it is mentioned that one component is "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The singular representation includes multiple representations unless otherwise noted.

As used herein, terms such as "include" and "have" are intended to specify the existence of features, numbers, stages, actions, components, parts or combinations thereof described herein. It should be understood that the existence or addability of one or more other features, numbers, stages, actions, components, parts or combinations thereof is not preliminarily excluded.

The present invention is for providing a related information providing method and system capable of providing useful information to a user by utilizing a moving image format. In particular, the present invention is for providing a related information providing method and system for constructing a user environment capable of providing related information regarding the content provided by the moving image together with the moving image.

Hereinafter, the present invention will be specifically described with reference to the accompanying drawings. FIG. 1 is a conceptual diagram for explaining a related information providing system according to the present invention.

As shown in FIG. 1, the related information providing system 100 according to the present invention may include at least one of a communication unit 110, a storage unit 120, and a control unit 130. Here, the related information providing system 100 according to the present invention is not limited to the above-mentioned components, and may further include components that play the same or similar roles as the functions described in the present specification.

The related information providing system 100 according to the present invention may be operated in conjunction with the electronic device 10. Here, the electronic device 10 may include at least one of a smartphone, a mobile phone, a tablet PC, a kiosk, a computer, a notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), and a PMP (Portable Multimedia Player). good.

Further, the related information providing system 100 according to the present invention may be realized by an application or software. Here, the application or software of the related information providing system 100 may be installed in the electronic device 10 and perform a process by the related information providing method described below.

For example, the communication unit 110, the storage unit 120, and the control unit 130 of the related information providing system 100 according to the present invention use the hardware configuration of the electronic device 10 to execute a series of functions for providing related information. You may. For example, the communication unit 110 uses the configuration of a communication module (for example, a mobile communication module, a short-range communication module, a wireless Internet module, a location information module, a broadcast reception module, etc.) provided in the communication unit of the electronic device 10. Data may be sent and received to and from at least one external server 140 (for example, a database server (DB server), a cloud server, etc.).

The storage unit 120 uses the memory (or inserted) provided in the electronic device 10 or the external server 140 (for example, a database server, a cloud server, etc.) to provide the moving image content and the related information of the moving image content according to the present invention. May be saved.

At least a part of the storage unit 120 may be an external server (at least one of a cloud server and a database (DB)). That is, the storage unit 120 is sufficient as long as it is a space that can store the information necessary for providing the related information according to the present invention, and there is no limitation on the physical space. Therefore, in the following, the configurations of the storage unit 120 and the external server 140 are not classified, the cloud server and the database are not separately classified, and all are expressed as the storage unit 120. Here, the cloud server can be "cloud storage". On the other hand, the access to the external server may be performed by the communication unit 110.

The control unit 130 may use at least one of the CPU and the cloud server provided in the electronic device 10 to control each configuration and perform general control according to the present invention. Further, the control unit 130 may have the control authority of at least one component of the electronic device 10 in order to realize the present invention. Further, the control unit 130 may control the display unit 11 of the electronic device 10 to output the moving image content provided in the present invention and related information thereof to the display unit 11. Here, the control unit 130 may directly control the display unit 11, and indirectly controls the display unit 11 by transmitting a control command to the control unit of the electronic device 10 that controls the display unit 11. It may be controlled.

As described above, in the present invention, it is possible to provide related information regarding the moving image content together with the moving image content.

In the present invention, the moving image content is a content having a moving picture, video format, and the content constituting the moving picture content is very diverse. For example, the moving image content according to the present invention may be composed of not only news articles, newspaper articles, magazine articles or the text of books, but also contents related to a specific theme.

Further, the moving image content according to the present invention is the content included in the posted material posted (or posted) on a site accessible via the Internet (for example, a blog, a moving image content providing site, other web pages, etc.). It may be configured. In this case, the type of bulletin board is not limited, and the bulletin board may include at least one of an image, a sound, a text, and a moving image. Here, the image may be at least one of a static image and a dynamic image.

The moving image content according to the present invention may be configured to include image data, audio data, and subtitle data. Here, the image data may be composed of at least one of a static image and a dynamic image.

For example, when the moving image content includes image data and subtitle data corresponding to a static image, in the moving image content according to the present invention, the text corresponding to the subtitle data overlaps with the still image corresponding to the static image. It may be content.

When the moving image content according to the present invention is reproduced by the electronic device 10, at least two of the image, the sound, and the subtitle may be combined and output together.

The moving image content according to the present invention may be generated by the control unit 130 according to the present invention, or may be generated by a third subject (for example, a video producer, a content producer). ..

As an example of the method for generating moving image content according to the present invention, the control unit 130 may generate moving image content by converting text into voice based on TTS (Text to Speech) technology. .. Here, the converted audio becomes the audio data of the moving image content, and the text to be converted becomes the subtitle data of the moving image content. In this case, the control unit 130 uses the image data of the moving image content as a static image or a dynamic image included in the document including the text to be converted or having the content related to the text to be converted. It may be used as. Further, the control unit 130 may use an arbitrary image preset to be used when generating the moving image content as the image data of the moving image content.

On the other hand, the control unit 130 controls the sync between the subtitle data and the audio data so that when the audio data is output by the electronic device 10, the subtitle data related to the audio data is output. May be generated.

When video content is generated based on TTS technology in this way, the text to be converted provides information on a particular subject, such as news articles, newspaper articles, magazine articles, book text, and so on. It may be the text contained in the document created to do so.

As another example of the method for generating moving image content according to the present invention, the control unit 130 is configured to generate moving image content by generating subtitle data related to audio data of an existing moving image file. May be good.

Here, the control unit 130 may convert the audio data included in the moving image file into text based on the STT (Speech to Text) technology, and generate the subtitle data using the converted text. .. Then, the moving image content according to the present invention may be generated by inserting the generated subtitle data into the moving image file.

Here, the moving image file can be a moving image file of various themes.

As described above, the moving image content according to the present invention is an image including subtitle data, and in the present invention, the subtitle data and the subtitle are unified and referred to as "subtitles, caption" for convenience of explanation. The subtitle consists of at least one of letters, numbers and symbols, and may be configured to match or correspond to the audio data provided by the moving image content.

Further, as described above, the subtitles included in the moving image content according to the present invention are generated or inserted by various methods.

On the other hand, in the following, the above-mentioned moving image content generation method is not classified, and the description will be made under the assumption that the moving image content including subtitles exists.

The control unit 130 according to the present invention provides information related to or related to a part of the text included in the subtitle. In the present invention, for convenience of explanation, this is referred to as "related information". The relevant information may be about a portion of the text contained in the subtitles.

Here, the partial text may correspond to a word included in the subtitle. A word is a word that can be used independently and independently, or a word equivalent to this, and corresponds to the part excluding particles when the text is divided into verse units.

In the present invention, the partial text may be a text that matches or corresponds to a keyword stored (or registered) in a preset database. Here, the preset database is a database in which information (or contents) related to keywords is stored, and there is no limitation on the type thereof.

In the preset database, related information about the keyword corresponding to the part of the text may be stored in a match with the keyword. The information stored in the preset database may be divided by keyword. Here, the keyword can be said to be a tag, a representative word, or the like.

On the other hand, the preset database (hereinafter referred to as "DB") may be composed of a language dictionary DB, a knowledge encyclopedia DB, a music information DB, a person information DB, and the like.

Examples of related information provided based on such a preset DB include person information corresponding to a part of text, dictionary definition, encyclopedia definition, search information in a preset search engine, and so on. Issue information, news information, newspaper information, article information, etc. can be mentioned.

On the other hand, the information constituting the preset DB (for example, individual content) may match at least one keyword. Each of the individual contents may match a plurality of keywords.

Further, the same keyword may be matched to each of different individual contents. For example, the keyword corresponding to the person information "Hong Gildon" is included in the first individual content (for example, person information content) and the second individual content (for example, review content related to a store with the trade name "Hong Gildon"), respectively. You may try to match.

The control unit 130 may determine whether or not the text included in the subtitle contains the same or similar text as the keyword registered in the preset DB.

The control unit 130 may divide the text constituting the subtitle into verse units, remove particles from the divided verses, and extract meaningful words. Further, it may be possible to search from a preset DB whether or not related information corresponding to the word exists.

On the other hand, as described above, when there are a plurality of contents (or related information) linked to the keyword, the control unit 130 has a relationship with the moving image content based on an algorithm related to a preset weighted value. High content may be searched for as related information.

In this way, the control unit 130 can search whether or not the related information corresponding to the partial text exists in the preset DB. The control unit 130 can search whether or not related information exists in a preset DB using the partial text as a search term (or keyword). Here, the partial text may be understood as a word included in the subtitles.

For example, if the partial text is a "restraint warrant" and the knowledge encyclopedia DB has related information with the keyword "restraint warrant" (for example, the definition of the word "restraint warrant"), the control unit 130 , It is determined that the relevant information corresponding to the part of the text exists.

Further, when the keyword registered in the preset DB and a part of the text included in the subtitle correspond to each other, the control unit 130 displays the part of the text separately from the text of the other part of the subtitle. You may do so. In the present invention, displaying a part of the text separately from the text of the other part is referred to as "highlighting" the part of the text.

There are various methods for highlighting the part of the text, and the control unit 130 displays the visual appearance of the part of the text so as to be distinguished from the visual appearance of the other part of the text. Can be done.

The highlighting process is i) blinking the emphasized part, ii) displaying with a thick line, iii) increasing the contrast, iv) inverting the black and white part of the screen (reverse video), v) underlining. It means software-like processing such as subtracting or displaying in other colors.

That is, the control unit 130 displays at least one of the color, size, weight, and font type of the part of the text so as to be different from the text of the other part, so that the related information regarding the part of the text can be obtained. The user may be made aware that it exists.

Further, the control unit 130 may perform highlight processing on the part of the text by displaying the graphic object on the part of the subtitle in which the part of the text is arranged.

By doing so, the word corresponding to the "restraint warrant" in the above example is displayed so as to have a visual appearance different from that of the text of other parts.

In this way, the control unit 130 provides related information about a part of the text of the subtitle, and in various ways, the related information about the part of the text exists so that the user can intuitively access the related information. The user can be notified.

In the present invention, the control unit 130 organically provides the moving image content and the related information by using switching between a plurality of pages having different characteristics in order to provide the related information regarding the moving image content. You can also do it.

Hereinafter, the method for providing related information according to the present invention will be specifically described with reference to the accompanying drawings. FIG. 2 is a flowchart for explaining the related information providing method according to the present invention, and FIG. 3 is a conceptual diagram for explaining the related information providing method according to the present invention and the method for shifting to the function in the system. Further, FIGS. 4a, 4b, 4c, 4d, 4e and 4f are conceptual diagrams for explaining the method for providing related information according to the present invention and the method for providing related information in the system. Further, FIGS. 5a, 5b, 5c, 6a and 6b are conceptual diagrams for explaining the related information providing method and the moving image content reproduction function in the system according to the present invention.

In the related information providing method according to the present invention, first, as shown in FIG. 2, a process of reproducing a moving image (or moving image content) including subtitles in the first region is performed (S210).

First, in the present invention, the transition to the moving image content reproduction function is performed by various routes.

The control unit 130 reproduces moving image contents based on various methods of user input to the electronic device 10.

As an example, as shown in FIG. 3A, the control unit 130 may reproduce the moving image content based on the input of the user's voice. The control unit 130 may control the electronic device 10 based on an utterance word (command word, starting word, or starting word) that matches the reproduction of the moving image content so that the moving image content is reproduced. For example, when "Let me hear the news" 301 is set as the utterance word, the control unit 130 causes the user to play the moving image content based on the voice corresponding to the set utterance word. You may do it.

On the other hand, the electronic device 10 may be in a state in which an application or a program according to the related information providing method according to the present invention is executed.

As another example, as shown in FIGS. 3B and 3C, the control unit 130 selects moving image contents based on the selection of predetermined icons 302 and 304 output to the electronic device 10. May be played. Here, the predetermined icons 302 and 304 may be output on the migration page of a specific application as shown in FIG. 3 (b), and may be output as shown in FIG. 3 (c). It may be output on the main page of (for example, function providing page, information providing page, etc.).

Here, as shown in FIGS. 3B and 3C, the control unit 130 outputs the guidance information 303 and 305 notifying that the moving image content can be provided to the display unit 11. You may. Therefore, the user can recognize that the moving image content can be reproduced and can immediately access the moving image content providing function.

In the present invention, as shown in FIGS. 4a (a) and 4a, the control unit 130 is a moving image on a page including the first region a and the second region b (hereinafter referred to as “first page”). The image content may be played back.

As shown in FIGS. 4a (a) and 4a, the first area a is an area in which the moving image content is reproduced, and the second area b is an area in which related information regarding the moving image content is provided. There may be. As described above, the second region b may be provided with related information regarding the texts 401 and 402 of a part of the subtitle included in the moving image content.

On the other hand, as shown in FIGS. 4a (a) and 4a, the moving image content reproduced in the first region a is configured to include subtitles, and the information related to the preset DB of the subtitles is included. A portion of the text (or word) in which is present (eg, forced ugly line 401, Hong Gildon 402) may be highlighted and displayed.

Further, as shown in FIG. 4b (a), in the first region a, a progress bar (or timeline) 410 indicating the degree of reproduction of the moving image (or moving image content) reproduced in the first region a. May be further included.

The progress bar 410 may provide information on the current reproduction progress of the moving image content being reproduced with respect to the total reproduction time.

In the present invention, the progress bar 410 may be configured to include at least one indicators 410a and 410b. The control unit 130 identifies a position corresponding to the output point (or reproduction point, reproduction time point) of a part of the text in which the related information exists in the progress bar 410, and includes the indicators 410a and 410b at the specified position. You may do it. The control unit 130 may match the indicators 410a and 410b with a part of the text (or word), respectively. Therefore, when the control unit 130 receives the selection signal for selecting the indicators 410a and 410b, the control unit 130 controls the reproduction of the moving image content so that the portion including a part of the text matching the selected indicator as a subtitle is reproduced. You may try to do it. Based on the indicator being selected, the control unit 130 is a display unit so that the moving image content is reproduced from the point where the selected indicator is matched and the highlighted part of the text is included as the subtitle. 11 may be controlled.

On the other hand, when the first page is output to the display unit 11 or the control unit 130 has not received a separate control command regarding the cancellation of the reproduction of the moving image content, the moving image content continues in the first area a. The display unit 11 may be controlled so that the content can be reproduced. The control unit 130 may continuously reproduce the moving image content having a predetermined reproduction time in the first region a.

Here, the types or themes of the moving image contents reproduced in the first region a are very diverse. The control unit 130 may select the type or theme of the moving image content to be reproduced in the first region a by the user's selection, the user's preference, the situation information, and the control of the control unit.

As an example, the control unit 130 may play back moving image content related to a user's interest theme (for example, current affairs, economy, etc.) based on the user history information. In this case, the control unit 130 may play back different moving image contents related to the user's theme of interest in the first region a. The user history information may be set to match the user account information.

As another example, the control unit 130 may provide moving image content relating to the type or theme input by the user. There are various methods for inputting the type or theme of the moving image content from the user, and as shown in FIG. 3A, the type or theme of the moving image content to be reproduced can be selected by the user by the voice recognition method. It may be entered.

Further, when the user inputs information about the type or theme of the moving image content, the control unit 130 may select the moving image content corresponding to the input information.

The control unit 130 may specify the moving image content provided in the first region a based on a preset standard. Here, the preset criteria can be set in a great variety.

As an example, when a user inputs a specific keyword and is requested to provide news, the control unit 130 responds to a preset number (for example, 10) of news related to the specific keyword. You may try to specify the moving image content to be performed. In this case, the control unit 130 may specify a preset number of moving image contents based on an index such as the number of accesses, the number of high-ranking displays, and the degree of popularity.

As another example, when the user requests the provision of news, the control unit 130 preliminarily divides the news into various categories (for example, categories such as politics, economy, society, IT, life, and the world). The set number of moving image contents may be specified respectively.

As yet another example, when the user requests the provision of popular news by category, the control unit 130 corresponds to the most accessed (or most displayed) news by category in advance. A set number of moving image contents may be specified.

As yet another example, when the user requests the provision of news about the real-time popular search term (or keyword), the control unit 130 may specify the moving image content of the news about the real-time popular search term. ..

As yet another example, when the user requests the provision of the latest news, the control unit 130 may specify the moving image content based on the latest uploaded news.

As yet another example, when the user requests the provision of news of a specific media, the control unit 130 may specify the moving image content of the news provided by the specific media.

As yet another example, the control unit 130 may select the type or theme of the video content to be provided based on the situation information.

Here, the types of situational information are very diverse. For example, the status information includes time information (or time zone information), location information, device connection information, weather information, and the like.

Such status information includes the time when the moving image content is provided, the location where the electronic device provided with the moving image content is located, and the device connected to the electronic device provided with the moving image content (for example, wireless communication (for example, wireless communication)). , Bluetooth®) may be the status information about the device to which the video content is provided or the video content is provided, such as a device (eg, a speaker) connected via Bluetooth.

As an example, the control unit 130 may provide different types of moving image content depending on the time zone in which the moving image content is provided (for example, the time zone in which the user requests the provision of the moving image content). The control unit 130 may specify moving image contents of different types or themes for different time zones. For example, the control unit 130 displays moving image contents of themes such as weather, traffic conditions, front page articles of paper newspapers, and news frequently read during the night in the first time zone (for example, the morning time zone). , May be specified as the content to be provided to the electronic device 10. In this case, the control unit 130 may continuously provide the moving image content of the theme to the electronic device 10.

Further, the control unit 130 is requested to specify the moving image content of the theme related to cooking, the most read news today, etc. as the content to be provided to the electronic device 10 in the second time zone (for example, the evening time zone). You may do it.

As another example, when the electronic device 10 communicates with a device included in an automobile (for example, a vehicle speaker), the control unit 130 provides moving image content of a theme related to traffic information to the electronic device 10 as content. You may try to specify.

As yet another example, the control unit 130 is of a type or theme related to the screen information output by the electronic device 10, the function most recently executed by the electronic device 10, or the screen information most recently output by the electronic device 10. The moving image content may be specified as the content to be provided to the electronic device 10.

Therefore, even if the same user input for providing the moving image content is performed to the electronic device 10, the user can receive the moving image content of a different type or theme suitable for the current situation.

On the other hand, the control unit 130 specifies the moving image content based on the criteria in the various examples described above by the control unit 130 itself in addition to the user's request, and the specified moving image content is stored in the first region a. It may be provided.

Next, in the present invention, a process of displaying a graphic object corresponding to a part of the text included in the subtitle in the second area b is performed (S220).

Here, a part of the text (or a part of the texts 401 and 402) means a highlighted part of the subtitles included in the moving image content reproduced in the first region a.

As shown in FIG. 4a (a), the graphic objects 404, 407 may be configured to include a highlighted portion of the text (401, 402). Therefore, the user can recognize which part the related information exists.

The control unit 130 may make the graphic objects 404 and 407 include at least one of the partial text and the additional text included in the subtitle.

Here, the additional text is guidance information for eliciting the user's interest in a part of the text, and as shown in FIG. 4a, the question-type text (for example, "Would you like to inform us of the person information of **?" , "What is the exact meaning of **?").

The control unit 130 may generate additional text according to the type, theme or category (hereinafter, referred to as “attribute”) of related information regarding a part of the text. As an example, when the related information is person information, as shown in FIG. 4a (a), the control unit 130 notifies the additional text (** person information) including the keyword (person information) corresponding to the related information. Shall we do it?) May be generated. As another example, when the related information is information corresponding to the definition of the dictionary, as shown in (a) of FIG. 4a, the control unit 130 has an additional text (*) including a keyword (meaning) corresponding to the related information. What is the exact meaning of *?) May be generated.

In this way, the control unit 130 generates an additional text (or question) including a keyword corresponding to the attribute of the related information according to the attribute of the related information, and puts it in the second area b together with a part of the text included in the subtitle. Can be provided. Therefore, the user can immediately access the information of interest or interest.

On the other hand, the graphic objects 404 and 407 output to the second region b may be configured to include at least one of characters, symbols, numbers and images.

Further, as shown in FIG. 4a (a), when there are a plurality of highlighted partial texts in the subtitles (401, 402), the graphic object 404 corresponding to the plurality of partial texts in the second area b, respectively. , 407 may be output.

On the other hand, the second area b may include not only a graphic object including a part of the text included in the subtitle but also a graphic object including related information of the moving image content output to the first area a. Here, the information related to the moving image content may be information in the form of a question or an instruction. The control unit 130 may generate information in the form of a question or a command based on the content of the moving image content output to the first area a, and output it to the second area b. Here, the information in the form of a question or instruction is the output of other content related to the moving image content output to the first area a (for example, a news article, an SNS (Social Network Service) bulletin board, a blog bulletin board, etc.). It may be information for inducing.

If there is other content related to the moving image content output in the first area a, the control unit 130 may provide the second area b with information in the form of a question or an instruction.

For example, the information output to the second area b is "Tell me about the background of this case", "What is the point of discussion?", "What happened to the judgment in this case?" , May consist of information that guides access to other content.

Here, the control unit 130 analyzes the content of the moving image content output to the first region a based on a predetermined algorithm, and generates appropriate question or instruction format information based on the analyzed content. You may do so.

Further, the control unit 130 may generate appropriate question or command format information based on the contents included in other contents related to the moving image contents output to the first area a. For example, when there is content related to a background incident of the moving image content output to the first region a as other content, the control unit 130 generates information in the form of an instruction "Tell me about the background of this incident". , It may be output to the second area b.

The control unit 130 may generate information in the form of a question or an instruction related to the moving image content output to the first region a based on a deep learning or artificial intelligence-based algorithm.

On the other hand, the control unit 130 may provide the other content to the electronic device 10 based on the selection of the graphic object including the question or instruction format information. This can be configured in the same manner as in the embodiment of "outputting information (or content) based on the selection of a graphic object including a part of the text included in the subtitle" described later together with step S230. Is omitted.

Further, the second area b includes various information in addition to the information regarding the subtitles. As shown in (a) of FIG. 4a and (a) of FIG. 4f, the graphic object 403 for receiving the input of the user's preference information regarding the moving image content being reproduced in the first area a in the second area b. 460, 461, 462 may be included.

The control unit 130 may update the user's history information when the user's input is received by the graphic objects 403, 460, 461, and 462 shown in FIG. 4a (a) and FIG. 4f (a). ..

The control unit 130 updates the user's history information so that the user's preference information regarding attributes (for example, type, theme, category, etc.) related to the moving image content being played is reflected in the user's history information. May be good.

Further, the control unit 130 may select (or specify) the moving image content to be reproduced in the first region a by using the user's history information including the user's preference information.

For example, when the graphic object 461 "interested" is selected in the second area b, the control unit 130 sets the attribute related to the moving image content being played back as the user's preference information (or interest information). You may. In this case, the control unit 130 reproduces the moving image content related to the attribute of the moving image content currently being reproduced when the reproduction of the moving image content currently being reproduced is completed and the next moving image content is reproduced. You may do so.

On the other hand, when the graphic object 462 "not interested" is selected in the second area b, the control unit 130 sets the attribute related to the moving image content being played as the user's non-preference information (or non-interest information). You may set it. In this case, when the control unit 130 finishes playing the moving image content currently being played and plays the next moving image content, the control unit 130 has an attribute different from the attribute of the moving image content currently being played. May be played.

Further, the control unit 130 may provide information on the theme of interest to the user to the second region b, as shown in FIG. 4f (b). The control unit 130 may count the number of times of input of interest degree information (or preference information) regarding the theme in which the user is interested. Further, by providing the count information to the user, the control unit 130 may notify that the user's degree of interest is accurately recognized and monitored.

In addition, the second region b can be configured to include various information about the moving image content provided in the first region a.

On the other hand, the graphic object output to the second area b may be changed in conjunction with the change of the subtitle of the moving image content output in the first area a, and the moving image content is changed. It may be changed in conjunction with the above. That is, when the highlighted text is changed due to the change of the subtitle in the first area a, the graphic object corresponding to the changed text is output to the second area b in conjunction with it. You may do so. When the subtitle is changed due to the reproduction of the moving image content, the control unit 130 can change the information output to the second area b in conjunction with the change.

Next, in the present invention, a process of displaying related information regarding a part of the text included in the subtitle is performed based on the selection of the graphic object output to the second area b (S230).

The control unit 130 may output the related information regarding the part of the text to the display unit 11 based on the selection signal for the graphic object regarding the part of the text included in the subtitle.

As described above, the related information is the information stored in the preset DB, and includes the content related to the highlighted part of the text.

The control unit 130 can also provide related information 409 and 414 to the second region b as shown in (b) of FIG. 4a and (b) of FIG. 4b, and as shown in (c) of FIG. 4b. , You can also provide relevant information on the new page 415.

In the present invention, the new page 415 is a page different from the first page including the first region a and the second region b, and is referred to as a "second page".

The control unit 130 may switch the first page to the second page 415 in order to provide relevant information regarding a part of the text. The control unit 130 may control the display unit 11 so that the second page 415 is output based on the selection signal for the graphic object. The second page 415 may include relevant information about a portion of the text.

On the other hand, the related information output to the second area b may be at least a part of the related information output to the second page 415. The control unit 130 may provide at least a part of the related information to the second region b based on the selection signal for the graphic object, and may provide the related information to the second page 415 based on the further selection signal. good.

For convenience of explanation, the related information provided in the second area b is referred to as "first related information", and the related information provided in the second page 415 is referred to as "second related information".

The control unit 130 may sequentially output the first related information and the second related information based on the selection signals that are sequentially input. Here, the selection signal may be a user input (for example, touch input or the like) to the display unit 11. In this case, the first related information may be at least a part of the second related information.

In the present invention, the selection signal for the graphic object corresponding to the text of the highlighted portion of the subtitle may include the first selection signal and the second selection signal.

Here, the first selection signal, as shown in (a) of FIG. 4a and (a) of FIG. 4b, before the relevant information regarding a part of the text (for example, Hong Gildon, restraint warrant) is output. It may be a selection signal for selecting graphic objects 404, 406, 411, 413. Based on such a first selection signal, the control unit 130 relates to the part of the text (for example, Hong Gildon, restraint warrant) as shown in FIG. 4a (b) and FIG. 4b (b). 1 Related information 409, 414 may be output to the second area b on the first page. The first related information may be output as voice information at the request of the user or at the control of the control unit 130.

Further, the second selection signal may be a selection signal for selecting the graphic object 412 in a state where the first related information 414 is output, as shown in FIG. 4b (b). The second selection signal may be a user input (eg, touch input, etc.) to any of the graphic object 412 and the first related information 414. When the second selection signal to the graphic object 412 is input, the control unit 130 has a second page containing a second related information regarding a part of the text (for example, a restraint warrant) as shown in FIG. 4b (c). The 415 may be output to the display unit 11.

That is, the control unit 130 can switch the first page output to the display unit 11 to the second page based on the second selection signal. In this way, the control unit 130 can sequentially display the first related information and the second related information based on the first selection signal and the second selection signal. At this time, when the user wants to know the contents related to the first related information in more detail, the user can receive the provision of the second related information by inputting the second selection signal.

On the other hand, as shown in FIGS. 4c (a) and 4c, the second page 422 is output based on not only the selection signal for the graphic object related to a part of the text but also the selection of the preset function icon 421. You may do so. In this case, the control unit 130 may output the related information about the graphic object related to a part of the text or the related information about the moving image content (for example, a news article) to the second page 422.

On the other hand, as described with reference to FIGS. 4b and 4c, when the second page 415 and 422 are output to the display unit 11, the moving image content reproduced in the first region a of the first page is continuously reproduced. It may be done. That is, the control unit 130 controls the electronic device 10 so that the reproduction of the moving image content reproduced in the first region a is maintained even if the second page 415 and 422 are output to the display unit 11. You may do so. In this case, the electronic device 10 may process the reproduction function of the moving image content and the output function of the second page 415 and 422 by a multitask method. In this case, the second page 415 and 422 are output to the display unit 11 of the electronic device 10, and the sound corresponding to the audio data of the moving image content is output from the acoustic output unit of the electronic device 10.

On the other hand, when the first page output to the display unit 11 is switched to the second page 415, 422, the control unit 130 puts an image (or graphic object,) related to the control of the moving image content in one area of the display unit 11. The GUI object 416, 423) may be output. The GUI objects 416 and 423 can also be said to be floating objects or floating icons.

As shown in (c) of FIG. 4b and (b) of FIG. 4c, one area of page 415 and 422 is a GUI relating to control of moving image content reproduced in the first area a of page 1. Objects 416 and 423 may be output.

The GUI objects 416 and 423 may be configured so that at least one of the output position and the output size on the second page 415 and 422 is changed based on the user input (for example, touch input) to the GUI objects 416 and 423. ..

The GUI objects 416 and 423 serve as a medium with the first page on which the moving image content is being reproduced, and the control unit 130 is used by the user input (for example, preset) to the GUI objects 416 and 423. The first page may be output to the display unit 11 based on the user input (or touch input) of the method. In this case, the second page 415 and 422 output to the display unit 11 may be switched to the first page based on the user input of the preset method.

On the other hand, as shown in FIG. 5a (a), the control unit 130 may use the GUI object 510 to provide related information regarding the moving image content. The control unit 130 changes the subtitles of the moving image content by playing back the moving image content, and shows a part of the text (or word) in which the related information is present, as shown in FIG. 5a (a). As such, the graphic object 511 corresponding to a part of the text in which the related information exists may be output to the GUI object 510 or around the GUI object 510. The control unit 130 may output the graphic object 511 to the display unit 11 when there is related information regarding the moving image content of the portion currently being reproduced.

Here, the graphic object 511 may include at least a part of the text (for example, Daiho-in Temple) contained in the subtitles.

Further, the control unit 130 displays related information regarding a part of the text related to the graphic object 511 (for example, Daiho-in Temple) as shown in FIG. 5a (b) based on the selection signal for the graphic object 511. It may be output to. The control unit 130 changes the related information already output to the second page, or provides the related information about a part of the text related to the graphic object 511 (for example, Daiho-in Temple) by the new third page 502. You may do so.

Further, the control unit 130 may use the GUI object 510 as a control screen for receiving input of a control command for the moving image content to be reproduced in the first region a.

For example, as shown in FIG. 5b, the GUI object 510 may include menu screens 520 and 521 for receiving selection of attributes (eg, themes) of moving image content to be reproduced in the first region a. When a specific theme is selected on the menu screens 520 and 521, the control unit 130 may control the electronic device 10 so that the moving image content of the selected specific theme is reproduced.

Further, as shown in FIG. 5c, the GUI object 510 controls the reproduction of the moving image content to be reproduced in the first region a (for example, pause, resume reproduction, volume adjustment, select the next moving image content, and before. Control menu screens 531, 532, 533, 534 for selecting moving image contents, etc.) may be included.

Such a menu screen may be output around the graphic object 510 as shown in the figure based on a user input (for example, long touch input) of a preset method to the graphic object 510. good. Further, such a menu screen may be provided as a function key included in the graphic object 510.

As described above, in the present invention, by maintaining the reproduction of the moving image content and organically providing the related information regarding the moving image content, the user can access the necessary information more intuitively. Can be done.

It has been described above that the moving image content according to the present invention can be generated by the control unit 130. The control unit 130 may generate moving image content by converting text into voice based on TTS technology. Here, the converted audio becomes the audio data of the moving image content, and the text to be converted becomes the subtitle data of the moving image content. In this case, the control unit 130 uses the image data of the moving image content as a static image or a dynamic image included in the document including the text to be converted or having the content related to the text to be converted. It may be used as. Further, the control unit 130 may use an arbitrary image (see FIG. 4d) preset to be used when generating the moving image content as the image data of the moving image content.

As shown in the figure, the preset image may be an image including a human face. As shown in the figure, the preset image may be an image of an actual person's face, avatar, or character format. When the image corresponding to the human face is output in this way, the user feels as if the announcer or the anchor has spoken the audio data corresponding to the moving image content.

In this case, the voice data may be generated by the TTS method, and the attributes of the voice that utters the voice data may be changed based on the user's selection. As shown in FIG. 4e, the control unit 130 is provided with a control area 452 for receiving selection of voice attributes (for example, the atmosphere given by the voice, the gender of the virtual person who speaks the voice, the voice, etc.). You may. When the user selects any graphic object 453 from the graphic objects matching different voice attributes by the control area 452, the control unit 130 has a voice attribute (voice attribute matching to any of the selected graphic objects 453). For example, the electronic device 10 may be provided with voice data corresponding to (for example, a serious and professional feeling).

Further, the control unit 130 may change the image data output to the first region a based on the selection of the audio attribute. The control unit 130 may provide the first region a with a face image (or an avatar, a character) that matches the attributes of the selected voice as image data. Here, the image data output to the first region a includes at least one of a static image and a dynamic image.

In the present invention, the first area a can be used not only as an output area for providing moving image contents but also as a control command input area. The control unit 130 may control the moving image content based on the user input (for example, touch input) to the first area a.

Here, the user input may be a touch input of a preset method to the first area a. In the present invention, different control instructions may be matched for each touch input of different preset methods. The control unit 130 can perform various controls related to the reproduction of the moving image content based on the touch input method input to the first area a.

As an example, as shown in FIG. 6a (a), the control unit 130 is based on a preset one-way (for example, right to left) drag touch input, as shown in FIG. 6a (b). , The moving image content provided in the first region a may be changed. Based on the drag touch input, the control unit 130 ends the reproduction of the first moving image content 610 that was reproduced in the first region a as shown in FIG. 6a (a), and the reproduction of the first moving image content 610 is completed, and the reproduction of the first moving image content 610 is completed. As shown in the above, the second moving image content 620 may be reproduced in the first region a.

Here, the theme of the second moving image content 620 may or may not be related to the theme of the first moving image content 610.

As an example, as shown in FIG. 6b (a), the control unit 130 selects the theme of the second moving image content 642 based on the preference information input from the user to the first moving image content 641. You may do so. The preference information may be input via the electronic device 10 during the reproduction of the first moving image content 641.

When the preference information that the user is interested in the first moving image content 641 is input, the control unit 130 sets the moving image content related to the theme or content of the first moving image content 641. It may be specified as 2 moving image contents 642.

On the other hand, when the non-preference information that the user is not interested in the first moving image content 641 is input, the control unit 130 has no relation to the theme or the content of the first moving image content 641. The moving image content may be specified as the second moving image content 642.

Further, the touch input of a different preset method to the first region a is i) the playback of the moving image content is stopped, ii) the playback of the previously played moving image content, and iii) the voice that utters the voice data. It may be matched with various control commands for the moving image content, such as changing the type.

On the other hand, when the control unit 130 can receive the input of the control command corresponding to the touch input input from the user by the voice command, as shown in FIG. 6b (b), "the next article" Please say! You may output guide information such as ". Therefore, the present invention can make it easier for the user to control the moving image content.

As described above, in the related information providing method and system according to the present invention, the display area of the display unit is divided into a plurality of areas, the moving image is provided in any one area, and the information provided by the moving image is related in the other area. Information can be provided. Therefore, the user can more intuitively use the information related to the information provided by the moving image at any time while using the moving image. As a result, the user can avoid the complexity of moving to another function to search for related information and searching for the information.

Further, in the related information providing method and system according to the present invention, the information provided by the moving image can be provided as subtitles, and the information regarding the text included in the subtitles can be provided. That is, in the present invention, by visually providing the related information that can be provided to the user by using the subtitles, the user can access the related information more easily.

Further, in the related information providing method and system according to the present invention, it is possible to provide a user environment for switching pages in order to provide related information while the moving image is being reproduced. Therefore, in the present invention, the related information can be provided in the entire display area, and in that case, the GUI for controlling the reproduction of the moving image can be provided. Therefore, the user can receive the related information and can continue to use the sound information provided by the moving image, and can receive the complex provision of various information.

Computer-readable media, on the other hand, include all types of recording devices on which data readable by computer systems is recorded. Examples of computer-readable media include HDDs (Hard Disk Drives), SSDs (Solid State Disks), SDDs (Silicon Disk Drives), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. Be done.

Further, the computer-readable medium may be a server or cloud storage that includes storage and can be accessed by electronic devices by communication. In this case, the computer can download the program according to the present invention from the server or cloud storage by wired or wireless communication.

Further, in the present invention, the computer described above is an electronic device equipped with a processor, that is, a central processing unit (CPU), and the type thereof is not particularly limited.

On the other hand, the detailed description of the present invention is exemplary and should not be construed in a limited way in all respects. The scope of the invention should be defined by a reasonable interpretation of the appended claims, and any modification within the scope of the invention is included in the scope of the invention.

Claims (17)

The step of playing the moving image content including subtitles in the first area of the display unit,
A step of outputting a graphic object including a part of the text constituting the subtitle to the second area of the display unit, and a step of outputting the graphic object.
Including a step of providing relevant information about the portion of the text based on the selection signal to the graphic object.
A method for providing related information, in which, in the subtitle, the text of the part is highlighted and displayed so as to be distinguished from the text of the other part. Further including the step of generating the subtitles
In the step of generating the subtitles,
Search for whether or not there is a part corresponding to a keyword already registered in a specific database in the text constituting the subtitle.
The related information providing method according to claim 1, wherein a portion corresponding to the already registered keyword is highlighted in the text constituting the subtitle. The partial text corresponds to a particular keyword already registered in the particular database.
The related information providing method according to claim 2, wherein in the subtitle, the text of the part is displayed in a color different from the text of the other part. The particular database
The related information providing method according to claim 3, wherein the related information regarding the specific keyword corresponding to the partial text is stored in a match with the specific keyword. A first page including a first area in which the moving image content is reproduced and a second area in which the graphic object is displayed is output to the display unit.
The first page output to the display unit is
The related information providing method according to any one of claims 1 to 4, wherein the second page different from the first page is switched based on the selection signal for the graphic object. When the second page is output to the display unit,
The related information providing method according to claim 5, wherein a GUI object relating to control of the moving image content reproduced in the first area of the first page is output to one area of the second page. The related information providing method according to claim 6, wherein even if the second page is output to the display unit, the moving image content reproduced in the first region is continuously reproduced. The GUI object is
The related information providing method according to claim 7, wherein at least one of the output position and the output size on the second page is changed based on the user input to the GUI object. The second page output to the display unit is
The related information providing method according to claim 8, wherein the user can switch to the first page based on a user input of a preset method to the GUI object. The selection signal for the graphic object includes a first selection signal and a second selection signal.
When the first selection signal for the graphic object is input, the first related information regarding the part of the text is output to the second area.
The related information providing method according to claim 5, wherein when the second selection signal for the graphic object is input, the second page including the second related information regarding the part of the text is output. The related information providing method according to claim 10, wherein the first related information includes a part of the second related information. In the first area, a progress bar indicating the degree of reproduction of the moving image content is displayed.
The related information providing method according to any one of claims 1 to 11, wherein the progress bar includes an indicator indicating a reproduction point at which the highlighted portion of the text is output. Further, a step of controlling the reproduction of the moving image content so that when the selection signal for selecting the indicator is received, the highlighted portion of the text is reproduced from the point included as the subtitle. The related information providing method according to claim 12, which includes. The claim further comprises a step of switching the first moving image content reproduced in the first area to the second moving image content different from the first moving image content based on the user input to the first area. The method for providing related information according to any one of 1 to 13. The related information providing method according to claim 14, wherein the second moving image content reproduced in the first region based on the user input has a theme different from the theme of the first moving image content. Communication department and
A display unit including a display area divided into a first area and a second area,
The first area includes a control unit that controls the display unit so as to play the moving image content including the subtitles and output the graphic object including a part of the text constituting the subtitles to the second area.
The control unit
The display unit is controlled so as to output related information regarding the part of the text based on the selection signal for the graphic object.
A related information providing system in which, in the subtitle, the text of the part is highlighted and displayed so as to be distinguished from the text of the other part. A program that is executed by one or more processes in an electronic device and can be stored in a computer-readable medium.
The program
A step of playing moving image content including subtitles in the first area of the display unit, a step of outputting a graphic object including a part of text constituting the subtitle to the second area of the display unit, and a selection signal for the graphic object. Contains a command to perform a step that provides relevant information about the portion of the text based on.
In the subtitle, the part of the text is highlighted and displayed so as to be distinguished from the text of the other part, and is a program that can be stored in a computer-readable medium.

Images