KR20210022360A - A method and apparatus for automatically converting web content to video content - Google Patents

Abstract

An embodiment of the present invention discloses a method and device for automatically converting web content on the Internet, such as a news web page, consisting of images and texts, into video content for easy use by a user. The disclosed device comprises: a web page search module fetching a web page to be converted through the Internet; a video component selection module selecting a component to be converted into a video by analyzing a DOM model of a filtered target web page component; a subtitle conversion module converting text into subtitles among the selected components; a timeline configuration module automatically composing a video timeline with images and subtitles of selected components; a timeline editing module automatically editing the composed video timeline according to subtitles or user needs; and a video creation module creating a video of the edited components of the timeline. According to the present embodiment, since a process of configuring and arranging the timeline, which is the most difficult part in producing a video, is automatically performed, content on the web can be conveniently and easily produced as the video.

Description

A method and apparatus for automatically converting web content to video content

The present invention relates to a web content conversion technology, and more particularly, to a method and apparatus for automatically converting web content on the Internet consisting of images and texts, such as news web pages, into video content for easy use by users.

In general, numerous multimedia web contents are provided on the World Wide Web (WWW), and web contents are used by various mobile devices such as smart phones and tablet PCs as well as laptops and personal computers through web surfing.

In general, web content may be implemented in a multimedia manner such as text, image, and video. As the use of smart phones increases, content in the form of video has been widely used in recent years.

Therefore, in the case of web content mainly composed of text and images, such as news web pages of newspapers, it is necessary to convert and provide video content so that users can view it in the form of a video.

However, in order to provide web content consisting of text and images as video content, the content provider directly inputs an image or text on the Internet into a video production tool, produces it in the form of video content, and provides it, which is cumbersome and time-consuming. There is a problem with this.

'Apparatus and method for providing a video for smart TV using still images' announced in the Korean Intellectual Property Office as registration number 10-1607084 is an instruction to edit the user's video through the video editing tool unit after the server receives the user's still image. According to this, a video for a smart TV is generated and provided so that it can be played on a smart TV.

In addition, the'content creation service device that generates a video based on personal content, a method of generating a video based on personal content' disclosed in Korean Patent Application Publication No. 10-2015-0093263 is from a pre-stored template and a user device. It is a technology that creates customized videos tailored to users by combining transmitted personal contents.

In addition, the'web-based video generation apparatus and method' announced in the registered patent publication as registration number 10-1498233 is a web-based personalized video that can express the user's intentions or emotions using user location-based map information and user-related information. It is a technology that supports to be created.

It is an object of the present invention to provide a method and apparatus for automatically analyzing web contents made of images and texts on the Internet, such as news web pages, and automatically converting them into moving picture contents so that users can easily use them.

Another object of the present invention is to provide a technology for automatically converting web content on the Internet into video content in a software as a service (SaaS) method.

Another object of the present invention is to provide a method of automatically configuring an image editing timeline by automatically analyzing a region of interest (ROI) of a web page.

Another object of the present invention is to provide a method of automatically configuring a caption by calculating the length of a text and the length of a caption that can be exposed on a screen.

Another object of the present invention is to provide a method of automatically converting text collected using TTS (Text-to-Speech) into voice and including it in an image, and automatically matching the synchronization between the voice generated in TTS and the subtitle. will be.

The present embodiment discloses an apparatus for automatically analyzing web content made of an image and text on the Internet, such as a news web page, and automatically converting it into video content.

The disclosed device includes a web page navigation module that retrieves a web page to be converted through the Internet, and a video component selection that selects a component to be converted into a video by analyzing a document object model (DOM) model of the filtered target web page component. The module, a subtitle conversion module that converts text from among the selected components into subtitles, a timeline configuration module that automatically configures a video timeline with images and subtitles of the selected components, and It includes a timeline editing module that automatically edits according to a request, and a video generation module that generates a moving picture of the timeline-edited components.

The apparatus may further include an advertisement filtering module for filtering an advertisement area on the searched web page or a TTS module for converting subtitles into voice.

Another embodiment of the present invention discloses a method of automatically analyzing web content on the Internet consisting of images and text, such as a news web page, and automatically converting it into video content.

The disclosed method includes the steps of searching for a web page to be converted to a video, selecting a video component by analyzing the searched web page, constructing a video timeline with the selected components, and constructing a configured video timeline. And editing, and generating a video according to the edited video timeline.

The device or method of the above embodiment may be provided to a content provider or a content user in a software as a service (SaaS) method.

According to an embodiment of the present invention, as the process of configuring and arranging a timeline, which is the most difficult part in producing a video, is automatically performed, content on the web can be produced simply and easily into a video.

In addition, according to an embodiment of the present invention, it is possible to automatically configure a timeline centering on a region of interest with a high preference by managing the preference of the region of interest for each web page, and the length of the text and the subtitles that can be exposed on the screen. It is easy to use as it automatically composes subtitles by calculating the length.

In addition, according to an embodiment of the present invention, when the service of the present invention is used in a media field such as a news agency, media content distributed on the web can be easily converted and distributed, and when used by an individual, the self-published post , Photos, images can be converted into videos in an easy way to be owned or distributed.

1 is a schematic diagram of a service system according to an embodiment of the present invention;
2 is a block diagram showing the configuration of an automatic content conversion service server according to an embodiment of the present invention;
3 is a flow chart showing a procedure for automatically converting a web page for content distribution by a content provider according to an embodiment of the present invention;
4 is a flow chart showing a procedure for automatically converting and using a web page by a content user according to an embodiment of the present invention;
5 is a flow chart showing a content conversion procedure according to an embodiment of the present invention;
6 is a flow chart showing a procedure for automatically performing the step of selecting a video component in FIG. 5;
FIG. 7 is a flow chart showing a procedure for collecting user experience of an ROI while manually performing the step of selecting a video component in FIG. 5;
FIG. 8 is a flow chart showing a procedure for automatically performing the step of selecting a video component in FIG. 5 using a preference of an ROI;
9 is a flow chart showing a procedure of automatically performing the step of configuring a video timeline in FIG. 5 through caption generation;
10 is a flow chart showing a procedure for automatically performing the step of configuring a video timeline in FIG. 5 using TTS;
11 is a flow chart showing a procedure for automatically configuring a video timeline using a text file in another embodiment of the present invention;
12 is an example of a video timeline configuration screen for video editing in an embodiment of the present invention;
13 is an example of a text editing screen in an embodiment of the present invention;
14 is an example of a background image editing screen in an embodiment of the present invention;
15 is an example of a scene change effect setting screen in an embodiment of the present invention.

The present invention and the technical problems achieved by the implementation of the present invention will become more apparent by the preferred embodiments of the present invention described below. The following examples are merely illustrative for explaining the present invention, and are not intended to limit the scope of the present invention.

1 is a schematic diagram of a service system according to an embodiment of the present invention, and FIG. 2 is a block diagram of a configuration of an automatic content conversion service server according to an embodiment of the present invention. 3 is a flow chart showing a procedure for a content provider to automatically convert a web page for content distribution according to an embodiment of the present invention, and FIG. 4 is a content user automatically converts a web page according to an embodiment of the present invention. It is a flow chart showing the procedure to be used.

The service system 100 according to an embodiment of the present invention distributes content through the service server 110 and the Internet 102 providing a content conversion service according to an embodiment of the present invention, as shown in FIG. 1. It is composed of a content provider (CP) 120 corresponding to an individual or a company, and content user terminals 130 that use the video content converted according to the service of the present invention through the Internet 102. Clients 140-1 and 140-2 for a service according to the present invention may be executed in the content provider 120 or the content user terminals 130.

As shown in FIG. 2, the service server 110 includes a web page search module 111, an advertisement filtering module 112, a component selection module 113, an ROI management module 114, and a caption generation module 115. , A TTS module 116, a timeline configuration module 117, a timeline editing module 118, and a video generation module 119.

Referring to FIG. 2, the web page search module 111 retrieves a web page to be converted through the Internet 102 through web crawling, and the advertisement filtering module 112 is a form of advertisement commonly provided on the web. By registering patterns in advance, elements such as advertisement images are automatically filtered so that they are not included in the timeline.

The component selection module 113 analyzes the DOM model of the filtered target web page component and selects a component to be converted into a video. In this case, the component selection module 113 may analyze the DOM model of the target web page component or select a component to be converted into a video according to a preference of a region of interest (ROI) previously obtained from the conversion information.

The ROI management module 114 stores and manages preference information of a region of interest (ROI) for web pages to be converted from past content conversion information, and the caption generation module 115 is a selected component. Among them, text is converted into subtitles, and the text-to-speech (TTS) module 116 converts the subtitles into speech. Here, the ROI is an image or text selected as an element to be converted from among images or texts on a web page to be converted, and may be leveled as a preference by accumulation of user experience. That is, the'region of interest (ROI) preference' is the frequency of users' selection for the corresponding region of interest, and the frequently selected region of interest has a high preference.

The timeline configuration module 117 configures a video timeline with images or subtitles or TTS of the selected component. Here, the subtitle is edited to display the text of the selected component on the video content, and the TTS is converted from the subtitle to audio. The timeline configuration module 117 may automatically configure a timeline using data 104 designated by the user, such as a text file.

The timeline editing module 118 automatically edits the video timeline configured by the timeline configuration module 117 according to the length of a subtitle or TTS or a user's request, and the video generation module 119 The components are created as video content.

Referring to FIG. 1, a content provider (CP) 120 may be composed of a corporate site that provides commercial web pages such as news web pages, and an individual who provides personal web pages through SNS. As shown in Fig. 3, after converting one's web page into video content using the service of the present invention in a SaaS method, it can be distributed through a web site, portal, or SNS of the content provider 120.

Referring to FIG. 3, the content provider 120 registers with the service server 110 and uses the service of this embodiment in a SaaS method (S11, S12). When a content conversion is requested by using the service of the present invention, the service server 110 automatically converts the web page requested by the content provider 120 into video content according to a predetermined procedure and delivers it to the content provider 120, thereby Accordingly, the content provider 120 posts the converted video content (S13 to S15). Thereafter, the content user 110 may use the converted content distributed by the content provider 120.

Referring back to FIG. 1, the content user 110 uses video content that is converted and provided by the content provider 120, as shown in FIG. 3, or provided by the content provider 120, as shown in FIG. The web page to be converted into video content through the service server 110 may be used.

4, the content provider 120 is posting its own web page (S21). The service server 110 manages the preference of a region of interest (ROI) for each web page as a database from past conversion information for each web page (S22). When the content user 130 wants to view the web page distributed by the content provider 120 using the service of the present invention, the content user 130 requests the service server 110 to convert the web page (S23). . The service server 110 automatically converts the web page to video content by referring to the preference of the previous ROI for the web page, and then provides the converted video content to the content user 130 ( S24~S26). Accordingly, the content user 130 may use the converted video content (S27).

5 is a flowchart illustrating a content conversion procedure according to an embodiment of the present invention.

As shown in FIG. 5, the procedure of converting a web page to video content according to an embodiment of the present invention includes a step of searching a web page (S1) and a step of selecting a video component within the web page (S2). , Comprising a step of constructing a video timeline (S3), a step of editing the video timeline (S4), and a step of generating a video (S5).

Referring to FIG. 5, the step of searching for a web page (S1) is a step of searching for a web page for which a service user wants to be converted, and obtaining web page data. In general, a web page is an individual document on the world wide web, and unlike a page in a book, two or more web pages may be linked to each other by hyperlinks. Most web pages are stored on a web server and consist of videos, text and images such as HTML (or XHTML), CSS, JavaScript, pictures, and Flash. These web pages can be sent via HTTP, and can be sent via a web browser. Web pages can be read through. These web pages are expressed as a structured document of the Document Object Model (DOM). DOM is the W3C's official standard for expressing a structured document that is platform/language neutral, and web pages are tree-structured using DOM. Can be obtained.

In the step (S2) of selecting a video component within a web page, as shown in FIG. 6, an automatic selection method, a manual selection method as shown in FIG. 7, and as shown in FIG. 8 It can be classified in a manner that is automatically selected according to the preference of the region of interest (ROI).

6 is a flowchart illustrating a procedure of automatically performing the step of selecting a video component in FIG. 5.

Referring to FIG. 6, in the process of first searching for a web page and then checking the components of the searched web page, the DOM model of the component is analyzed, and it is determined whether the component meets a preset condition, and the composition meets the set condition. If it is an element, the component is added to the'selection component' list to complete automatic selection of the video component (S101 to S105).

FIG. 7 is a flowchart illustrating a procedure of collecting user experience of a region of interest (ROI) while manually performing the step of selecting a video component in FIG. 5.

Referring to FIG. 7, when a user manually selects a video component from a searched web page, the clients 140-1 and 140-2 analyze the DOM property of the selected component and transmit the DOM property to the service server 110. In addition to transmitting the and page address, a list of the selected components is generated to complete the manual selection of the video component (S201 to S206).

The service server 110 stores the web page address received from the clients 140-1 and 140-2 and the DOM properties of the components in a database, and uses the cumulative experience of the components with respect to the page address to create a region of interest (ROI). After calculating the preference of, ROI preference data for the site (web page) is generated (S207 to S209). Here, the ROI preference data is data obtained by quantifying (or leveling) the frequency of users' selection for the ROI.

FIG. 8 is a flowchart illustrating a procedure for automatically performing the step of selecting a video component in FIG. 5 using a preference of a region of interest (ROI).

Referring to FIG. 8, after searching for a web page, the clients 140-1 and 140-2 request the service server 110 to search for a video component (S301 and S302 ).

When a video component search is requested, the service server 110 inquires the ROI preference data for the site, and then returns the DOM component details for the ROI in the page to the client (140-1, 140-2). Do (S303, S304).

Accordingly, the clients 140-1 and 140-2 add the automatically selected component to the list of selected components if there is an automatically selected component based on the ROI preference, and if the automatically selected component does not exist according to the ROI preference, , After manually selecting a video component, a list of selected components is generated to complete automatic and manual selection of the video component (S301 to S309).

As described above, in an embodiment of the present invention, a user interface is provided that allows a user to easily import a video material (elements such as images and text) to be drawn from content on the web, and a DOM model of the video material selected from the user interface. After analyzing and collecting the ROI of users according to the configuration of each site, the ROI preference of each site is converted into data and stored in the service server 110. In addition, as the number of users increases, weights are assigned to objects used by users to increase accuracy, and components that are meaningful to users can be presented with a high probability, and for known services used by many users, preference of ROI is set in advance. Thus, image elements can be automatically selected.

In addition, if the end user is a company other than an individual, if the end user is using their web content for the purpose of converting their web content into a video, it is possible to use preset ROI preference data to configure the video component by analyzing the web site in advance.

As described above, according to an embodiment of the present invention, even if a user has no experience using the web page, a timeline can be automatically configured using a preset ROI preference and an ROI preference based on accumulated data according to the user experience of other users. have. In this case, a user interface listing all components not selected by the ROI may be provided to allow the user to directly select the components.

Referring back to FIG. 5, in the video timeline construction step (S3), after selecting a video component in any one of FIGS. 6 to 8, an image and a subtitle are displayed based on the selected component as shown in FIG. 9. Create a timeline.

Referring to FIG. 9, when the selection of the video component is completed, an image order is arranged among the selected components, and a list is created by dividing the text of the web page by paragraph (S401 to S403). Next, the screen resolution and subtitle display option settings are loaded, and the subtitle display speed (exposure speed per character) setting is loaded, and then subtitle data for each paragraph is generated (S404, S405).

In the process of creating caption data for each paragraph in the video timeline configuration step (S3), either: ① Automatically create a timeline for video editing through caption generation, or ② automatically generate a timeline through TTS.

In the case of automatically generating a timeline through subtitle generation, as shown in FIG. 9, it is determined whether each paragraph can be displayed on one screen to generate a subtitle and set a time (S406, S407).

Then, after determining whether there is a sentence separator and dividing it into sentence separators or time division by word unit to divide subtitles, the time of each subtitle is set by setting, added to the subtitle list, and then the timeline of the image and subtitle Generates (S408 to S413).

As described above, in the embodiment of the present invention, text is automatically cut into a caption having a length that can be viewed on one screen according to a preset value, and the caption is calculated to have an easy-to-read exposure time through the length of the caption. It is exposed for as long as time.

Meanwhile, in the case of automatic timeline configuration through TTS, as shown in FIG. 10, timelines of images, captions, and audio are generated using TTS.

Referring to FIG. 10, when the selection of the video component is completed, the image order is arranged among the selected components, and a list is created by dividing the text of the web page by paragraph (S501 to S503). Next, the screen resolution and subtitle display option settings are loaded, and the subtitle display speed (exposure speed per character) setting is loaded, and then subtitle data for each paragraph is generated (S501 to S505).

Subsequently, it is determined whether each paragraph can be displayed on one screen, a subtitle is generated, and a time is set (S506, S507), and then the existence of a sentence separator is determined, and if there is a sentence separator, it is divided into a sentence separator. Then, after dividing the subtitles by time division by word unit, the divided subtitles are converted to voice using TTS, the subtitle time is set based on the converted voice, and after adding it to the subtitle list, the timeline of the image, subtitle and voice Generates (S508 to S514). At this time, the audio and subtitles need to be synchronized.

As described above, in an embodiment of the present invention, when subtitles are converted to voice using TTS (Text To Speech) and included in the video, the TTS converts the subtitle into voice as much as the length to be exposed on one screen, and analyzes the time length of the subtitle. You will determine the exposure time.

11 is a flowchart illustrating a procedure for automatically configuring a timeline for image editing using user-specified data such as a text file, according to another embodiment of the present invention.

Another embodiment of the present invention is an embodiment of automatically converting user-specified content (eg, text file) other than a web page into an image. That is, another embodiment of the present invention is to produce a video by configuring a timeline by selecting a video component from a source such as PDF, PowerPoint, and text, as well as a method of converting a video from a web page.

Referring to FIG. 11, in the case of automatically configuring a timeline through a text file, a list is created by dividing the text file by paragraph (S601, S602). Next, the screen resolution and subtitle display option settings are loaded, and the subtitle display speed (exposure speed per character) setting is loaded, and then subtitle data for each paragraph is generated (S603, S604).

While generating subtitle data for each paragraph, it is determined whether each paragraph can be displayed on one screen, and if it can be displayed on one screen, a subtitle is generated and a time is set (S605 and S606).

If the length of a paragraph exceeds one screen, it is necessary to divide the paragraph. If there is a sentence separator, it is determined whether there is a sentence separator, and if there is a sentence separator, the subtitle is divided by time division by word units (S607~ S609). Subsequently, the divided subtitles are converted into audio using TTS, added to the subtitle list, and then a timeline of images and subtitles is generated (S610 to S612).

As described above, in another embodiment of the present invention, a function of inserting other data (text or image) that the user has or inputting a caption may also be provided.

Referring back to FIG. 5, in the video timeline editing step (S4), as illustrated in FIGS. 12 to 15, text or video may be edited on a timeline configuration screen and a scene change effect may be set.

12 is an example of a screen for editing a timeline configuration in an embodiment of the present invention.

Referring to FIG. 12, the editing screen of the timeline configuration for creating a video includes a playback area M6 for previewing an edited image, a timeline area M5 of components, a text editing area M1, and an image editing area. It includes (M2), transition effect setting area (M3), and encoding start button (M4). A timeline for text, background image, audio, and logo is displayed in the timeline area M5.

13 is an example of a screen for editing text in an embodiment of the present invention.

Referring to FIG. 13, there are items for setting the font, margin, position, size, etc. of the text on the text editing screen, and the text being edited is displayed.

14 is an example of a screen for editing an image in an embodiment of the present invention.

Referring to FIG. 14, an extracted image, an added image, a loaded image, etc. are registered as a background image on the image editing screen, and a desired image may be selected from among them and used as a video component.

15 is an example of a screen for setting a scene change effect in an embodiment of the present invention.

Referring to Figure 15, the scene change effect is no effect, disappear, push right, push left, push down, push up, cover left, cover right, cover up, cover down, etc. It can be set in one way.

Referring back to FIG. 5, after editing the timeline configuration on the editing screen, if the encoding start button M4 is pressed, video content according to the editing is generated (S5). As described above, in the embodiment of the present invention, when the video is produced, the video may be produced as it is according to a pre-configured timeline, or the video may be produced by reconfiguring through timeline editing.

In the above, the present invention has been described with reference to an embodiment shown in the drawings, but those of ordinary skill in the art will understand that various modifications and other equivalent embodiments are possible therefrom.

102: Internet 104: Client designated data
110: service server 111: web page navigation module
112: advertisement filtering module 113: component selection module
114: ROI management module 115: caption generation module
116: TTS module 117: Timeline configuration module
118: timeline editing module 119: video generation module
120: content provider 130: content user
140-1, 140-2: client

Claims (5)

A web page search module for fetching a web page to be converted through the Internet;
A video component selection module that analyzes a DOM model of the filtered target web page component and selects an component to be converted into a video;
A caption conversion module for converting text from among the selected components into captions;
A timeline configuration module for automatically configuring a video timeline with images and subtitles of the selected component;
A timeline editing module for automatically editing the configured video timeline according to subtitles or a user's request; And
A device that automatically converts web content into a video, including a video generation module that generates timeline-edited components into video. The method of claim 1, wherein the device,
A device for automatically converting web content to video, further comprising an advertisement filtering module for filtering an advertisement area in the searched web page. The method of claim 1, wherein the device,
A device that automatically converts web content to video, which further includes a TTS module that converts subtitles to audio. Searching for a web page to be converted into a video;
Analyzing the searched web page and selecting a video component;
Configuring a video timeline with the selected components;
Editing the configured video timeline; And
A method of automatically converting web content to video, comprising generating a video according to the edited video timeline. A service method for automatically converting web content to video, characterized in that the device of claim 1 or the method of claim 4 are provided to content providers or content users in a SaaS (Software as a Service) method.

Images