KR20200048772A - Method for providing video and apparatus using the same - Google Patents

Abstract

Provided are a method for providing an image and a device for providing an image using the same according to one embodiment of the present invention. The method for providing an image comprises the steps of: extracting a partial area of the image; recognizing and detecting a character displayed on the extracted partial area; determining whether the detected character corresponds to a condition; determining a time when the character corresponds to the condition; generating an overlay that the determined time is displayed on timeline; and providing the image and the generated overlay to a viewer device.

Description

METHOD FOR PROVIDING VIDEO AND APPARATUS USING THE SAME

The present invention relates to an image providing method and an image providing apparatus using the same.

There are various types of information that can be provided to viewers in the Internet broadcasting replay service. Viewers who watch the Internet broadcast can watch the desired broadcast again with the viewing service even after the real-time broadcast ends.

If the viewer has not been able to watch the Internet broadcast in real time, watch the broadcast video from the beginning until the desired scene comes out to see a specific scene in the desired broadcast, or if the viewer has watched the live broadcast, remember the time zone of the desired scene The placing method is not an efficient method.

In order to solve this difficulty, a video providing method has been proposed to provide a more convenient method for watching broadcast video to Internet broadcast viewers.

[Advanced technical literature]

Patent Publication No. 10-0764175 Apparatus and method for detecting important subtitles of video for customized broadcasting service

After the Internet broadcast is over, it is difficult to accurately and quickly find the viewpoint and the section desired by the viewer in the broadcast video provided.

After the real-time broadcast is over, the broadcast video provided generally does not have a short broadcast time, and if the viewer has not watched the real-time broadcast, the viewer wants to know whether the desired scene is from the broadcast or if it is, where the point in time is. Can not. In addition, even when watching a real-time broadcast, it is difficult to find the desired section unless the point of view of the desired section is memorized or memorized.

The inventors of the present invention have recognized that there is such discomfort in finding the section desired by the viewer in the video. Accordingly, the inventors of the present invention have come to invent an image providing method and an image providing apparatus using the same, which can easily find a viewpoint and a section with high viewer interest, thereby providing convenience for viewing the image.

Accordingly, a problem to be solved by the present invention is to provide an image providing method and an image providing apparatus using the same so that the viewer can find a section in the broadcast image that causes the viewer's interest and interest after the real-time broadcasting ends.

Furthermore, another problem to be solved by the present invention is to provide an image providing method and an image providing apparatus using the image so that the viewer may be provided with additional information on a section and broadcasting content desired to be viewed in the image.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above-described problems, an image providing method according to an embodiment of the present invention is provided. The image providing method includes extracting a partial region of the image, recognizing and detecting a character displayed in the extracted partial region, determining whether the detected character corresponds to a condition, and a viewpoint when the character corresponds to the condition. Determining, generating an overlay in which the determined time point is displayed on the timeline, and providing the viewer device with an image and the generated overlay.

According to another feature of the present invention, when a character falls under a condition, it includes a case where the character matches a predetermined keyword.

According to another feature of the present invention, the method includes: extracting an additional region different from a partial region, recognizing and detecting additional characters displayed in the additional region, and determining whether the detected additional characters correspond to conditions And determining an additional point in time when the additional character corresponds to the condition, and generating the overlay includes generating an overlay in which the starting point and the added point are displayed in the timeline.

According to another feature of the present invention, the case where a character is a condition includes a case where the character fluctuates faster than a predetermined time.

According to another feature of the present invention, when a viewpoint is determined, the overlay displays a viewpoint that is earlier than a predetermined time from the determined viewpoint as an actual viewpoint.

According to another feature of the present invention, the letter is a number, and the step of generating an overlay includes the step of generating a graph displaying the number in time series as an overlay.

According to another feature of the present invention, the step of providing the viewer device with an image and the generated overlay is to selectively provide at least one overlay type to the viewer device, when a selection for the overlay type is received, the selected overlay The method further includes generating an additional overlay based on data associated with the type, and providing the additional overlay with an image.

According to another feature of the invention, the additional overlay is displayed overlapping the overlay.

According to another feature of the present invention, the overlay type is a graph type of at least one of the number of users, response, broadcast contribution, and the like.

According to another feature of the present invention, the overlay type is an overlay type in which the time when the emotional voice of the broadcast creator is identified is displayed on the timeline, and the identification of the emotional voice of the broadcast creator is a sound prediction configured to predict when the emotional voice comes out. It is performed by predicting the point in time when there is an emotional voice by inputting the audio of the image to the model.

According to another feature of the present invention, the method further includes determining a region of interest through analysis of the image, before extracting a portion of the image.

In order to solve the problems as described above, the image providing apparatus according to an embodiment of the present invention includes a communication unit configured to receive an image and a processor operatively connected to the communication unit, and the processor extracts and extracts a partial region of the image After recognizing and displaying the characters displayed in some areas, it is determined whether the detected characters are in a condition, determining when the characters are in the condition, and generating an overlay in which the determined points are displayed in the timeline. And to provide the viewer device with an image and the generated overlay.

The present invention can provide an image providing method and an apparatus using the same so that when the viewer fails to watch the real-time broadcast video, the viewer can watch the broadcast as a replay service capable of viewing the already finished broadcast.

Furthermore, the video providing apparatus according to an embodiment of the present invention may provide information on a section of a viewer's interest and interest in a real-time broadcast and a highlight section of the broadcast video together with the video provided by the viewing service. In addition, the present invention can provide an image providing method, an image providing method using the same, and an apparatus using the same, providing a section of an image by predicting a time point that a viewer wants to watch and finding a previous time point from the corresponding time point.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is a block diagram illustrating a system including an image providing apparatus according to an embodiment of the present invention.
2A to 2B are schematic block diagrams illustrating an image providing apparatus according to an embodiment of the present invention.
3 is a schematic flowchart illustrating an image providing method according to an embodiment of the present invention.
4A and 4B are schematic diagrams for describing an image providing method according to an embodiment of the present invention.
5A and 5B are schematic diagrams for describing an image providing method according to another embodiment of the present invention.
6A to 6D are schematic diagrams for describing an image providing method according to another embodiment of the present invention.
7A to 7D are schematic diagrams for describing an image providing method according to various embodiments of the present invention.
8A and 8B are schematic diagrams for describing an image providing method according to various embodiments of the present invention.
9 is a schematic diagram illustrating a method of providing an image according to various embodiments of the present invention.
10A to 10C are schematic diagrams for describing an image providing method according to various embodiments of the present invention.

Advantages and features of the present invention, and methods for achieving them will be clarified with reference to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person having the scope of the invention, and the present invention is only defined by the scope of the claims.

Although the first, second, etc. are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are only used to distinguish one component from another component. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical spirit of the present invention.

The same reference numerals refer to the same components throughout the specification.

Each of the features of the various embodiments of the present invention may be partially or totally combined or combined with each other, and technically various interlocking and driving may be possible as those skilled in the art can fully understand, and each of the embodiments may be implemented independently of each other. It can also be implemented together in an associative relationship.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a system including an image providing apparatus according to an embodiment of the present invention. The video providing system 1000 includes a video providing apparatus 100, a viewer device 200, and a broadcast creator device 300. In the present specification, the viewer device 200 and the broadcast creator device 300 are not limited, and may include, for example, a general-purpose computer, notebook, mobile device (eg, smartphone, tablet device), and the like. The broadcast creator device 300 transmits the real-time internet broadcast of the broadcast creator to the image providing apparatus 100. In addition, the broadcast image recorded from the moment the broadcast creator device 300 starts broadcasting to the moment when the broadcast ends is stored in the image providing apparatus 100. Accordingly, the image providing apparatus 100 provides a replay service function for viewing the recorded broadcast image with the viewer device 200 even after the broadcast ends. At this time, the video providing apparatus 100 may provide data of the sections having high interest and interest of viewers who watched the broadcast in real time from the broadcast video and highlight sections of the broadcast video together to the viewing service.

The sections of high interest and interest of viewers and the highlight section of the broadcast video may be displayed as a timeline overlay on the broadcast video, and the overlay may be displayed in one or more various types. In addition, the viewer device 200 may be provided with information on the flow of broadcast content previously identified in real-time broadcasting even without viewing the broadcast image. For example, the character of the number of survivors is recognized and detected on the game screen of the broadcast creator performing the survival game with the broadcast content, and the content related to the data can be displayed as a graph type overlay on the broadcast image. Accordingly, the viewer device 200 may be provided with a section in which a viewer's interest and interest are high in a broadcast video and a highlight section of the broadcast video with various information provided by the video providing apparatus 100.

2A is a schematic block diagram illustrating an image providing apparatus 100 according to an embodiment of the present invention.

Referring to FIG. 2A, an image providing apparatus 100 according to an embodiment of the present invention includes a communication unit 110, an input unit 120, a storage unit 130, and a processor 140.

The video providing apparatus 100 serves to communicate the viewer device 200 and the broadcast creator device 300 with each other, thereby enabling practical data communication between the viewer device 200 and the broadcast creator device 300. . The video providing apparatus 100 according to an embodiment of the present invention allows viewers who have not watched the Internet broadcast in real time to watch the broadcast video even after the broadcast ends, and the sections in which the broadcast video is expected to be of interest to viewers Alternatively, various data may be provided together with a viewpoint.

The communication unit 110 enables the image providing apparatus 100 and the broadcast creator device 300 and the viewer device 200 to communicate.

The input unit 120 may be configured to receive various inputs necessary for control of the image providing device 100. The input unit 120 may be configured of a mouse, a keyboard, a key pad, a dome switch, a touch pad (static pressure / power outage), a jog wheel, a jog switch, or the like.

The storage unit 130 stores a broadcast image for re-viewing and overlay information associated with it. Here, the broadcast video is a game screen as a background screen, and a broadcast creator's face appears on a PIP (Picture in Picture) screen in some areas, or a sports game screen is a background screen, and the broadcast creator broadcasts with voice, etc. This is not limited to this, and may be a variety of videos, such as a video that only illuminates a broadcast creator. In addition, the overlay is a timeline on graphs and broadcast images in one or more various types, and may be superimposed on a broadcast image to be translucent. Also, the overlay can be placed close to the time search area. In addition, the overlay information may include a section having a high interest and interest of viewers and a highlight section of a broadcast video, and includes all information that can constitute an overlay such as time information and a degree of interest of the section having a high interest.

The processor 140 is operably connected to the communication unit 110, the input unit 120, and the storage unit 130, and performs an operation in which the image providing apparatus 100 provides an image to the viewer. Hereinafter, an operation of providing a broadcast image will be briefly described.

The processor 140 determines a section that caused viewers' interest and concentration in the broadcast video, flow data of broadcast content, and a highlight section of the broadcast video. The section that caused interest and concentration, the flow data of the broadcast content, and the highlight section of the broadcast video, for example, when playing a survival game from the broadcast video to the broadcast content, the section where the number of kills increases within a short time becomes the highlight section of the broadcast video The number of survivors can be displayed on the timeline of the video as a graph overlay to display the flow date of the broadcast content.

The processor 140 extracts some regions in order to obtain data on a section that caused viewers' interest and concentration while broadcasting is being performed, and a highlight section of the broadcast video, and the degree of response received by the viewer device 200 Extract data. Here, the partial area may be an area in which a broadcast creator's face is displayed in a broadcast video, or an area in which important data in a game for displaying a highlight section of the broadcast video is displayed. For example, a part in which the number of survivors is displayed on the game broadcast screen may be extracted as a partial area. In various embodiments of the present invention, some areas may be a fixed part of the screen, but are not limited thereto, and may be changed to other parts of the screen according to time.

The response data may be data accumulated as many as the selected number of viewers by selecting a specific icon that can be clicked at a time that caused viewers' interest and concentration in a broadcast screen that can be viewed by the viewer device 200 display unit 230.

The processor 140 applies a character detected in the extracted region to the condition and, if the condition is met, predicts a broadcast time to determine the corresponding section. For example, when a character detected in the extracted region matches a predetermined keyword, the processor 140 determines a corresponding point in time of the broadcast image, and generates an overlay to be displayed at the point in time. In addition, the processor 140 may generate an overlay in which the corresponding viewpoint is displayed by extracting the voice from the image and inputting it into the sound prediction model to predict the point at which the emotional voice comes out.

2B is a schematic block diagram illustrating a viewer device 200 according to an embodiment of the present invention.

Referring to FIG. 2B, the viewer device 200 includes a communication unit 210, an input unit 220, a display unit 230, a storage unit 240, and a processor 250.

The communication unit 210 connects the video providing apparatus 100 and the viewer device 200 to be communicatively and receives data transmitted from the video providing apparatus 100 to the viewer device 200. The data received by the viewer device 200 may include broadcast video recorded by the broadcast creator device 300, chat data from the broadcast video, and overlay data generated and received by the video providing apparatus 100.

The input unit 220 receives a command in an operation received from the viewer device 200 and collects response data during broadcast. For example, the viewer response data may be input with a mouse click on a button for the response data during broadcast, and the number of clicks is transmitted to the processor 250 as viewer response data.

The display 230 operates to display the broadcast image on the screen of the user device 200, and the overlay generated by the processor 250 may be provided as the broadcast image screen of the user device 200.

The display unit 230 displays information processed by the device. For example, the device displays a user interface (UI) or a graphical user interface (GUI) associated with the system. The display 230 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible display) ), At least one of a 3D display.

The storage unit 240 stores images and data transmitted from the image providing device 100. The storage unit 240 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (for example, SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EPMROM), Programmable Read-Only Memory (PROM), magnetic memory, It may include a storage medium of at least one of a magnetic disk and an optical disk. The device may operate in connection with a web storage that performs a storage function of the storage unit 240 on the Internet.

The processor 250 is operably connected to the communication unit 210, the input unit 220, the display unit 230, and the storage unit 240, and an overlay for displaying viewer response data input from the viewer device 200 on a broadcast image Is produced by

The processor 250 may generate an overlay for displaying the received series of response data on an image, and display the result of collecting the response data at the number of clicks for each time point as an overlay to provide the viewer device 200.

According to a hardware implementation, embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions. The described embodiments can be implemented by the control unit itself including the processor 250.

According to a software implementation, embodiments such as procedures and functions described herein may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented in a software application written in an appropriate programming language. The software code may be stored in the storage unit 240 and executed by the processor 250.

Hereinafter, with reference to FIG. 3, in a video providing method according to an embodiment of the present invention, a section that caused viewers' interest and concentration while broadcasting is being performed, a highlight section of a broadcast video, and viewer response data are provided. How to do this will be described in detail. 3 is a schematic flowchart illustrating an image providing method according to an embodiment of the present invention, and FIGS. 4A to 4B are schematic diagrams illustrating an image providing method according to an embodiment of the present invention. For convenience of description, description will be given with reference to components of FIGS. 2A to 2B.

First, the processor 140 receives the recorded real-time broadcast image from the broadcast creator device 300 through the communication unit 100 (S300). Referring to FIG. 4A, an image screen 430 is displayed on the display unit 230. The video screen 430 displays a video of a broadcast creator in a PIP manner against a game screen. In addition, the video screen 430 includes a chat area. Meanwhile, main data related to the game may be shown on the game screen. For example, in the case of the game implemented in FIG. 4A, the number of surviving players in the game, the number of kills obtained by the player, and the nickname of the player who killed the kill are displayed. The video screen 430 may include a broadcast control area 410. The broadcast control area 410 is an area for performing screen control such as playback, pause, voice control, screen maximization, and playback control. In various embodiments, a selection button may be further included to deliver a response, and a selection button for displaying an overlay in an embodiment of the present invention may be further included. When the selection button for overlay display is selected, a plurality of overlay types can be displayed to be selectable. The display of a plurality of overlay types will be described later with reference to FIGS. 8A, 8B, and 9.

Some regions are extracted from the received image (S310). Some areas may include characters that need to be detected in order to discriminate the section that caused viewers' interest and concentration in the broadcast video, the flow data of the broadcast content, and the highlight section of the broadcast video. For example, in the case of the game implemented in FIGS. 4A to 4B, when an area in which the number of survivors is displayed is extracted, the time when the number of survivors is few remaining is the time when the game win is approaching, and the section and broadcast video that cause viewers' interest and interest It can be a highlight section of. The area is not limited to this, and some areas may include an area in which a game score of a sports game broadcast is displayed, an area in which a game score of a battle game broadcast is displayed, and the like. In various embodiments, the extractable item may include not only an area for recognizing characters, but also audio. For example, when audio extracted from a broadcast image is input to a sound prediction model, a time point at which the emotional voice of the broadcast creator is extracted can be predicted.

Next, the character displayed in the extracted area is recognized (S320). The OCR function may be used as a character recognition method, and the OCR function is a function for recognizing characters and replacing them with text data, and an unrestricted method such as a machine-learned character recognition model may be used. For example, in the case of the game implemented in FIG. 4A, the recognized characters may be the number of kills, the number of survivors, and so on. Without being limited thereto, the recognized characters may further include a game score of a sports game broadcast, a game score of a battle game broadcast, and the like.

Characters recognized in the extracted region are detected (S330). The detected text may be used to determine a section that caused viewers' interest and concentration, a highlight section of the broadcast video, and the like.

Next, it is determined whether the detected character corresponds to the condition (S340). For example, in the case of the game implemented in FIGS. 4A to 4B, in the case where only one person survives and wins and extracts some areas 420 representing the number of survivors, the recognized character is a number, and if it matches a predetermined keyword, We decide that it is condition. Without limitation, the condition of the detected character may include a case in which the character matches a predetermined keyword, a character is changed more quickly than a predetermined time point, and the like. Further, for example, a case where the detected character matches a keyword that means a plurality of predetermined kills may be determined as a highlight section of a broadcast video, which corresponds to a condition.

The time point when the character meets the condition is determined (S350). The video view point when the detected character is a condition is determined. The determined time point may be a time point displayed as a highlight section of the broadcast video. The present invention is not limited thereto, and in various embodiments of the present invention, a time point when the character corresponds to a condition may further include a time point earlier than a predetermined time from the determined time point.

An overlay to display the determined time point on the timeline is generated (S360). The overlay may include one or more types of graphs, figures, hatching, and the like. In addition, the overlay may be placed close to the time search area, but is not limited thereto, and the overlay may be displayed as translucent.

The viewer device 200 is provided with an image and an overlay (S370). Referring to FIG. 4B, the provided overlay may be provided as information such as a section in which a viewer has high interest in a broadcast video and a highlight section in the broadcast video. Without being limited thereto, the overlay may be provided as information capable of grasping the flow of the game in the broadcast video even without viewing the broadcast video. In addition, the overlay may be superimposed on the broadcast image, it may further include that the superimposed on the broadcast image may be displayed in translucent. The overlay provided with the video on the viewer device 200 may be an effect provided by information such as a section in which a viewer's interest is high in a broadcast video, a highlight section of the broadcast video, and a section in which the viewer wants to watch the video. .

5A and 5B are schematic diagrams for describing an image providing method according to another embodiment of the present invention.

5A and 5B exemplarily show images of a battle game in which game players compete with each other. The conditions for determining the section that caused the viewer's interest and concentration and the highlight section of the broadcast video may be a case in which a keyword representing a number of kills appears in the game. Referring to FIG. 5A, a region in which a keyword appears is extracted 520, and a recognized character is detected from the extracted region 520. Referring to FIG. 5B, when the detected character matches a predetermined keyword, a corresponding time point of the image is determined as a highlight section. Accordingly, an overlay 540 for displaying the highlight section may be generated and superimposed on the broadcast image and output. The overlay in one embodiment of the present invention is displayed on the timeline of the image, and the overlay form may be displayed in one or more types. The section in which the keyword meaning a number of kills appears may be a section that caused viewers' interest and concentration, a highlight section of the broadcast video, and the corresponding section as an overlay displayed on the timeline of the broadcast video on the viewer device 200 It can be an effect that information about is provided.

6A to 6D are schematic diagrams for describing an image providing method according to another embodiment of the present invention.

6A to 6D illustrate broadcast video screens of broadcast creators who perform survival games as examples. Referring to FIG. 6A, the number of kills and number of survivors acquired by the player is displayed on the broadcast image screen 630. Without being limited to this, the broadcast video screen 630 may include a player's character shape, information about a weapon in the game, and the like. The area 620 where the number of kills the player has obtained is extracted, and the recognized character is detected from the extracted area 620. At this time, the corresponding time point of the detected character can be determined and stored. The number of kills of the player on the broadcast video screen 630 may be displayed on the broadcast control area 610, and the corresponding time point of the video can be seen to be 3 hours 55 minutes 26 seconds of the broadcast video.

Referring to FIG. 6B, it can be seen that the region 620 in which the number of kills obtained by the extracted player is 6 is the number of kills, and the corresponding time is 3 hours 56 minutes 28 seconds of the broadcast.

Referring to FIG. 6C, it can be seen that the region 620 in which the number of kills obtained by the extracted player appears is the number of kills 7, and the corresponding time point is 3 hours 59 minutes 30 seconds of the broadcast.

Referring to FIG. 6D, it is checked whether the number of kills obtained by the players extracted in FIGS. 6A to 6C corresponds to a condition, and when the condition is met, an overlay displayed on the screen is generated. The stored viewpoints may be displayed as an overlay on the broadcast image screen 630. In this case, the condition is a case in which the character recognized in the extracted region changes faster than the predetermined time. The point in time when the number of kills changes rapidly may be a highlight section in the game, or an effect in which information about the highlight section of a broadcast video is provided to viewers.

7A to 7D are schematic diagrams for describing an image providing method according to various embodiments of the present invention.

7A to 7D illustrate a method for providing broadcast flow data as a broadcast video screen 730 for performing a survival game. Referring to FIG. 7A, the number of survivors alive among players in the game may be displayed on the game screen. In addition, the broadcast video screen 730 may further include a broadcast control area 710 in which the viewpoint of the video is displayed. It can be seen that the region 720 in which the number of survivors in the game is displayed can be extracted, and the number of survivors is 4. It can be seen that the time when the number of survivors is 4 is 3 hours 55 minutes 26 seconds, which is displayed on the broadcast control area 710, and this can be determined as the corresponding time.

Referring to FIG. 7B, it can be seen that the region 720 in which the number of survivors to be extracted is 3 is the number of survivors, and the corresponding time point is 3 hours, 56 minutes, and 28 seconds of the broadcast, and it can be determined as the corresponding time point.

Referring to FIG. 7C, it can be seen that the region 720 of the number of survivors to be extracted is 2 survivors, and the corresponding time is 3 hours 59 minutes 30 seconds of the broadcast, and it can be determined as the corresponding time.

Referring to FIG. 7D, an overlay is generated to display the number of survivors of the in-game players extracted in FIGS. 7A to 7C and the corresponding time point on the video timeline. The generated overlay may be displayed on the broadcast video screen 730. Without being limited thereto, the overlay may be generated as a graph displayed in time series on the broadcast video screen 730. If the number of survivors is displayed as an overlay on the broadcast video screen 730, an effect capable of grasping the flow of the game broadcast may be provided without viewing the video.

8A and 8B are schematic diagrams for describing an image providing method according to various embodiments of the present invention.

8A and 8B are exemplary views for explaining the display of the overlay type.

Referring to FIG. 8A, in the broadcast video screen 830, the broadcast control area 810 includes buttons for performing screen control such as playback, pause, voice adjustment, screen maximization, and playback control, and displays an overlay. It may further include a selection button for.

When the selection button for overlay display is selected in the broadcast control area 810, the overlay display list 820 is displayed on the broadcast image screen 830. The overlay display list 820 may be provided with information on a section that caused viewers' interest and concentration, a highlight section of a broadcast video, broadcast flow data, and viewer compatibility data, but is not limited thereto. It may further include that information about the contribution is provided. In addition, a button for ending the overlay display list 820 window may be further included in the overlay display list 820 window.

Referring to FIG. 8B, when a button for displaying a graph of the number of users is selected in the overlay display list 820, a graph type overlay may be displayed on the broadcast image screen 830. Without being limited thereto, one or more various types of overlays may be displayed on the overlay display area 840 of the broadcast image screen 830 according to the selected button. Further, the overlay displayed on the overlay display area 840 may further include one or more numbers displayed. The display number of the overlay will be described later with reference to FIG. 9. The overlay display list 820 may be configured with information on a section that caused viewers' interest and concentration without watching a broadcast video, a highlight section of the broadcast video, broadcast flow data, and viewer response data. Accordingly, the overlay display list 820 may be an effect of providing information on a section that caused viewers' interest and concentration and a highlight section of the broadcast video without watching the broadcast video.

9 is a schematic diagram illustrating a method of providing an image according to various embodiments of the present invention.

9 is an exemplary diagram for explaining the display of the overlay type.

When the selection button for overlay display is clicked in the broadcast control area 910 of the broadcast video screen 930, the overlay display list 920 is displayed on the broadcast video screen 930. Two or more types of overlays selected from the displayed overlay display list 920 may be overlapped and displayed in the overlay display area 940. The overlay displayed overlaid may be superimposed on the broadcast image screen 930 and displayed as translucent. Without being limited thereto, the types of overlays displayed may include that they may or may not match. The overlapping and displayed overlay may provide one or more pieces of information regarding the number of users, the degree of response, the degree of contribution to the broadcast, and the section that caused viewers' interest and concentration, and may have the effect of providing one or more pieces of information simultaneously.

10A to 10C are schematic diagrams for describing an image providing method according to various embodiments of the present invention.

Referring to FIG. 10A, a voice 1020 of a broadcast creator in a broadcast video is extracted. Without being limited thereto, the extracted voice may be all other than the voice of the broadcast creator in the broadcast image. It can be seen that the corresponding time point of the image in the broadcast control area 1010 is 7 minutes 33 seconds.

Referring to FIG. 10B, the extracted broadcast creator's voice 1020 is input to a sound prediction model. The sound prediction model may predict a time point in which there is an emotional voice in the input voice. The time point displayed on the broadcast control area 1010 may be predicted as 7 minutes 33 seconds. The predicted time point may be determined as the corresponding time point.

Referring to FIG. 10C, one or more types of overlays are generated to indicate the predicted viewpoint in the sound prediction model. The generated overlay can be displayed at the predicted corresponding time point. Without being limited to this, after selecting a button for selecting an overlay display list in the broadcast control area 1010, information on a point in time when there is an emotional voice in the displayed overlay display list may be provided. In addition, the overlay displayed on the broadcast video may be provided with information on the viewpoint of the emotional voice of the broadcast creator without watching the broadcast video. Referring to FIG. 10C, a broadcast creator's laughter voice 1020 is extracted from a broadcast video. The laughter voice 1020 of the extracted broadcast creator is input to the sound prediction model, and the corresponding time point is predicted. The predicted viewpoint is displayed over the broadcast control area 1010 of the image as an overlay 1040 generated to indicate the viewpoint. The displayed overlay may have an effect of providing information on a section in which the emotional voice of the broadcast creator appears, a section in which the viewer's interest is high, etc. without viewing the broadcast video.

100: video providing device
110: communication unit
120: input unit
130: storage unit
140: processor
200: viewer device
210: communication unit
220: input unit
230: display unit
240: storage unit
250: processor
300: broadcast creator device
410: broadcast control area
420: extraction area
430: broadcast video screen
440: overlay display area
510: broadcast control area
520: extraction area
530: broadcast video screen
540: overlay display area
610: broadcast control area
620: extraction area
630: Broadcast video screen
640: overlay display area
710: broadcast control area
720: extraction area
730: Broadcast video screen
740: overlay display area
810: broadcast control area
820: overlay display list
830: broadcast video screen
840: overlay display area
910: broadcast control area
920: Overlay display list
930: broadcast video screen
940: overlay display area
1010: broadcast control area
1020: extraction area
1030: Broadcast video screen
1040: overlay display area

Claims (12)

Extracting a partial area of the image;
Recognizing and detecting characters displayed in the extracted partial region;
Determining whether the detected character corresponds to a condition;
Determining a point in time when the character corresponds to a condition;
Generating an overlay in which the determined time point is displayed in a timeline; And
And providing the video and the generated overlay to a viewer device. According to claim 1,
When the character is a condition, including the case that the character matches a predetermined keyword, the video providing method. According to claim 1,
Extracting additional regions different from the partial regions;
Recognizing and detecting additional characters displayed in the additional area;
Determining whether the detected additional character corresponds to a condition; And
The method further includes determining an additional time point when the additional character corresponds to a condition, and generating the overlay includes generating an overlay displayed on the timeline and the time point. How to provide video. According to claim 1,
When the character is a condition, including the case that the character fluctuates faster than a predetermined time, video providing method. According to claim 1,
When the time point is determined, the overlay displays a time point earlier than a predetermined time from the determined time point as an actual time point. According to claim 1,
The character is a number,
Generating the overlay comprises:
And generating a graph displaying the numbers in time series as an overlay. According to claim 1,
The step of providing the viewer device with the video and the generated overlay,
Selectively providing at least one overlay type to the viewer device;
If a selection for the overlay type is received, generating an additional overlay based on data associated with the selected overlay type;
And providing the additional overlay together with the image. The method of claim 7,
The additional overlay is displayed overlaid with the overlay, the image providing method. The method of claim 7,
The overlay type is the number of users, response, broadcast contribution, and at least one graph type, video providing method. The overlay type is an overlay type in which the time when the emotional voice of the broadcast creator is identified is displayed on the timeline,
The identification of the emotional voice of the broadcast creator is performed by inputting audio of the video into a sound prediction model configured to predict when the emotional voice comes out, and predicting a point in time when the emotional voice is present. According to claim 1,
Before the step of extracting a partial region of the image, further comprising the step of determining the partial region of interest by analyzing the image. A communication unit configured to receive the image; And
And a processor connected to be operable with the communication unit,
The processor extracts a partial region of the image,
Recognize and detect the characters displayed in the extracted partial area,
Determine whether the detected character is a condition,
Determine the point in time when the character falls under the condition,
The determined time point generates an overlay displayed on the timeline,
And providing the viewer device with the image and the generated overlay.

Images