KR102291542B1 - Method and apparatus for synchronizing user feedback information of content streaming service - Google Patents

Abstract

The present invention relates to a method for synchronizing user feedback information in a content streaming service, which includes the steps of: receiving and playing a video from a server (hereinafter, 'server') which provides a video streaming service to a plurality of devices; displaying a playback screen of the played video; receiving at least one piece of feedback information on the video input by at least one external device among the plurality of devices from the server; and displaying the received feedback information on the playback screen of the video.

Description

Apparatus and method for synchronizing user feedback information of content streaming service

The present invention relates to an apparatus and method for synchronizing user feedback information of a content streaming service.

The main broadcasting contents that were produced and provided by existing broadcasting companies and large content agencies are now being transformed into one-person media contents produced and shared by various individuals. The number of users of platform services that support one-person media such as YouTube and Afreeca TV is also increasing every year.

This change in content has also changed the way services are provided to users. For example, in the past, it was a one-way service that simply receives and reproduces content, such as radio and television, but now it is a two-way service that not only receives content, such as a smartphone or laptop, but also provides feedback information about the content in real time. changed in the form of services. Specifically, users may exchange opinions on the same content in real time, and, in particular, express the user's feelings about the content in real time through images such as emoticons.

However, in the case of such an interactive service, a difference in the reproduction speed of the same content occurs on each user device. This is caused by various causes, such as network environment, performance of the user device, pause operation by the user, and environment setting related to the reproduction of the corresponding content. However, feedback information that is input, transmitted, and displayed by each user device does not consider a difference in the reproduction speed of the corresponding content.

Accordingly, the user may be provided with feedback information input by another user device that does not match the content being reproduced on the user device due to the difference in the content reproduction speed. Accordingly, the user may be disturbed by receiving specific information related to a specific point in time of the content in advance before an image corresponding to the specific point in time is reproduced on the user device. If such a situation occurs frequently, the user's interest in viewing the corresponding content is lowered, and eventually, the user who uses the service providing the corresponding content leaves the user.

Republic of Korea Patent Publication No. 10-1900471, (2018.09.13)

An object of the present invention for solving the above problems is to provide an apparatus and method for synchronizing feedback information of a user of a content streaming service.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In the method performed by the device according to an aspect of the present invention for solving the above-described problems, in the method performed by the device, a server (hereinafter, 'server') that provides a video streaming service to a plurality of devices Receiving and playing the video from the server, displaying a playback screen of the reproduced video, and receiving at least one piece of feedback information about the video input by at least one external device among the plurality of devices from the server and displaying the received feedback information on a playback screen of the moving picture, wherein the feedback information is displayed on the external device at the time information and the input time when the feedback information is input by the external device. Including frame information of a moving picture, wherein the displaying step includes: identifying frame information of a moving picture displayed on the playback screen; and based on the frame information, feedback information matching each frame is reproduced by each frame and displaying it on the playback screen at the time when it becomes available.

In addition, in the display step, when the number of matching feedback information for the same frame is plural, the plurality of pieces of feedback information are sequentially displayed based on the input time of each feedback information.

The method further includes calculating the number of feedback information matching each frame, wherein the displaying includes displaying the feedback information when the calculated number is equal to or greater than a preset value.

Also, in the displaying step, when feedback information matching the specific frame is received after displaying the specific frame of the moving picture, the received feedback information is not displayed on the playback screen.

In addition, the calculating step further includes identifying a feedback type of feedback information matching each frame and calculating the number of feedback information for each feedback type for each frame, wherein the displaying step includes a preset first The method further includes not displaying all feedback information matching the frame when the number of feedback information corresponding to the feedback type is greater than or equal to a preset value.

In addition, the step of receiving the first feedback information about the video from the user and transmitting the first feedback information about the video to the server, wherein the server is a difference in the playback time of the video between a plurality of devices receiving the video The method further includes a first feedback information transmission step of calculating , and determining a device for transmitting the first feedback information based on the difference in the playback time, wherein the displaying step includes displaying the received feedback information on the playback screen displayed on the

In addition, when the difference between the reproduction time is less than a preset time, the server is characterized in that it does not transmit the first feedback information.

The method further includes: calculating time difference information between the video and the device based on the first time information on the video and the second time information on the device, wherein the displaying step is based on the time difference information receiving reference frame information corresponding to a point in time at which the feedback information is input among a plurality of frames of a video, and a frame corresponding to the received reference frame information among a plurality of frames in the video currently being reproduced is reproduced , displaying the feedback information, wherein the displaying of the feedback information does not display the feedback information on the playback screen of the video when the number of feedback information received from the server is less than a preset number. .

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium for recording a computer program for executing the method may be further provided.

According to the present invention as described above, to a plurality of users receiving the same video streaming content, the effect of displaying the feedback information of other users matching the content reproduced in each user device in the time information of the reproduced video streaming content perform More specifically, due to feedback information input by another user device, spoiler information related to the video content that has not yet been played in the device may be prevented from being provided.

In addition, when a user inputs feedback information for a video frame in which feedback information is input by other user devices, a visual or auditory effect corresponding to a synergistic effect is reproduced, thereby facilitating the user to input feedback and working together with other users. It can give you a sense of presence as if you were watching a video.

In addition, feedback less than a preset number generated for a specific video frame by a small number of other user devices is not displayed, thereby improving the immersive feeling of video viewing.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a configuration diagram of a system for synchronizing feedback information of a user of a streaming service according to an embodiment of the present invention.
2 is a flowchart schematically illustrating a method for synchronizing user feedback information of a content streaming service according to an embodiment of the present invention.
3 is an exemplary diagram illustrating a display unit displaying user feedback information according to an embodiment of the present invention.
4 is a flowchart schematically illustrating a method for displaying user feedback in synchronization according to an embodiment of the present invention.
5 is an exemplary diagram illustrating sequential output of a plurality of pieces of feedback information based on time information at which each piece of feedback information is input.
6 is a flowchart schematically illustrating a method of displaying feedback information on a reproduction screen based on the number of feedback information according to an embodiment of the present invention.
7 is a flowchart schematically illustrating a method of displaying feedback information according to the number of feedback information for each feedback type according to an embodiment of the present invention.
8 is a flowchart schematically illustrating a method of transmitting first feedback information input by a user device to another device according to an embodiment of the present invention.
9 is a flowchart schematically illustrating a method of displaying feedback information based on time difference information and reference frame information according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system 10 for synchronizing feedback information of a user of a streaming service according to an embodiment of the present invention.

Referring to FIG. 1 , a system 10 for synchronizing feedback information of a user of a streaming service according to an embodiment of the present invention includes a server 100 and a user device 200 that provide a content streaming service.

The server 100 (hereinafter, 'server') that provides a content streaming service includes a content collecting unit 110 that collects content, a content providing unit 120 , a feedback information collecting unit 130 , and a feedback information transmitting unit 140 . ) and a time difference information collecting unit 150 and a processor 160 of the user device.

Although not clearly shown in the drawings, the content collecting unit 110, the content providing unit 120, the feedback information collecting unit 130, the feedback information transmitting unit 140, and the time difference information collecting unit 150 of the user device are each It includes a communication module.

The server 100 may transmit/receive various information between the user devices 200 through the communication module. For example, content may be transmitted to each user device 200 , and feedback information input through each user device 200 may be received. In addition, time difference information about each user device 200 may be received. Meanwhile, through the communication module, the server 100 can use various types of communication networks, for example, WLAN (Wireless LAN), Wi-Fi, Wibro, Wimax, HSDPA (High Speed Downlink Packet). Access) or wired communication methods such as Ethernet, xDSL (ADSL, VDSL), HFC (Hybrid Fiber Coax), FTTC (Fiber to The Curb), FTTH (Fiber To The Home) may be used. can

The server collects content such as video and voice through the content collection unit 110 . For example, assuming that the content is a soccer game, the server receives image information about the corresponding soccer game from a camera and an audio device communicating with the server. Meanwhile, although not clearly shown in the drawings, the content collection unit 110 may include a plurality of content collection units 110a, 110b, etc. according to the subject and type of the content.

Meanwhile, the server transmits the content received through the content collecting unit 110 to the user device 200 registered in the server through the content providing unit 120 . In this case, the user may be provided with specific content by accessing a channel classified according to the content provided by the corresponding server. This will be described in more detail in the user device 200 .

Meanwhile, the feedback information collecting unit 130 collects feedback information input by a user who has been provided with specific content. The feedback information means a specific emoticon, text, etc. which is input through the user device after being set in advance by the server. Feedback information is collected for each content. For example, it is assumed that the first content related to the soccer game and the second content related to the basketball game are simultaneously collected by the server and provided to each user device 200 connected to each content channel. In this case, the server separates and collects the first feedback information input from the user device 200 provided with each first content and the second feedback information input from the user device 200 provided with the second content.

Meanwhile, the feedback information transmitting unit 140 transmits the collected user feedback information. This may be displayed on a screen on which the corresponding content is displayed, or it may be displayed on a separate web page (eg, a chat room opened for specific content) associated with the content. However, the present invention is not limited thereto. Meanwhile, the feedback information is transmitted only to the feedback information input from the user device 300 provided with the corresponding content. Referring back to the above example, feedback information collected from the user device 200 that has received the second content related to the basketball game is not transmitted to the user device 200 that has received the first content related to the soccer game.

Meanwhile, the server collects time difference information of content for each user device 200 through the time difference information collecting unit 150 . Specifically, in the case of each user device receiving the same content, the playback speed displayed on the user device 200 may be different depending on a network environment or an environment setting related to content playback of the user device 200 . Accordingly, the server collects time difference information related to the corresponding content from each user device 200 receiving the same content ( S150 ). A method of generating time difference information will be described in detail below.

Referring to FIG. 1 , according to an embodiment of the present invention, a user device 200 includes a communication unit 210 , a display unit 220 , and a processor 230 .

2 is a flowchart schematically illustrating a method for synchronizing user feedback information of a content streaming service according to an embodiment of the present invention.

Referring to FIG. 2 , in the user device 200 , the communication unit 210 receives and reproduces content from the server 100 ( S310 ). In addition, the display unit of the user device 200 displays a playback screen of the received content (eg, a video of a soccer game) ( S320 ).

The processor 230 receives at least one piece of feedback information on the video input by at least one external device among a plurality of devices from the server through the communication unit (S330).

Specifically, when receiving specific content from the server 100 , feedback information input from the user device 200 receiving the corresponding content and from another user device other than the user device 200 is received. For example, it is assumed that an image about a soccer game is received from the server 100 . At this time, a plurality of users who watch the soccer game, more precisely, connect to a channel related to the soccer game, and receive the video of the soccer game, may input feedback information in real time through each user device. . At this time, the server 100 transmits the feedback information to a plurality of user devices 200 receiving the corresponding soccer game image, and each user device 200 receives the feedback information through the communication unit 210 . will be.

On the other hand, the processor 230 displays the received feedback information on the playback screen of the video (S340).

3 is an exemplary diagram illustrating a display unit displaying user feedback information according to an embodiment of the present invention.

Referring to FIG. 3 , it can be seen that the feedback information input from the first user device 200a and the second user device 200b receiving the soccer game video is displayed on the third user device 200c. Of course, although not clearly shown in the drawings, the feedback information is simultaneously transmitted to the first user device 200a and the second user device 200b as well as to the third user device 200c, and each of the first user devices 200a ) and displayed on the display unit of the second user device 200b.

Meanwhile, according to an embodiment of the present invention, the feedback information includes time information at which the feedback information is input by the external device and frame information of a moving picture displayed in the external device at the input time.

Specifically, referring to FIG. 3 , the feedback information input from the first user device 200a includes time information at which the corresponding feedback information was input by the first user device 200a and the first user device 200a at the time of input. Contains frame information of the video displayed in . For example, if it is assumed that the first user device 200a received input from the server 100 at 13:20 and then inputted when the 128th frame of a video reproduced is displayed on the display unit, the corresponding feedback information is It includes information about the time of 13:20 and frame order information (128th frame) of the video displayed on the screen at the time of input. That is, each piece of feedback information includes a time at which each feedback is input and frame information within a moving picture. Through this, it is possible to search for feedback information input from another external device that matches each frame in the video displayed on the user device.

4 is a flowchart schematically illustrating a method for displaying user feedback in synchronization according to an embodiment of the present invention.

Meanwhile, in an embodiment of the present invention, the displaying step (S340) includes identifying frame information of a video displayed on the playback screen (S341) and matching each frame based on the frame information (S341). and displaying the feedback information on the reproduction screen at the time each frame is reproduced (S342).

Specifically, when the video received from the server 100 is displayed on the playback screen, the processor 230 identifies each frame information constituting the video. For example, order information of a frame currently being reproduced is identified by accumulating and calculating the order of frames in a moving picture. Alternatively, order information of a frame received from the server 100 but still waiting to be reproduced as well as a frame currently being reproduced is identified.

Then, based on the identified frame information, feedback information matching each frame is displayed on the reproduction screen at the time the corresponding frame is reproduced (S342). Specifically, it is assumed that the first feedback information and the second feedback information are respectively input from the first user terminal and the second user terminal with respect to the twelfth frame among the frames constituting the moving picture. The processor 230 searches for feedback information matching the twelfth frame among the feedback information received from the server 100 . At this time, the first feedback information and the second feedback information will be searched. Then, when the twelfth frame is displayed on the reproduction screen, the processor 230 displays the first feedback information and the second feedback information.

Through this, the user may be provided with feedback information of other users synchronized with the video. For example, it is assumed that a soccer game is received from the server 100 . In this case, if the first user has a faster playing speed of the soccer game video than the second user, the first user may watch the soccer match video faster than the second user. If a goal scene is displayed in the corresponding soccer game, the content is recognized in advance even though the goal scene has not yet been displayed on the second user's terminal due to the feedback information of the first user. In other words, the immersion of the video is inevitably reduced. As described above, the present invention does not immediately display the feedback information of the user who receives the same video as described above, but searches for and displays the feedback information of the user matching the frame being played in each user device, so that each user can Feedback information of other users corresponding to the video being viewed may be recognized.

On the other hand, in an embodiment of the present invention, in the displaying step, when the number of matching feedback information for the same frame is plural, a plurality of feedback information is sequentially displayed based on the time at which each feedback information is input. can

As described above, each feedback information includes input time information of each feedback information.

For example, the first feedback information input from the first user terminal is input at 13:20:34, the second feedback information is input at 13:20:36, and the second feedback information input from the second user terminal Assume the information was entered at 13:20:35. In this case, the above-described first feedback information, 1-2 feedback information, and second feedback information were all input for the 18th frame of the same moving picture. At this time, on the playback screen of the third user terminal receiving the same video, feedback information 1-1 is displayed when the 18th frame of the video is displayed, and then feedback information 1-2 is finally displayed after the second feedback information is displayed. information will be displayed.

That is, when there is a plurality of feedback information for the same frame, the processor 230 sequentially displays the plurality of pieces of feedback information on the user device based on the input time of each feedback information.

5 is an exemplary diagram illustrating sequential output of a plurality of pieces of feedback information based on time information at which each piece of feedback information is input.

Referring to FIG. 5 , first feedback information is input from a second user device 320 other than the first user device 310 among a plurality of users receiving the same video provided from the server 100 and the third user device Second feedback information is input from 330 . In this case, the first feedback information and the second feedback information include time information at which each feedback information is input. That is, the first feedback information includes time information of 14:22:30, which is the time when the corresponding information is input by the second user device 320 , and the second feedback information corresponds to the corresponding information by the third user device 330 ). It includes time information of 14:21:15, which is the time when the information was input. In this case, after receiving the first feedback information and the second feedback information, the processor of the third user device 330 receives the first feedback information and the second feedback information at a point in time when a specific frame corresponding to the corresponding feedback information is reproduced (14:22:45). 2 Display feedback information. At this time, based on the time information, the second feedback information is displayed before the first feedback information.

6 is a flowchart schematically illustrating a method of displaying feedback information on a reproduction screen based on the number of feedback information according to an embodiment of the present invention.

Meanwhile, referring to FIG. 6 , the method for synchronizing user feedback information of the above-described content streaming service according to an embodiment of the present invention may further include calculating the number of feedback information matching each frame ( S331 ). In this case, in the displaying step (S340), when the calculated number is equal to or greater than a preset value, the feedback information is displayed.

The processor 230 calculates the number of feedback information matching each frame. In this case, the frame may be a frame within the video that has not yet been received from the server 100 . That is, even when the frame of the video currently displayed on the playback screen of the user device 200 is the 15th frame, if the specific feedback information received from the server 100 is for the 20th frame, the processor 230 is configured to display the 20th frame. It is possible to calculate the number of feedback information matching with .

Meanwhile, after calculating the number of feedback information matching each frame, the processor 230 displays the received feedback information in the corresponding frame only when the number is equal to or greater than a preset value (the first reference value).

For example, it is assumed that there are five first reference values. In this case, it is assumed that the number of feedbacks calculated by the processor 230 is as follows. It is assumed that the number of feedback information received for the 15th frame is 10, the number of feedback information received for the 16th frame is 4, and the number of feedback information received for the 17th frame is 6 pieces. In this case, the processor 230 will display the matching 10 pieces of feedback information on the playback screen when the 15th frame is displayed while displaying the video. However, 4 pieces of feedback information matching the 16th frame will not be displayed. Finally, six pieces of feedback information matching the 17th frame are displayed on the playback screen when the 17th frame is displayed.

This is in order not to display erroneously input feedback information of other user devices. That is, the user is provided with accurate feedback information of other users for each frame, in order to increase the sense of immersion.

Meanwhile, the first reference value may be determined according to the number of user terminals that watch a video or receive a video.

In addition, according to an embodiment of the present invention, in the displaying step ( S340 ), when feedback information matching the specific frame is received after displaying a specific frame of the video, the received feedback information is displayed on the playback screen. may not

That is, the processor 340 does not display feedback information for a frame already displayed on the screen. For example, after the 15th frame of the video is displayed on the playback screen, even if the fourth feedback information is received from the server 100 from the fourth user device, which is another device for the frame, the processor 340 is displayed on the playback screen. The fourth feedback information is not displayed.

At this time, according to an embodiment of the present invention, after displaying a specific frame, feedback information matching the specific frame is received, and the difference between the time at which the corresponding feedback information is input and the time at which the specific frame is displayed is a preset value (the second reference value) or less, corresponding feedback information may be displayed.

Referring back to the above-described example, the second reference value is 2 seconds, the time when the 15th frame is displayed on the playback screen is 13:20:52, and the time when the received fourth feedback information is input to the fourth user device Assume 13:20:53. At this time, even if the frame after the 15th frame, for example, the 16th frame is displayed on the user device, the processor 340 displays the fourth feedback information matching the 15th frame on the reproduction screen.

That is, even if feedback information on a frame that has already been displayed on the playback screen is input within a predetermined time, it is displayed on the playback screen. At this time, the above-described embodiment of the present invention determines whether the difference between the time when the feedback information is received from the server 100 and the time when a specific frame is displayed on the playback screen is equal to or less than a second reference value, and displays the feedback information. You may decide whether to

7 is a flowchart schematically illustrating a method of displaying feedback information according to the number of feedback information for each feedback type according to an embodiment of the present invention.

Referring to FIG. 7 , in the above-described calculating step (S331) according to an embodiment of the present invention, the step of identifying a feedback type of feedback information matching each frame (S331_a) and the number of feedback information for each feedback type for each frame Further comprising the step of calculating (S331_b), wherein the displaying step (S340) does not display all feedback information matching the frame when the number of feedback information corresponding to the preset first feedback type is greater than or equal to a preset value It may further include a step (not shown).

Specifically, each piece of feedback information may be classified according to a preset feedback type. For example, the feedback type may be set to positive, neutral, or negative. Alternatively, each feedback type may be set according to an emotional state such as joy, sadness, anger, and the like.

Meanwhile, the processor 230 identifies a feedback type of each feedback information. The feedback type may be identified based on respective metadata of each emoticon provided by the server 100 .

Meanwhile, the processor 230 calculates the number of pieces of feedback information matching each frame according to the feedback type. For example, for the 15th frame, the number of feedback information corresponding to positive, the number of feedback information corresponding to neutral, and the number of feedback information corresponding to negative are respectively calculated. In this case, when the number of feedback information corresponding to the preset first feedback type is greater than or equal to a preset value, the processor 230 may not display all feedback information matching the frame. For example, assume that the first feedback type is negative. And if it is assumed that the preset value (the third reference value) is 100, the processor 230 responds to the frame if there are 110 feedback information corresponding to the negative type among the feedback information input by other user devices for a specific frame. Information may not be displayed. That is, the remaining feedback information corresponding to positive and neutral for the frame as well as negative may not be displayed.

In another embodiment of the present invention, the displaying step ( S340 ) may display all feedback information matching the frame only when the number of feedback information corresponding to the preset first feedback type is equal to or greater than a preset value. There will be. That is, only when the first feedback type set by the user is equal to or greater than a preset value, the entire feedback information matching the frame is displayed.

8 is a flowchart schematically illustrating a method of transmitting first feedback information input by a user device to another device according to an embodiment of the present invention.

Referring to FIG. 8 , the processor 230 receives first feedback information about the video from the user ( S350 ). Then, the first feedback information about the video is transmitted to the server (S360). In this case, the server 100 may calculate a difference in playback time of the video among a plurality of devices receiving the video, and determine a device for transmitting the first feedback information based on the difference in the playback time. Meanwhile, the processor 230 displays the received first feedback information on the reproduction screen.

Specifically, the processor 230 displays the first feedback information input from the user on the reproduction screen, and transmits the first feedback information to the server 100 . And the server 100 transmits the first feedback information to a device other than the user device. That is, as described above, feedback information of a plurality of users for the same video can be shared.

Meanwhile, the server 100 calculates a difference in the playback time of a video between a plurality of devices receiving or watching the same video. More precisely, the difference in playback speed is to calculate the time difference in which the same frame is played back on each device. Then, an external device to which the first feedback information is transmitted is determined based on the calculated time difference.

In this case, according to an embodiment of the present invention, the server 100 may not transmit the first feedback information when the difference between the reproduction times is less than a preset time. Specifically, the server 100 may not transmit the first feedback information to an external device in which the difference between the playback time of the user device that has transmitted the first feedback information and the other external device is less than a preset time (fourth reference time). have. That is, if the time is less than the preset time, even if the first feedback information is transmitted, the frame matching the first feedback information may already be displayed on the playback screen of the corresponding external device, so that the frame that is displayed thereafter is exactly matched. This is to transmit only feedback information. Meanwhile, the reproduced time difference is calculated based on the device to which the first feedback information is input. Therefore, it is natural that a negative (-) value may be calculated for a device having a faster playback speed than a device to which the first feedback information is input.

On the other hand, although not clearly shown in the drawings, according to an embodiment of the present invention, based on the first time information about the video and the second time information about the device, calculating the time difference information between the video and the device It may further include a step (not shown).

Specifically, the processor 230 calculates time difference information between the video and the device based on the first time information on the video and the second time information on the device (not shown).

In this case, in an embodiment of the present invention, the moving picture is collected in real time by the server and provided to a plurality of devices, and the time difference information is a first calculated based on meta information of each frame constituting the moving picture. It may be periodically calculated based on the time information and the second time information received from the time server.

Meanwhile, when the content received from the server 100 is a moving picture, the first time information is calculated based on meta information about the frame of the moving picture. That is, based on the metadata of each frame of the corresponding video collected by the server 100 , based on the second time information, which is real-time time information when each frame is reproduced in the user device 200 , the video and the This is to calculate time difference information between user devices.

For example, it is assumed that the first time information about the nth frame included in the moving picture is 13:23:11 (13:23:11). In this case, it is assumed that the second time information obtained through the time module in the user device when the corresponding n-th frame is reproduced in the user device is 13:23:14 (13:23:14). In this case, the time difference information will correspond to 2 seconds. Meanwhile, in order to calculate the second time information, the user device 200 acquires the second time information through communication with the server 100 or a specific time information providing server (not shown) instead of a time module (not shown). it might be

Meanwhile, the time difference information may be generated by various factors as described above. For example, buffering may occur in playing a video according to the network environment of each user device 200 , or may occur according to an environment setting of video playback set in each user device 200 .

Meanwhile, although not clearly shown in the drawing, each user device 200 may transmit time difference information to the server 100 . Through this, the server 100 may acquire time difference information of a plurality of user devices 200 that are provided with a specific content (reproduce a specific content), respectively.

9 is a flowchart schematically illustrating a method of displaying feedback information based on time difference information and reference frame information according to an embodiment of the present invention.

Referring to FIG. 9 , in an embodiment of the present invention, the processor 340 sets reference frame information corresponding to a time point at which the feedback information is input among a plurality of frames of a video based on the time difference information (S343), The feedback information is displayed at a point in time when a frame corresponding to the received reference frame information among a plurality of frames in the video currently being reproduced is reproduced (S344).

More specifically, it has been described above that even when the same content is provided from the server 100, the reproduction speed of the content reproduced in each user device may be different due to various factors. However, in this case, feedback information input in real time from each user device 200 regarding the corresponding content may be transmitted and displayed to each user device regardless of the difference in the corresponding reproduction speed.

For example, it is assumed that a goal scene is included in the nth frame in relation to a soccer image. At this time, the user device receiving the corresponding image will input feedback information such as an emoticon corresponding to the goal scene. In this case, the feedback information may be received and displayed from the server even though the image of the nth frame is not reproduced in a specific user device in which a difference in reproduction speed occurs due to a network environment. That is, the content displayed through the user device 200 may not match the feedback information received and displayed through another device. That is, the user is provided with information on a scene to be reproduced in advance according to the feedback information input by the other device even though the video scene to which the video scene reacted by the other device or to which the feedback information is input has not been reproduced.

To this end, the processor 230 sets a reference frame for a plurality of feedback information. Alternatively, information on the reference frame may be received from the server 100 and set. That is, the reference frame information corresponding to the time when the feedback information is input by the user device 200 in relation to the content is received.

Referring back to the above example, the reference frame may be the nth frame including the goal scene. In addition, the processor 230 displays feedback information input by another device received from the server 100 at the time the reference frame is reproduced. That is, when the nth frame, that is, a goal scene, is reproduced in the user device 300 , feedback information input by another device is displayed. Through this, feedback information input by another device matching the content reproduced by the user device 300 can be displayed to the user. In this case, the feedback information input by another device may be displayed in the form of blinking, image overlap, and the like.

Meanwhile, in an embodiment of the present invention, the reference frame may be set based on the initial feedback information input to the server during the precious period.

That is, a frame corresponding to a time point at which the first feedback information is input among the plurality of user devices 300 receiving the same video is set as the reference frame.

In this case, in an embodiment of the present invention, when the amount of feedback information received from the server is less than a preset number (a fifth reference value), the processor 340 may cancel the setting of the reference frame.

That is, when feedback information equal to or greater than a preset number (fifth reference value) is input by a plurality of other user devices other than the user device inputting the first feedback information after the time when the first feedback information is input, the first feedback information is input A frame corresponding to the time point may be set as a reference frame.

Specifically, it is assumed that the first feedback information is input from the first user device to the nth frame constituting the moving picture. In this case, when the number of feedback information input from a plurality of user devices other than the first user device (second user device, third user device, etc.) is equal to or greater than a preset number, a frame corresponding to the time point at which the initial feedback information is input is set as the reference frame.

2 and 4 are described as sequentially executing steps S310 to S352, but this is merely illustrative of the technical idea of this embodiment, and those of ordinary skill in the art to which this embodiment belongs As it will be applicable by changing the order described in FIGS. 2 and 4 and executing one or more of steps S310 to S352 in parallel within a range not departing from the essential characteristics of the present embodiment, various modifications and variations may be applied. 2 and 4 are not limited to a chronological order.

The method according to an embodiment of the present invention described above may be implemented as a program (or application) to be executed in combination with a server, which is hardware, and stored in a medium.

The above-described program is C, C++, JAVA, machine language, etc. that a processor (CPU) of the computer can read through a device interface of the computer in order for the computer to read the program and execute the methods implemented as a program It may include code (Code) coded in the computer language of Such code may include functional code related to functions defining functions necessary for executing the methods, etc. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer should be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected by a network, and a computer readable code may be stored in a distributed manner.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10: Synchronization system of user feedback information of streaming service
100: a server providing a content streaming service
110: content collection unit
120: content providing unit
130: feedback information collection unit
140: feedback information transmitter
150: time difference information collection unit of the user device
160: processor
200: user device
210: communication department
220: display unit
230: processor

Claims (10)

A method performed by an apparatus comprising:
Receiving and playing the video from a server (hereinafter, 'server') that provides a video streaming service to a plurality of devices;
displaying a reproduction screen of the reproduced video;
receiving, from the server, at least one piece of feedback information about the video input by at least one external device among the plurality of devices;
calculating the number of feedback information matching each frame;
Displaying the received feedback information on a playback screen of the video; including,
The feedback information is
The feedback information includes time information input by an external device and frame information of a video displayed on the external device at the input time,
The display step is
identifying frame information of a video displayed on the playback screen; and
Based on the frame information, displaying feedback information matching each frame on the playback screen when each frame is played back;
When the number of matching feedback information for the same frame is plural, the device sequentially displays a plurality of pieces of feedback information based on the time each feedback information is input, wherein the calculated number of feedback information is preset. If more than the value, display the feedback information,
The calculation step is
identifying a feedback type of feedback information that matches each frame; and
Calculating the number of feedback information for each type of feedback for each frame; further comprising,
The device, characterized in that when the number of feedback information corresponding to a preset first feedback type is greater than or equal to a preset value, not displaying all feedback information matching the frame,
How to synchronize user feedback information of content streaming services. A method performed by an apparatus comprising:
Receiving and playing the video from a server (hereinafter, 'server') that provides a video streaming service to a plurality of devices;
displaying a reproduction screen of the reproduced video;
receiving, from the server, at least one piece of feedback information about the video input by at least one external device among the plurality of devices; and
Displaying the received feedback information on a playback screen of the video; including,
The feedback information is
The feedback information includes time information input by an external device and frame information of a video displayed on the external device at the input time,
The display step is
identifying frame information of a video displayed on the playback screen; and
Based on the frame information, displaying feedback information matching each frame on the playback screen when each frame is played back;
receiving first feedback information about the video from a user; and
Transmitting first feedback information about the video to the server, wherein the server calculates a difference in playing time of the video between a plurality of devices receiving the video, and based on the difference in the playing time, the first feedback A first feedback information transmission step of determining a device to transmit the information; further comprising,
The step of displaying on the playback screen comprises:
displaying the received first feedback information on the playback screen,
How to synchronize user feedback information of content streaming services. 3. The method of claim 2,
Calculating the number of feedback information matching each frame; further comprising,
The display step is
When the calculated number is equal to or greater than a preset value, displaying the feedback information;
How to synchronize user feedback information of content streaming services. 3. The method of claim 1 or 2,
The display step is
When feedback information matching the specific frame is received after displaying a specific frame of the video, the received feedback information is not displayed on the playback screen,
How to synchronize user feedback information of content streaming services. 4. The method of claim 3,
The calculation step is
identifying a feedback type of feedback information that matches each frame; and
Calculating the number of feedback information for each type of feedback for each frame; further comprising,
The display step is
When the number of feedback information corresponding to the preset first feedback type is greater than or equal to a preset value, the method further comprising the step of not displaying all feedback information matching the frame,
How to synchronize user feedback information of content streaming services. 3. The method of claim 2,
The server is
If the difference between the reproduction time is less than a preset time, characterized in that the first feedback information is not transmitted,
How to synchronize user feedback information of content streaming services. 3. The method of claim 1 or 2,
Calculating time difference information between the video and the device based on the first time information on the video and the second time information on the device; further comprising,
The display step is
setting reference frame information corresponding to a time point at which the feedback information is input among a plurality of frames of a video based on the time difference information; and
Displaying the feedback information at a point in time when a frame corresponding to the received reference frame information among a plurality of frames in the video currently being reproduced is reproduced,
The step of displaying the feedback information,
When the feedback information received from the server is less than a preset number, the feedback information is not displayed on the playback screen of the video,
How to synchronize user feedback information of content streaming services. a communication unit configured to receive the video from a server that provides a video streaming service to a plurality of devices;
a display unit for displaying a playback screen of the received video; and
Receive at least one piece of feedback information about the video input by at least one external device among the plurality of devices from the server through the communication unit, calculate the number of feedback information matching each frame, and A processor for displaying feedback information on a playback screen of a moving picture;
The processor is
Identifies frame information of a video displayed on the playback screen, and based on the frame information, displays feedback information matching each frame on the playback screen at the time each frame is played,
When the number of matching feedback information for the same frame is plural, a plurality of pieces of feedback information are sequentially displayed based on the input time of each feedback information, but when the calculated number of feedback information is equal to or greater than a preset value , display the feedback information,
When calculating the number of the feedback information, identifying a feedback type of feedback information matching each frame, calculating the number of feedback information for each frame and each feedback type,
When the number of feedback information corresponding to the preset first feedback type is greater than or equal to a preset value, the entire feedback information matching the frame is not displayed,
A device for synchronizing feedback information from users of content streaming services. a communication unit configured to receive the video from a server that provides a video streaming service to a plurality of devices;
a display unit for displaying a playback screen of the received video; and
a processor configured to receive at least one piece of feedback information on the video input by at least one external device among the plurality of devices from the server through the communication unit, and display the received feedback information on a playback screen of the video; including,
The processor is
Identifies frame information of a video displayed on the playback screen, and based on the frame information, displays feedback information matching each frame on the playback screen at the time each frame is played,
Receive first feedback information about the video from the user,
Transmitting first feedback information about the video to the server, calculating a difference in playback time of the video between a plurality of devices receiving the video, and transmitting the first feedback information based on the difference in the playback time Decide on a device to
displaying the received first feedback information on the playback screen,
A device for synchronizing feedback information from users of content streaming services. A computer program stored in a computer-readable recording medium in combination with a computer to execute the method for synchronizing user feedback information of claim 1 or claim 2 of the content streaming service.

Images