KR20220157033A - Video streaming service server for displaying advertisement information related to video and operating method thereof - Google Patents

Abstract

Disclosed are an image streaming service server and an operating method thereof. The present invention proposes the image streaming service server and the operating method for displaying advertisement information related to the image. By inducing users to purchase products related to content exposed in the image, it is possible to support companies to enjoy greater profits.

Description

Video streaming service server for displaying advertisement information related to video and its operating method

The present invention relates to a video streaming service server and its operating method for displaying advertisement information related to video.

Recently, as the corona era is prolonged, the ratio of watching video content indoors is increasing. Accordingly, the amount of real-time TV viewing has increased, and the number of VOD views has also increased.

In this way, as the ratio of watching video content indoors increases, the popularity of video streaming services is also increasing, and as a result, the need to support more various additional functions in video streaming services is also increasing.

In this regard, in the existing technology, when a viewer watches video content, advertisement information related to the video is not displayed, so it is not easy to induce viewers to purchase a product related to the content exposed in the video. .

If a technology is introduced that enables viewers to display advertising information related to video content when viewing video content, thereby inducing viewers to purchase products related to the content exposed in the video. will enjoy greater benefits.

Therefore, it is necessary to study a video streaming service technology that supports the display of advertisement information related to video.

The present invention proposes a video streaming service server and its operating method for displaying advertisement information related to video, thereby inducing users to purchase products related to content exposed in video, so that companies can enjoy greater profits. We want to help you make it happen.

According to an embodiment of the present invention, a video streaming service server for displaying advertisement information related to video includes a video data storage unit storing a plurality of different video data targeted for a streaming service, and advertisement selection. A word database in which different N (N is a natural number of 2 or more) words are stored, a plurality of preset different advertisement information, and a preset representation indicating the degree of association between each advertisement information and each of the N words. weight - the higher the weight, the higher the association between the advertisement information and the words, and the sum of the weights between each advertisement information and each of the N words is 1 - a weight storage unit in which correspondingly stored weights are stored, and a first user When the first video data is received from the terminal and an upload request command for the first video data is received from the first user terminal, the first video data is divided into a plurality of divisions by dividing the first video data by a first playback time interval set in advance. A segmentation unit that generates image data and checks information on the reproduction time of each of the plurality of divided image data in the entire reproduction time of the first image data, and converts the plurality of divided image data to a pre-built speech recognition model. A caption generating unit generating a plurality of subtitles corresponding to each of the plurality of divided image data by sequentially applying the input as an input and performing voice recognition, and the plurality of subtitles and the divided image corresponding to each of the plurality of subtitles. a management table generation unit for generating a management table in which information on reproduction time of data is recorded in association with each other; when the management table is generated, storing the first image data and the management table in association with each other in the image data storage unit After the first image data and the management table are associated and stored in the information storage processing unit and the image data storage unit, when a playback command for the first image data is received from a second user terminal, the image data storage unit Saved reminder management Referring to the table, an appearance frequency check unit for checking the frequency of occurrence of each of the N words in the plurality of subtitles, and the frequency of occurrence of each of the N words in the plurality of subtitles. By constructing a feature vector generator for generating an N-dimensional feature vector having as components, an N-dimensional vector having weights between each advertisement information and each of the N words for each of the plurality of pieces of advertisement information as a component, An advertisement vector generator that generates an advertisement vector corresponding to each of the plurality of advertisement pieces of advertisement information, configures an advertisement vector of each of the plurality of pieces of advertisement information as a row vector, and configures the feature vector as a column vector, and then configures the plurality of advertisements. A matching score calculation unit that calculates a matching score corresponding to each of the plurality of pieces of advertisement information by calculating a matrix multiplication between an advertisement vector corresponding to each piece of advertisement information and the feature vector for each piece of information; The first image data stored in the image data storage unit is transmitted to the second user terminal and an advertisement information selection unit that selects first advertisement information having the maximum matching score in a streaming manner, and a streaming transmission unit generating an advertisement display control command to display the first advertisement information on a preset display area of a screen on which video data is reproduced and additionally transmitting the command to the second user terminal.

In addition, according to an embodiment of the present invention, a method of operating a video streaming service server for displaying advertisement information related to a video includes maintaining a video data storage unit in which a plurality of different video data that are objects of a streaming service are stored. Step, maintaining a word database in which N (N is a natural number of 2 or more) preset words for advertisement selection are stored, a plurality of preset different advertisement information, each advertisement information and the N words A pre-set weight representing the degree of association between each of the above - the higher the weight, the higher the degree of association between the advertisement information and the word, and the sum of the weights between each advertisement information and each of the N words is 1 - is stored in correspondence Maintaining a weight storage unit having a weight storage unit configured to store the first image data when first image data is received from a first user terminal and an upload request command for the first image data is received from the first user terminal; Generating a plurality of divided image data by dividing them by a first reproduction time interval, and checking information about a reproduction time point of each of the plurality of divided image data in the entire reproduction time of the first image data; Generating a plurality of subtitles corresponding to each of the plurality of split image data by sequentially applying the divided image data as inputs to a pre-built speech recognition model and performing voice recognition; generating a management table in which information on reproducing time of segmented video data corresponding to each of subtitles is recorded in association with each other; when the management table is created, the first video data and the management table are stored in the video data storage unit. When a reproduction command for the first image data is received from a second user terminal after the first image data and the management table are stored in correspondence with each other in the image data storage unit, the image stored in the data storage Checking the frequency of occurrence of each of the N words in the plurality of subtitles by referring to the management table, determining the frequency of occurrence of each of the N words in the plurality of subtitles. generating an N-dimensional feature vector having as a component, constructing an N-dimensional vector having as a component a weight between each advertisement information and each of the N words for each of the plurality of pieces of advertisement information, Generating an advertisement vector corresponding to each piece of advertisement information, configuring an advertisement vector of each of the plurality of pieces of advertisement information as a row vector and configuring the feature vector as a column vector, and then for each of the plurality of pieces of advertisement information, calculating a matching score corresponding to each of the plurality of pieces of advertisement information by calculating a matrix multiplication between an advertisement vector corresponding to each piece of advertisement information and the feature vector; Selecting advertisement information and transmitting the first video data stored in the video data storage to the second user terminal in a streaming manner, wherein the first display area of the screen where the first video data is reproduced is displayed. generating an advertisement display control command for displaying the first advertisement information and additionally transmitting the command to the second user terminal.

The present invention proposes a video streaming service server and its operating method for displaying advertisement information related to video, thereby inducing users to purchase products related to content exposed in video, so that companies can enjoy greater profits. can support you to do so.

1 is a diagram showing the structure of a video streaming service server for displaying advertisement information related to video according to an embodiment of the present invention.
2 is a flowchart illustrating an operating method of a video streaming service server for displaying advertisement information related to video according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. This description is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. While describing each drawing, similar reference numerals have been used for similar components, and unless otherwise defined, all terms used in this specification, including technical or scientific terms, are common knowledge in the art to which the present invention belongs. has the same meaning as commonly understood by the person who has it.

In this document, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. In addition, in various embodiments of the present invention, each component, functional block, or means may be composed of one or more sub-components, and the electrical, electronic, and mechanical functions performed by each component are electronic It may be implemented with various known elements or mechanical elements such as circuits, integrated circuits, ASICs (Application Specific Integrated Circuits), and may be implemented separately or two or more may be integrated into one.

On the other hand, the blocks of the accompanying block diagram or the steps of the flowchart are computer program instructions that perform designated functions by being loaded into a processor or memory of a device capable of data processing, such as a general-purpose computer, a special purpose computer, a portable notebook computer, and a network computer. can be interpreted as meaning Since these computer program instructions may be stored in a memory included in a computer device or in a computer readable memory, the functions described in blocks of a block diagram or steps of a flowchart are produced as a product containing instruction means for performing them. It could be. Further, each block or each step may represent a module, segment or portion of code that includes one or more executable instructions for executing specified logical function(s). Also, it should be noted that in some alternative embodiments, functions mentioned in blocks or steps may be executed out of a predetermined order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

1 is a diagram showing the structure of a video streaming service server for displaying advertisement information related to video according to an embodiment of the present invention.

The video streaming service server 110 according to the present invention includes an image data storage unit 111, a word database 112, a weight storage unit 113, a division unit 114, a subtitle generator 115, and a management table generator. 116, information storage processing unit 117, appearance frequency check unit 118, feature vector generator 119, advertisement vector generator 120, matching score calculation unit 121, advertisement information selection unit 122, and It includes a streaming transmission unit 123.

The image data storage unit 111 stores a plurality of different image data that are objects of a streaming service.

The word database 112 stores N (N is a natural number equal to or greater than 2) different preset words for advertisement selection.

In relation to this, when N is 4, information may be stored in the word database 112 as shown in Table 1 below.

4 words intellectual property Patent Copyright registration

In the weight storage unit 113, a plurality of preset different pieces of advertisement information are stored in correspondence with a preset weight representing a degree of association between each piece of advertisement information and each of the N words.

Here, the higher the weight, the higher the association between the advertisement information and the word, and the sum of the weights between each advertisement information and each of the N words is 1.

In relation to this, when N is 4, information as shown in Table 2 below may be stored in the weight storage unit 113.

Multiple advertisement information Weight with each of the 4 words intellectual property Patent Copyright registration Advertising information 1 0.1 0.45 0 0.45 Advertising information 2 0.1 0 0.45 0.45 Advertising information 3 0.5 0.3 0.2 0 ... ... ... ... ...

When the first image data is received from the first user terminal 140 and an upload request command for the first image data is received from the first user terminal 140, the division unit 114 divides the first image data. A plurality of divided image data is generated by dividing the image at a predetermined first reproduction time interval, and information about a reproduction time point of each of the plurality of divided image data is checked in the total reproduction time of the first image data.

The caption generator 115 sequentially applies the plurality of divided image data as inputs to a pre-built voice recognition model to perform voice recognition, thereby generating a plurality of captions corresponding to each of the plurality of divided image data. .

The management table generation unit 116 generates a management table in which the plurality of subtitles and information about reproduction time of the divided video data corresponding to each of the plurality of subtitles are recorded in association with each other.

When the management table is generated, the information storage processing unit 117 stores the first image data and the management table in correspondence with each other in the image data storage unit 111 .

Hereinafter, operations of the division unit 114, the caption generation unit 115, the management table generation unit 116, and the information storage processing unit 117 will be described in detail by way of example.

First, while first video data is received from the first user terminal 140 to the video streaming service server 110, an upload request command for the first video data from the first user terminal 140 is sent to the video streaming service server ( 110), the divider 114 divides the first image data by a preset first reproduction time interval to generate a plurality of divided image data.

If the preset first playback time interval is '1 minute' and the total playback time of the first image data is '5 minutes', the division unit 114 converts the first image data to '1 minute'. By splitting at intervals, a plurality of split image data having a playback time of '1 minute' can be created as 'segmented image data 1, split image data 2, split image data 3, split image data 4, and split image data 5'. can

Thereafter, the segmentation unit 114 divides 'segmented image data 1, segmented image data 2, segmented image data 3, segmented image data 4, and segmented image data 5' in the total reproduction time of '5 minutes' of the first image data. Information about each playback time, 'playback time 1 (0 seconds), playback time 2 (1 minute), playback time 3 (2 minutes), playback time 4 (3 minutes), playback time 5 (4 minutes)' You can check.

Then, the subtitle generator 115 sequentially applies 'segmented image data 1, segmented image data 2, segmented image data 3, segmented image data 4, and segmented image data 5' as inputs to the pre-built speech recognition model. By performing voice recognition, a plurality of subtitles corresponding to 'segmented image data 1, segmented image data 2, segmented image data 3, segmented image data 4, and segmented image data 5' are selected as 'caption 1, subtitle 2, subtitle 3, It can be created with subtitle 4 and subtitle 5'.

Thereafter, the management table generation unit 116 generates divided images corresponding to 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5' and 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5', respectively. Data 'Split video data 1, Split video data 2, Split video data 3, Split video data 4, Split video data 5' playback time 'playback time 1 (0 sec), playback time 2 (1 minute), playback A management table in which information on the time point 3 (2 minutes), playback time point 4 (3 minutes), and playback time 5 (4 minutes) are recorded in correspondence with each other can be generated as shown in Table 3 below.

multiple subtitles Playback point of split video data subtitles 1 Playback time of split video data 1 1 (0 sec) subtitles 2 Playback time of split video data 2 2 (1 minute) subtitles 3 Playback time of split video data 3 3 (2 minutes) subtitles 4 Playback time of split video data 4 4 (3 minutes) subtitles 5 Playback time of split video data 5 5 (4 minutes)

In this way, when the management table is generated by the management table generation unit 116, the information storage processing unit 117 associates the first image data with the management table shown in Table 3 in the image data storage unit 111 to each other. can be saved

After the first image data and the management table are stored in correspondence with the image data storage unit 111, the appearance frequency check unit 118 receives a playback command for the first image data from the second user terminal 150. If received, the frequency of appearance of each of the N words in the plurality of subtitles is checked with reference to the management table stored in the image data storage unit 111 .

The feature vector generation unit 119 generates an N-dimensional feature vector having, as components, the frequency of occurrence of each of the N words in the plurality of subtitles.

The advertisement vector generator 120 configures an N-dimensional vector having weights between each advertisement information and each of the N words as components for each of the plurality of pieces of advertisement information, so that each of the plurality of pieces of advertisement information Create a corresponding advertising vector.

The matching score calculation unit 121 configures an advertisement vector of each of the plurality of pieces of advertisement information as a row vector and a feature vector as a column vector, and then, for each of the plurality of pieces of advertisement information, the advertisement corresponding to each piece of advertisement information. A matching score corresponding to each of the plurality of pieces of advertisement information is calculated by calculating a matrix multiplication between the vector and the feature vector.

The advertisement information selection unit 122 selects first advertisement information having the highest matching score from among the plurality of advertisement information items.

The streaming transmission unit 123 transmits the first video data stored in the video data storage unit 111 to the second user terminal 150 in a streaming manner, and displays the screen on which the first video data is reproduced. An advertisement display control command for displaying the first advertisement information in a preset display area is generated and additionally transmitted to the second user terminal 150 .

Hereinafter, operations of the appearance frequency check unit 118, feature vector generator 119, advertisement vector generator 120, matching score calculator 121, advertisement information selector 122, and streaming transmission unit 123 will be described. For example, it will be described in detail.

For example, as in the above example, after the first image data and the management table shown in Table 3 are stored in correspondence with the image data storage unit 111, the first image data is stored from the second user terminal 150. When a play command for is received by the video streaming service server 110, the appearance frequency check unit 118 stores the management table as shown in Table 3 and the word database as shown in Table 1 stored in the video data storage unit 111 ( 112), in 'caption 1, subtitle 2, subtitle 3, subtitle 4, subtitle 5', it is possible to check the appearance frequency of each 'intellectual property right, patent, copyright, registration'.

At this time, the appearance frequency of each of 'intellectual property rights, patents, copyrights, and registrations' in 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5' by the appearance frequency check unit 118 is '2 (times), Let's assume that it is confirmed as '10 (times), 1 (times), 8 (times)'.

At this time, the feature vector generator 119 generates a 4-dimensional image having as a component the frequency of each occurrence of 'intellectual property rights, patents, copyrights, and registrations' in 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5'. '[2 10 1 8]' can be created as a feature vector.

Then, the advertisement vector generation unit 120 refers to the weight storage unit 113 as shown in Table 2 above, and for each 'advertisement information 1, advertisement information 2, advertisement information 3, ...', each advertisement information and 'intellectual property right, patent, copyright, registration' by constructing a 4-dimensional vector having weights between each of 'advertisement information 1, advertisement information 2, advertisement information 3, ...', respectively, to generate advertising vectors can do.

In this regard, for 'advertisement information 1', since the weights between 'advertisement information 1' and 'intellectual property, patent, copyright, registration' are '0.1, 0.45, 0, 0.45', '[0.1 0.45 0 0.45]' may be generated as an advertisement vector corresponding to information 1'. In this way, the advertisement vector generation unit 120 generates '[0.1 0.45 0 0.45], [0.1 0 0.45 0.45] as advertisement vectors corresponding to 'advertisement information 1, advertisement information 2, advertisement information 3, ...' respectively. , [0.5 0.3 0.2 0], ...'.

'과 같이 구성할 수 있다.In this way, when advertisement vectors corresponding to each of 'advertisement information 1, advertisement information 2, advertisement information 3, ...' are generated, the matching score calculation unit 121 calculates 'advertisement information 1, advertisement information 2, advertisement information 3, . '[0.1 0.45 0 0.45], [0.1 0 0.45 0.45], [0.5 0.3 0.2 0], ...', which are ad vectors corresponding to each of ..', as row vectors, '[0.1 0.45 0 0.45], [0.1 0 0.45 0.45], [0.5 0.3 0.2 0], ...', and the feature vector '[2 10 1 8]' as a column vector '

' can be configured.

After that, the matching score calculation unit 121 calculates the matrix multiplication between the feature vector and the advertisement vector corresponding to each advertisement information for each of 'advertisement information 1, advertisement information 2, advertisement information 3, ...', Matching scores corresponding to advertisement information 1, advertisement information 2, advertisement information 3, ...' may be calculated.

'이므로, 매칭 점수 연산부(121)는 '광고 정보 1'에 대응되는 매칭 점수로 '(0.2+4.5+0+3.6=8.3)(점)'을 생성할 수 있다. 이러한 방식으로, 매칭 점수 연산부(121)는 '광고 정보 1, 광고 정보 2, 광고 정보 3, ...' 각각에 대응되는 매칭 점수로 '8.3(점), 4.25(점), 4.2(점) ...'을 생성할 수 있다.In this regard, for 'Ad information 1', the advertisement vector corresponding to 'Ad information 1' is '[0.1 0.45 0 0.45]', and the feature vector is '

', the matching score calculation unit 121 may generate '(0.2 + 4.5 + 0 + 3.6 = 8.3) (points)' as a matching score corresponding to 'advertisement information 1'. In this way, the matching score calculating unit 121 sets '8.3 (points), 4.25 (points), 4.2 (points) as matching scores corresponding to 'advertisement information 1, advertisement information 2, advertisement information 3, ...' respectively. ...' can be created.

In this way, by the matching score calculation unit 121, '8.3 (points), 4.25 (points), 4.2 (points) are obtained as matching scores corresponding to 'advertisement information 1, advertisement information 2, advertisement information 3, ...' respectively. When the operation of '...' is completed, the advertisement information selection unit 122 may select the first advertisement information having the maximum matching score among 'advertisement information 1, advertisement information 2, advertisement information 3, ...'. .

If it is assumed that the matching scores are higher in the order of '8.3 (points), 4.25 (points), 4.2 (points) ...' among the matching scores, the advertisement information selector 122 selects 'advertisement information 1, advertisement information 2, advertisement information 3, ...', 'advertisement information 1' can be selected.

Then, the streaming transmission unit 123 transmits the first video data stored in the video data storage unit 111 to the second user terminal 150 in a streaming manner, and the first video data is reproduced. An advertisement display control command for displaying 'advertisement information 1' in a preset display area of the screen may be generated and additionally transmitted to the second user terminal 150 .

Through this, the user of the second user terminal 150 can not only enjoy the first video data in a streaming manner, but also check 'advertisement information 1' displayed on the display area of the screen, so that when purchasing a related product later can refer

At this time, according to an embodiment of the present invention, the caption generator 115, for each of the plurality of divided image data, uses a frequency band other than the first frequency band preset as a frequency band corresponding to human voice. After all sound components have been removed, each segmented image data from which all sound components in frequency bands other than the first frequency band have been removed is sequentially applied as an input to the voice recognition model to perform voice recognition, thereby generating the plurality of subtitles. can create

For example, assume that the first frequency band preset as a frequency band corresponding to human voice is '50 Hz to 4000 Hz'.

At this time, according to the above-described example, the caption generator 115 responds to human voice for 'segmented video data 1, segmented video data 2, segmented video data 3, segmented video data 4, and segmented video data 5' respectively. It is possible to remove all sound components of frequency bands other than the preset first frequency band '50Hz ~ 4000Hz'.

Thereafter, the caption generator 115 sequentially applies each segmented image data from which all sound components of the frequency band other than the first frequency band '50 Hz to 4000 Hz' have been removed to the speech recognition model as an input for speech recognition. , it is possible to generate the plurality of subtitles as 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5'.

According to one embodiment of the present invention, the video streaming service server 110 may further include a confirmation unit 124, an extraction unit 125, a transmission unit 126 and a streaming adjustment unit 127.

The confirmation unit 124 receives a first keyword from the second user terminal 150 while the first video data stored in the video data storage unit 111 is transmitted in a streaming manner, and the second user terminal ( 150), when a search command instructing to search for a playback time matching the first keyword in the first image data is received, the management table stored in the image data storage unit 111 is referred to, It is checked whether a subtitle including the first keyword exists among a plurality of subtitles.

The extraction unit 125 extracts a subtitle including the first keyword from among the plurality of subtitles, and when it is determined that at least one first subtitle exists, a subtitle corresponding to the at least one first subtitle is obtained from the management table. Information about at least one first reproduction time point is extracted.

The transmission unit 126 designates the information on the at least one first playback time point as a search result matching the first keyword and transmits the information to the second user terminal 150 .

After the information on the at least one first playback point is designated as a search result matching the first keyword and transmitted to the second user terminal 150, the streaming control unit 127 receives information about the at least one first playback time from the second user terminal 150. When a selection reproduction command for a second reproduction time point, which is any one of the at least one first reproduction time point, is received, the first image stored in the image data storage unit 111 for the second user terminal 150 Data is transmitted in a streaming method from the second reproduction time point.

Hereinafter, operations of the confirmation unit 124, the extraction unit 125, the transmission unit 126, and the streaming adjustment unit 127 will be described in detail, for example.

First, while the first video data stored in the video data storage unit 111 is transmitted in a streaming manner by the streaming transmission unit 123, a first keyword from the second user terminal 150 is sent to the video streaming service server. When received at (110), it is determined that the video streaming service server 110 has received a search command from the second user terminal 150 instructing to search for a reproduction time point matching the first keyword in the first video data. Let's assume.

Then, the confirmation unit 124 refers to the management table such as Table 3 stored in the image data storage unit 111, and selects the first one of 'subtitle 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5'. It is possible to check whether a subtitle including a keyword exists.

If, through the confirmation unit 124, 'caption 1, subtitle 3, and subtitle 5' exist as subtitles including the first keyword among 'caption 1, subtitle 2, subtitle 3, subtitle 4, and subtitle 5' If confirmed, the extraction unit 125 selects playback time points corresponding to 'caption 1, subtitle 3, and subtitle 5' from the management table shown in Table 3, 'playback time 1 (0 sec), playback time 3 (2 minutes) ), and information about playback time 5 (4 minutes)' can be extracted.

Then, the transmission unit 126 designates information on 'playback time 1 (0 second), play time 3 (2 minutes), and play time 5 (4 minutes)' as a search result matching the first keyword, It can be transmitted to the second user terminal 150 .

In this way, information on 'playback time 1 (0 sec), playback time 3 (2 minutes), and playback time 5 (4 minutes)' is designated as a search result matching the first keyword by the transmission unit 126 After being transmitted to the second user terminal 150, reproduction of any one of 'playback time 1 (0 second), play time 3 (2 minutes), and play time 5 (4 minutes)' from the second user terminal 150 When the video streaming service server 110 receives a selective playback command for the 'playback time point 5 (4 minutes)', the streaming control unit 127 sends the video data storage unit 111 to the second user terminal 150. The first image data stored in ) may be transmitted in a streaming method from 'playback time 5 (4 minutes)'.

Through this, the user of the second user terminal 150 can enjoy the first video data in a streaming manner from 'playback time 5 (4 minutes)'.

At this time, according to an embodiment of the present invention, the transmission unit 126 designates the information on the at least one first playback time as a search result matching the first keyword and transmits the information to the second user terminal 150. However, a guide message notifying that information on a playback time corresponding to the second subtitle by checking a second subtitle including the first keyword among the at least one first subtitle is information on a main playback time. may be additionally transmitted to the second user terminal 150 .

For example, according to the above example, the transmitter 126 transmits information about 'playback time 1 (0 second), play time 3 (2 minutes), and play time 5 (4 minutes)' to match the first keyword. It is specified as a search result and transmitted to the second user terminal 150, and among 'caption 1, subtitle 3, and subtitle 5', a second subtitle containing the first keyword the most may be checked.

If, through the transmission unit 126, the second caption including the most of the first keyword among 'caption 1, subtitle 3, and subtitle 5' is identified as 'caption 3', the transmitter 126 transmits 'caption 3'. A guide message notifying that the information on 'playback time 3 (2 minutes)', which is the playback time corresponding to subtitle 3', is information on the main playback time, may be additionally transmitted to the second user terminal 150 .

According to one embodiment of the present invention, the video streaming service server 110 includes a translated subtitle generator 128, a translated subtitle checker 129, a playback time extraction unit 130, a playback time transmission unit 131, and translation A streaming adjustment unit 132 may be further included.

The translated subtitle generator 128 receives a second keyword composed of a first foreign language from the second user terminal 150 while the first video data stored in the video data storage unit 111 is transmitted in a streaming manner. When a search command instructing to search for a reproduction point matching the second keyword in the first image data is received from the second user terminal 150, the image stored in the image data storage unit 111 is displayed. Referring to the management table, the plurality of translated subtitles are generated by translating the plurality of subtitles into the first foreign language through a pre-built translation engine.

The translated subtitle checking unit 129 checks whether a translated subtitle including the second keyword exists among the plurality of translated subtitles.

When it is confirmed that at least one first translated subtitle exists as a translated subtitle including the second keyword among the plurality of translated subtitles, the playback time extraction unit 130 extracts the at least one second translated subtitle from the management table. Information on at least one third reproduction time point corresponding to 1 translation subtitle is extracted.

The reproduction time transmission unit 131 designates information on the at least one third reproduction time as a search result matching the second keyword and transmits the information to the second user terminal 150 .

After the translation streaming adjustment unit 132 transmits the information on the at least one third reproduction time point to the second user terminal 150 after being designated as a search result matching the second keyword, the second user terminal 150 When a selection playback command for a fourth playback time point, which is any one of the at least one third playback time point, is received from, the first playback time stored in the image data storage unit 111 for the second user terminal 150 is received. Video data is transmitted in a streaming method from the fourth reproduction time point.

Hereinafter, operations of the translated caption generator 128, the translated caption checker 129, the playback time extraction unit 130, the playback time transmission unit 131, and the translation streaming control unit 132 will be described in detail, for example. I'm going to do it.

For example, as in the above example, it is assumed that the first video data and the management table shown in Table 3 are stored in correspondence with the video data storage unit 111, and the first video data is dubbed with 'Korean' voice. Assume that a plurality of subtitles recorded in the management table as shown in Table 3 are subtitles composed of 'Korean' and that the first foreign language is 'English' since they are image data.

At this time, while the first video data stored in the video data storage unit 111 is being transmitted in a streaming manner by the streaming transmission unit 123, the second keyword consisting of 'English' from the second user terminal 150 When is received by the video streaming service server 110, a search command instructing to search for a reproduction time matching the second keyword in the first video data from the second user terminal 150 is sent to the video streaming service server ( 110), the translated subtitle generator 128 refers to the management table such as Table 3 stored in the video data storage unit 111, and 'subtitle 1, subtitle 2, subtitle 3, subtitle 4, subtitle By translating '5' into 'English' through a pre-built translation engine, a plurality of translated subtitles can be generated as 'translation subtitle 1, translation subtitle 2, translation subtitle 3, translation subtitle 4, and translation subtitle 5'.

Then, the translated subtitle checking unit 129 may check whether a translated subtitle including the second keyword exists among 'translated subtitle 1, translated subtitle 2, translated subtitle 3, translated subtitle 4, and translated subtitle 5'. have.

If, through the translated subtitle checking unit 129, 'translated subtitles 1, translated subtitles 2, translated subtitles 3, translated subtitles 4, and translated subtitles 5' including the second keyword, 'translated subtitles 2, When it is confirmed that translated subtitle 4 exists, the playback time extraction unit 130 selects 'playback time 2 (1 minute)' corresponding to 'translated subtitle 2 and translated subtitle 4' from the management table shown in Table 3 above. ), information on playback time 4 (3 minutes)' can be extracted.

After that, the playback time transmission unit 131 designates information on 'playback time 2 (1 minute) and playback time 4 (3 minutes)' as a search result matching the second keyword, and the second user terminal 150 ) can be transmitted.

In this way, information on 'playback time 2 (1 minute) and playback time 4 (3 minutes)' is designated as a search result matching the second keyword by the playback time transmission unit 131, and the second user terminal 150 ), selection of 'playback time 2 (1 minute)', which is any one of 'playback time 2 (1 minute) and play time 4 (3 minutes)', from the second user terminal 150 When a playback command is received by the video streaming service server 110, the translation streaming controller 132 'plays' the first video data stored in the video data storage unit 111 to the second user terminal 150. It can be transmitted in a streaming method from 'time point 2 (1 minute)'.

Through this, the user of the second user terminal 150 can enjoy the first image data in a streaming manner from 'playback time 2 (1 minute)'.

2 is a flowchart illustrating an operating method of a video streaming service server for displaying advertisement information related to video according to an embodiment of the present invention.

In step S201, a video data storage unit is maintained in which a plurality of different video data objects of a streaming service are stored.

In step S202, a word database in which N (N is a natural number equal to or greater than 2) different preset words for advertisement selection are stored is maintained.

In step S203, a plurality of preset different pieces of advertisement information and a preset weight indicating a degree of association between each piece of advertisement information and each of the N words (the higher the weight, the higher the degree of association between the advertisement information and the words). and maintains a weight storage unit in which the sum of weights between each advertisement information and each of the N words is 1) correspondingly and stored.

In step S204, when the first video data is received from the first user terminal and an upload request command for the first video data is received from the first user terminal, the first video data is stored for a preset first playback time. A plurality of divided image data is generated by dividing the image data at intervals, and information about a reproduction time point of each of the plurality of divided image data is checked in the entire reproduction time of the first image data.

In step S205, a plurality of subtitles corresponding to each of the plurality of divided image data are generated by sequentially applying the plurality of divided image data as inputs to a pre-built voice recognition model to perform voice recognition.

In step S206, a management table is created in which the plurality of subtitles and information about reproduction time of the divided video data corresponding to each of the plurality of subtitles are recorded in correspondence with each other.

In step S207, when the management table is generated, the first image data and the management table are stored in correspondence with each other in the image data storage unit.

In step S208, if a reproduction command for the first image data is received from a second user terminal after the first image data and the management table are stored in correspondence with the image data storage unit, the image data storage unit With reference to the management table stored in , the number of occurrence frequencies of each of the N words in the plurality of subtitles is checked.

In step S209, an N-dimensional feature vector having as a component an appearance frequency of each of the N words in the plurality of subtitles is generated.

In step S210, for each of the plurality of pieces of advertisement information, an N-dimensional vector having weights between each piece of advertisement information and each of the N words as a component is constructed, so that advertisements corresponding to each of the plurality of pieces of advertisement information are constructed. create a vector

In step S211, an advertisement vector of each of the plurality of pieces of advertisement information is configured as a row vector, and the feature vector is configured as a column vector. A matching score corresponding to each of the plurality of pieces of advertisement information is calculated by calculating matrix multiplication between the feature vectors.

In step S212, the first advertisement information having the highest matching score is selected from among the plurality of advertisement information items.

In step S213, the first video data stored in the video data storage unit is transmitted to the second user terminal in a streaming manner, and the first video data is displayed on a preset display area of a screen where the first video data is reproduced. An advertisement display control command for displaying the first advertisement information is generated and additionally transmitted to the second user terminal.

At this time, according to one embodiment of the present invention, in step S205, for each of the plurality of divided image data, sound components of a frequency band other than the first frequency band preset to be a frequency band corresponding to human voice After all of the subtitles are removed, the plurality of subtitles are generated by sequentially applying each segmented image data from which all of the sound components of the frequency band other than the first frequency band have been removed as inputs to the speech recognition model to perform speech recognition. can

Further, according to an embodiment of the present invention, the operating method of the video streaming service server may include a first video data from the second user terminal while the first video data stored in the video data storage unit is being transmitted in a streaming manner. When a search command instructing to search for a reproduction point matching the first keyword in the first image data is received from the second user terminal while a keyword is received, the management stored in the image data storage unit is received. Checking whether a subtitle including the first keyword exists among the plurality of subtitles by referring to a table; selecting at least one subtitle including the first keyword among the plurality of subtitles If it is confirmed that 1 subtitle exists, extracting information on at least one first playback time point corresponding to the at least one first subtitle from the management table; designating as a search result matching the first keyword and transmitting the information to the second user terminal, and designating the information on the at least one first playback time as a search result matching the first keyword and transmitting the information to the second user terminal. After being transmitted to the terminal, when a selection playback command for a second playback time point, which is any one of the at least one first playback time point, is received from the second user terminal, for the second user terminal, the image data storage unit The method may further include transmitting the first image data stored in the streaming method from the second reproduction time point.

At this time, according to an embodiment of the present invention, the transmitting to the second user terminal designates the information on the at least one first reproduction time point as a search result matching the first keyword, and the second user terminal to inform that the information on the playback time corresponding to the second subtitle is the information on the main playback time by checking the second subtitle containing the first keyword the most among the at least one first subtitle. A guide message may be additionally transmitted to the second user terminal.

Further, according to an embodiment of the present invention, the operating method of the video streaming service server may include a first video data from the second user terminal while the first video data stored in the video data storage unit is being transmitted in a streaming manner. When a second keyword composed of a foreign language is received and a search command instructing to search for a reproduction point matching the second keyword in the first image data is received from the second user terminal, the image data storage unit generating a plurality of translated subtitles by translating the plurality of subtitles into the first foreign language through a pre-built translation engine with reference to the stored management table; and the second keyword among the plurality of translated subtitles. checking whether a translated subtitle including a subtitle exists; if it is confirmed that at least one first translated subtitle exists as a translated subtitle including the second keyword among the plurality of translated subtitles, the management extracting information on at least one third playback point of view corresponding to the at least one first translated subtitle from a table; converting the information on the at least one third playback point of view into a search result matching the second keyword; After the step of designating and transmitting to the second user terminal and the information on the at least one third playback time being designated as a search result matching the second keyword and transmitted to the second user terminal, the second user terminal When a selection playback command for a fourth playback time point, which is any one of the at least one third playback time point, is received from a terminal, the first image data stored in the image data storage unit is transmitted to the second user terminal. The method may further include transmitting from the fourth playback point in a streaming manner.

In the above, the operating method of the video streaming service server according to an embodiment of the present invention has been described with reference to FIG. 2 . Here, since the operation method of the video streaming service server according to an embodiment of the present invention may correspond to the configuration of the operation of the video streaming service server 110 described with reference to FIG. 1, a detailed description thereof is omitted. do it with

A method of operating an image streaming service server for displaying advertisement information related to an image according to an embodiment of the present invention may be implemented as a computer program stored in a storage medium for execution through a combination with a computer.

In addition, the operating method of the video streaming service server for displaying advertisement information related to video according to an embodiment of the present invention is implemented in the form of computer program instructions for execution through combination with a computer and recorded on a computer readable medium. can The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and it will be said that not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the present invention. .

110: Video streaming service server for displaying advertisement information related to video
111: image data storage unit 112: word database
113: weight storage unit 114: division unit
115: subtitle generator 116: management table generator
117: Information storage processing unit 118: Appearance frequency confirmation unit
119: feature vector generator 120: advertisement vector generator
121: Matching score calculation unit 122: Advertisement information selection unit
123: streaming transmission unit 124: confirmation unit
125: extraction unit 126: transmission unit
127: Streaming adjustment unit 128: Translation subtitle generation unit
129: translation subtitle check unit 130: playback time extraction unit
131: playback time transmission unit 132: translation streaming control unit
140: first user terminal
150: second user terminal

Claims (12)

In the video streaming service server for displaying advertisement information related to video,
an image data storage unit storing a plurality of different image data objects of a streaming service;
a word database in which N (N is a natural number equal to or greater than 2) preset words for selecting advertisements are stored;
A preset weight representing a plurality of preset different pieces of advertisement information and a degree of association between each piece of advertisement information and each of the N words - the higher the weight, the higher the degree of association between the advertisement information and words, a weight storage unit in which the sum of weights between each of the N words and − is stored in correspondence with each other;
When first video data is received from the first user terminal and an upload request command for the first video data is received from the first user terminal, the first video data is divided into a first playback time interval set in advance, a segmentation unit that generates a plurality of divided image data and checks information about a reproduction point of each of the plurality of divided image data in the entire reproduction time of the first image data;
a caption generating unit generating a plurality of subtitles corresponding to each of the plurality of divided image data by sequentially applying the plurality of divided image data as inputs to a pre-built voice recognition model and performing voice recognition;
a management table generating unit generating a management table in which the plurality of subtitles and information on reproducing time of divided image data corresponding to each of the plurality of subtitles are recorded in association with each other;
an information storage processing unit configured to store the first image data and the management table in correspondence with each other in the image data storage unit when the management table is generated;
After the first image data and the management table are stored in correspondence with the image data storage unit, when a reproduction command for the first image data is received from a second user terminal, the image data storage unit stores the first image data stored in the image data storage unit. an appearance frequency check unit checking an appearance frequency count of each of the N words in the plurality of subtitles with reference to a management table;
a feature vector generating unit generating an N-dimensional feature vector having as components an appearance frequency of each of the N words in the plurality of subtitles;
An advertisement generating an advertisement vector corresponding to each of the plurality of pieces of advertisement information by constructing an N-dimensional vector having weights between each piece of advertisement information and each of the N words as components for each of the plurality of pieces of advertisement information. vector generator;
After constructing an advertisement vector of each of the plurality of pieces of advertisement information as a row vector and a feature vector as a column vector, for each of the plurality of pieces of advertisement information, a matrix between an advertisement vector corresponding to each piece of advertisement information and the feature vector. a matching score calculation unit calculating a matching score corresponding to each of the plurality of pieces of advertisement information by calculating a product;
an advertisement information selector selecting first advertisement information having the highest matching score from among the plurality of advertisement information items; and
The first video data stored in the video data storage unit is transmitted to the second user terminal in a streaming manner, and the first advertisement information is displayed in a preset display area of a screen on which the first video data is reproduced. A streaming transmission unit generating an advertisement display control command to be displayed and additionally transmitting it to the second user terminal.
A video streaming service server including a. According to claim 1,
The subtitle generator
For each of the plurality of split image data, after removing all sound components in frequency bands other than the first frequency band preset as a frequency band corresponding to human voice, sound in frequency bands other than the first frequency band The video streaming service server according to claim 1 , wherein the video streaming service server generates the plurality of subtitles by sequentially applying the segmented image data from which all components have been removed as inputs to the speech recognition model to perform speech recognition. According to claim 1,
While the first image data stored in the image data storage unit is being transmitted in a streaming manner, when a first keyword is received from the second user terminal, the first image data is stored in the first image data from the second user terminal. 1 When a search command instructing to search for a reproduction point matching a keyword is received, the subtitle including the first keyword among the plurality of subtitles is referred to the management table stored in the image data storage unit. a check unit for checking whether or not exists;
If it is determined that at least one first subtitle among the plurality of subtitles including the first keyword exists, at least one first reproduction corresponding to the at least one first subtitle from the management table. an extraction unit for extracting information about a viewpoint;
a transmission unit configured to designate information on the at least one first playback point as a search result matching the first keyword and transmit the information to the second user terminal; and
After the information on the at least one first reproduction time point is designated as a search result matching the first keyword and transmitted to the second user terminal, any one of the at least one first reproduction time point from the second user terminal Streaming to transmit the first image data stored in the image data storage unit in a streaming manner to the second user terminal when a selection reproduction command for a second reproduction time point is received. coordination department
A video streaming service server further comprising a. According to claim 3,
the transmission unit
Designating the information on the at least one first reproduction time point as a search result matching the first keyword and transmitting the information to the second user terminal, including the first keyword among the at least one first subtitle; The video streaming service server of claim 1 , further transmitting to the second user terminal a guide message informing that information on a playback time corresponding to the second subtitle is information on a main playback time by checking a second subtitle that exists. According to claim 1,
While the first image data stored in the image data storage unit is being transmitted in a streaming manner, when a second keyword composed of a first foreign language is received from the second user terminal, the first keyword is received from the second user terminal. When a search command for instructing to search for a playback point matching the second keyword in video data is received, the plurality of subtitles are stored in a pre-built translation engine by referring to the management table stored in the video data storage unit. a translation subtitle generation unit that generates a plurality of translation subtitles by translating the first foreign language through;
a translated caption checking unit that checks whether or not a translated caption including the second keyword exists among the plurality of translated captions;
If it is confirmed that there is at least one first translated subtitle as a translated subtitle including the second keyword among the plurality of translated subtitles, at least one translation subtitle corresponding to the at least one first translated subtitle from the management table. a reproduction time extraction unit extracting information on a third reproduction time of the;
a reproduction time transmitting unit configured to designate information on the at least one third reproduction time as a search result matching the second keyword and transmit the information to the second user terminal; and
After the information on the at least one third reproduction time point is designated as a search result matching the second keyword and transmitted to the second user terminal, any one of the at least one third reproduction time point from the second user terminal Translation of transmitting the first video data stored in the video data storage unit in a streaming manner to the second user terminal when a selection reproduction command for a fourth reproduction time point is received Stream Coordination Department
A video streaming service server further comprising a. A method of operating a video streaming service server for displaying advertisement information related to video,
maintaining a video data storage unit storing a plurality of different video data objects of a streaming service;
maintaining a word database in which N (N is a natural number of 2 or more) preset different words for advertisement selection are stored;
A preset weight representing a plurality of preset different pieces of advertisement information and a degree of association between each piece of advertisement information and each of the N words - the higher the weight, the higher the degree of association between the advertisement information and words, maintaining a weight storage unit in which the sum of weights between each of the N words and − is stored in correspondence with each other;
When first video data is received from the first user terminal and an upload request command for the first video data is received from the first user terminal, the first video data is divided into a first playback time interval set in advance, generating a plurality of split image data and checking information about a playback time point of each of the plurality of split image data within an entire playback time of the first image data;
generating a plurality of subtitles corresponding to each of the plurality of divided image data by sequentially applying the plurality of divided image data as inputs to a pre-built voice recognition model and performing voice recognition;
generating a management table in which the plurality of subtitles and information on reproducing time of divided image data corresponding to each of the plurality of subtitles are recorded in correspondence with each other;
if the management table is generated, storing the first image data and the management table in correspondence with each other in the image data storage unit;
After the first image data and the management table are stored in correspondence with the image data storage unit, when a playback command for the first image data is received from the second user terminal, the image data storage unit stores the first image data stored in the image data storage unit. checking an appearance frequency of each of the N words in the plurality of subtitles by referring to a management table;
generating an N-dimensional feature vector having as a component an appearance frequency of each of the N words in the plurality of subtitles;
Generating an advertisement vector corresponding to each of the plurality of pieces of advertisement information by constructing an N-dimensional vector having weights between each advertisement information and each of the N words for each of the plurality of pieces of advertisement information as components. ;
After constructing an advertisement vector of each of the plurality of pieces of advertisement information as a row vector and constructing the feature vector as a column vector, for each of the plurality of pieces of advertisement information, a matrix between an advertisement vector corresponding to each piece of advertisement information and the feature vector. calculating a matching score corresponding to each of the plurality of pieces of advertisement information by calculating a product;
selecting first advertisement information having the highest matching score from among the plurality of advertisement information items; and
The first video data stored in the video data storage unit is transmitted to the second user terminal in a streaming manner, and the first advertisement information is displayed in a preset display area of a screen on which the first video data is reproduced. Generating an advertisement display control command to be displayed and additionally transmitting it to the second user terminal
Method of operating a video streaming service server comprising a. According to claim 6,
The step of generating the plurality of subtitles is
For each of the plurality of split image data, after removing all sound components in frequency bands other than the first frequency band preset as a frequency band corresponding to human voice, sound in frequency bands other than the first frequency band A method of operating a video streaming service server, characterized in that the plurality of subtitles are generated by sequentially applying each segmented image data from which all components have been removed as inputs to the speech recognition model to perform speech recognition. According to claim 6,
While the first image data stored in the image data storage unit is being transmitted in a streaming manner, when a first keyword is received from the second user terminal, the first image data is stored in the first image data from the second user terminal. 1 When a search command instructing to search for a reproduction point matching a keyword is received, the subtitle including the first keyword among the plurality of subtitles is referred to the management table stored in the image data storage unit. Checking whether or not exists;
If it is determined that at least one first subtitle among the plurality of subtitles including the first keyword exists, at least one first reproduction corresponding to the at least one first subtitle from the management table. extracting information about a viewpoint;
designating information on the at least one first playback point as a search result matching the first keyword and transmitting the information to the second user terminal; and
After the information on the at least one first reproduction time point is designated as a search result matching the first keyword and transmitted to the second user terminal, any one of the at least one first reproduction time point from the second user terminal transmitting, to the second user terminal, the first image data stored in the image data storage unit in a streaming manner from the second playback point in time, when a selective playback command for a second playback point in time is received;
Method of operating a video streaming service server further comprising a. According to claim 8,
The step of transmitting to the second user terminal is
Designating the information on the at least one first reproduction time point as a search result matching the first keyword and transmitting the information to the second user terminal, including the first keyword among the at least one first subtitle; The video streaming service server of claim 1 , wherein a guide message notifying that information on a playback time corresponding to the second subtitle is information on a main playback time is additionally transmitted to the second user terminal by checking a second subtitle that exists. how it works. According to claim 6,
While the first image data stored in the image data storage unit is being transmitted in a streaming manner, when a second keyword composed of a first foreign language is received from the second user terminal, the first keyword is received from the second user terminal. When a search command for instructing to search for a playback point matching the second keyword in video data is received, the plurality of subtitles are stored in a pre-built translation engine by referring to the management table stored in the video data storage unit. generating a plurality of translated subtitles by translating the first foreign language through;
checking whether a translated subtitle including the second keyword exists among the plurality of translated subtitles;
If it is confirmed that there is at least one first translated subtitle as a translated subtitle including the second keyword among the plurality of translated subtitles, at least one translation subtitle corresponding to the at least one first translated subtitle from the management table. extracting information about a third reproduction point of time;
designating information on the at least one third playback point as a search result matching the second keyword and transmitting the information to the second user terminal; and
After the information on the at least one third reproduction time point is designated as a search result matching the second keyword and transmitted to the second user terminal, any one of the at least one third reproduction time point from the second user terminal transmitting, to the second user terminal, the first image data stored in the image data storage unit in a streaming manner from the fourth playback time point, when a selective playback command for a fourth playback point in time is received;
Method of operating a video streaming service server further comprising a. A computer-readable recording medium recording a computer program for executing the method of any one of claims 6 to 10 through a combination with a computer. A computer program stored in a storage medium for executing the method of any one of claims 6 to 10 through a combination with a computer.

Images