KR20220168676A - Live streaming server which can insert the digital code, to be used for looking up contents data, in the realtime video, and the operating method thereof - Google Patents

Abstract

A live streaming server which can insert a digital code, to be used for looking up content data, in a real-time video, and an operating method thereof are disclosed. The present invention suggests a live streaming server capable of inserting a digital code used for looking up content data in a real-time video and an operating method thereof. Thus, the usability of the live streaming service can be increased.

Description

Live streaming server capable of inserting digital code used to look up content data in real-time video and its operation method THE OPERATING METHOD THEREOF}

The present invention relates to a live streaming server capable of inserting a digital code used to search content data into a real-time video and a method of operating the same.

Recently, as electronic terminals such as computers, smart phones, tablet PCs, etc. are widely spread, various types of video contents that can be viewed using such electronic terminals are appearing.

In addition, as the global epidemic of COVID-19 prolongs, the number of viewers who spend their leisure time watching various videos using electronic terminals instead of outdoor activities in a situation where contact and movement between people is restricted increases. there is.

In particular, recently, viewers who watch real-time video through a live streaming service that supports viewers to watch real-time video by transmitting real-time video data received from the video transmission terminal of the video sender to the video playback terminal of the viewer It is increasing.

On the other hand, since the information that the video sender can provide to viewers through real-time video is limited, there is a case where the video sender wants to provide additional content to supplement the contents of the real-time video to viewers while the real-time video is being played. many.

If, in a live streaming service, a technology capable of inserting a digital code used to search content data into a real-time video is introduced, the video transmitter can provide more diverse content to viewers, and viewers can use the digital code , the usability of the live streaming service can be increased in that the content provided by the video sender can be easily viewed.

Therefore, in order to increase the usability of the live streaming service, research on a technology capable of inserting a digital code used to retrieve content data into a real-time video is required.

The present invention provides a live streaming server capable of inserting a digital code used to search content data into a real-time video and a method of operating the same, to support increased usability of a live streaming service.

According to an embodiment of the present invention, a live streaming server capable of inserting a digital code used for content data inquiry into a real-time video is received from a video transmission terminal of a video sender, and the first content data is received in the real-time video. When a processing request instructing to include inquiry information for searching the first content data is received, a main identification number of n (n is a natural number of 2 or more) digits for use as the inquiry information and k (k is a natural number of 2 or more) natural number) an identification number generation unit that randomly generates a sub-identification number of digits, and when a first main identification number is generated as the main identification number and a first auxiliary identification number is generated as the sub-identification number, an authentication token is generated. An authentication token calculation unit for calculating a first authentication token by applying the first primary identification number and the first auxiliary identification number as inputs to a preset token generation function for Code for storing 1 authentication token and the first auxiliary identification number in correspondence with the first content data in a content storage, and generating a digital code including the first main identification number and data for the first authentication token After the digital code is generated, the generator performs voice recognition on the real-time video data through a pre-built speech recognition model when real-time video data, which is a subject of live streaming, is received from the video transmission terminal. A code insertion unit for generating a caption and inserting the digital code into each bitmap font image for each of at least one character constituting the caption, and each bitmap font image for each of the at least one character constituting the caption and a data transmitter for inserting the caption into the real-time video data when the digital code is inserted, and then transmitting the real-time video data with the caption inserted to a viewer's video playback terminal.

In addition, a method of operating a live streaming server capable of inserting a digital code used for content data inquiry into a real-time video according to an embodiment of the present invention, while receiving first content data from a video transmission terminal of a video sender, When a processing request instructing to include search information for searching the first content data in the real-time video is received, a main identification number of n (n is a natural number equal to or greater than 2) digits for use as the search information and k (k is a natural number of 2 or more) randomly generating an auxiliary identification number of digits, when a first main identification number is generated as the main identification number and a first auxiliary identification number is generated as the auxiliary identification number, an authentication token Calculating a first authentication token by applying the first primary identification number and the first auxiliary identification number as inputs to a preset token generation function for generating a first authentication token; Storing an authentication token and the first auxiliary identification number in correspondence with the first content data in a content storage, and generating a digital code including the first main identification number and data for the first authentication token; After the digital code is generated, when real-time video data, which is a subject of live streaming, is received from the video transmission terminal, subtitles are generated by performing voice recognition on the real-time video data through a pre-built speech recognition model. and inserting the digital code into bitmap font images for each of the at least one character constituting the subtitle, and inserting the digital code into each bitmap font image of each of the at least one character constituting the subtitle. , inserting the caption into the real-time video data, and then transmitting the real-time video data with the caption inserted to the viewer's video playback terminal.

The present invention provides a live streaming server capable of inserting a digital code used to search content data into a real-time video and a method of operating the same, so that the usability of a live streaming service can be increased.

1 is a diagram showing the structure of a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention.
2 is a diagram for explaining the operation of a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention.
3 is a flowchart illustrating a method of operating a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. This description is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. While describing each drawing, similar reference numerals have been used for similar components, and unless otherwise defined, all terms used in this specification, including technical or scientific terms, are common knowledge in the art to which the present invention belongs. has the same meaning as commonly understood by the person who has it.

In this document, when a certain component is said to "include", it means that it may further include other components without excluding other components unless otherwise stated. In addition, in various embodiments of the present invention, each component, functional block, or means may be composed of one or more sub-components, and the electrical, electronic, and mechanical functions performed by each component are electronic It may be implemented with various known elements or mechanical elements such as circuits, integrated circuits, ASICs (Application Specific Integrated Circuits), and may be implemented separately or two or more may be integrated into one.

On the other hand, the blocks of the accompanying block diagram or the steps of the flowchart are computer program instructions that perform designated functions by being loaded into a processor or memory of a device capable of data processing, such as a general-purpose computer, a special purpose computer, a portable notebook computer, and a network computer. can be interpreted as meaning Since these computer program instructions may be stored in a memory included in a computer device or in a computer readable memory, the functions described in blocks of a block diagram or steps of a flowchart are produced as a product containing instruction means for performing them. It could be. Further, each block or each step may represent a module, segment or portion of code that includes one or more executable instructions for executing specified logical function(s). Also, it should be noted that in some alternative embodiments, functions mentioned in blocks or steps may be executed out of a predetermined order. For example, two blocks or steps shown in succession may be performed substantially simultaneously or in reverse order, and in some cases, some blocks or steps may be omitted.

1 is a diagram showing the structure of a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention.

Referring to FIG. 1, the live streaming server 110 includes an identification number generation unit 111, an authentication token calculation unit 112, a code generation unit 113, a code insertion unit 114 and a data transmission unit 115. include

When the first content data is received from the video transmission terminal 151 of the video sender 150, the identification number generation unit 111 instructs the real-time video to include search information for searching the first content data. When the processing request indicating is received, a primary identification number of n (n is a natural number of 2 or more) digits and a secondary identification number of k (k is a natural number of 2 or more) digits for use as the inquiry information are randomly generated.

For example, when n is '3' and k is '4', 'contents data 1' is received from the video transmission terminal 151 of the video sender 150, and 'contents data 1' is displayed in the real-time video. When it is assumed that a processing request instructing the inclusion of inquiry information for querying is received, the identification number generation unit 111 provides a '3' digit primary identification number and a '4' digit secondary identification number for use as the inquiry information. An identification number can be randomly generated.

In this way, when the first primary identification number is generated as the primary identification number by the identification number generator 111 and the first auxiliary identification number is generated as the auxiliary identification number, the authentication token calculator 112 generates an authentication token. A first authentication token is calculated by applying the first primary identification number and the first auxiliary identification number as inputs to a preset token generation function for generation.

At this time, according to one embodiment of the present invention, when the first main identification number and the first auxiliary identification number are applied as inputs, the token generation function uses n numbers constituting the first main identification number as components. An operation matrix having a size of n x k is generated by calculating the Kronecker Product between an n-dimensional column vector having a k number and a k-dimensional row vector having k numbers constituting the first auxiliary identification number as components, , may be a function that calculates a Euclidean norm of the operation matrix as the first authentication token.

Here, the Euclidean norm is an L2 norm for representing the size of a vector or matrix, and can be calculated according to Equation 1 below.

는 유클리드 노름을 의미하고, x_i는 벡터나 행렬에 포함되어 있는 i번째 성분을 의미한다.In Equation 1 above

denotes the Euclidean norm, and x _i denotes the i-th element included in a vector or matrix.

In addition, the Kronecker product means a matrix specifically expressing the tensor product of two matrices, a matrix A having a size of a x b as in Equation 2 below and a matrix B having a size of p x q as in Equation 3 below Assuming that is given, the Kronecker product between matrix A and matrix B can be expressed as in Equation 4 below.

For example, as in the above example, let n be '3' and k be '4', and as the primary identification number, assume that the first primary identification number is generated as '123', and the auxiliary identification number Assume that the first auxiliary identification number is generated as '4567'.

Then, the authentication token calculation unit 112 may apply '123' as the first primary identification number and '4567' as the first auxiliary identification number to the token generating function as inputs.

'이고, 상기 제1 보조 식별번호인 '4567'을 구성하는 '4'개의 숫자들을 성분으로 갖는 '4'차원의 행벡터는 '[4 5 6 7]' 이므로, 상기 토큰 생성 함수는 '

'과 '[4 5 6 7]' 간의 크로네커 곱을 연산하여, '3 x 4'의 크기를 갖는 연산 행렬을 '

'과 같이 생성한 후, 상기 연산 행렬의 유클리드 노름인 '42'를 상기 제1 인증 토큰으로 산출할 수 있다.In this regard, a '3'-dimensional column vector having '3' numbers constituting '123', the first main identification number, as components is '

', and the '4'-dimensional row vector having '4' numbers constituting '4567', the first auxiliary identification number, as components is '[4 5 6 7]', so the token generating function is '

By calculating the Kronecker product between ' and '[4 5 6 7]', an operation matrix having a size of '3 x 4' is '

', then '42', which is the Euclidean norm of the operation matrix, can be calculated as the first authentication token.

In this way, when the first authentication token is calculated by the authentication token calculation unit 112, the code generation unit 113 matches the first authentication token and the first auxiliary identification number with the first content data to provide content. Store in a storage, and generate a digital code including data for the first primary identification number and the first authentication token.

For example, as in the above example, the first primary identification number is '123', the first auxiliary identification number is '4567', and the first authentication token is '4567' by the authentication token calculation unit 112. 42', the code generator 113 converts '42' as the first authentication token and '4567' as the first auxiliary identification number to 'content data 1' as shown in Table 1 below. It can be stored in the content storage in correspondence with.

authentication token Secondary identification number content data 42 4567 content data 1 ... ... ...

At the same time, the code generating unit 113 may generate a digital code including data for '123' as the first primary identification number and '42' as the first authentication token.

In this way, after the digital code is generated by the code generator 113, when real-time video data, which is a target of live streaming, is received from the video transmission terminal 151, the code inserter 114 transmits a pre-constructed audio. By performing voice recognition on the real-time image data through a recognition model, a caption is generated, and the digital code is inserted into each bitmap font image for each of at least one character constituting the caption.

For example, as in the above example, it is assumed that the digital code including data for '123' as the first main identification number and '42' as the first authentication token is generated by the code generator 113. , After that, when it is assumed that 'real-time video data 1', which is a target of live streaming, is received from the video transmission terminal 151, the code insertion unit 114 uses a pre-built voice recognition model to obtain 'real-time video data Subtitles may be generated by performing voice recognition for 1'.

At this time, according to one embodiment of the present invention, when the subtitle is generated by performing voice recognition on the real-time video data, the code insertion unit 114 performs a bitmap for each of the at least one character constituting the subtitle. After dividing the font image into t horizontal (t is a natural number equal to or greater than 2) and t vertical divisions to generate t x t divisions, the pixel values of some of the divisions are converted into preset pixels By converting to a value, the digital code can be inserted.

For example, as in the above example, the subtitle was created by performing speech recognition on 'real-time video data 1' through a pre-built speech recognition model, and the t is set to '16', and the subtitle is configured Let's assume that the at least one letter 'ha'. Then, as shown in FIG. After generating x 16' divided regions, the digital code may be inserted by converting pixel values of some of the divided regions into preset pixel values.

In this way, when the digital code is inserted into each bitmap font image for each of the at least one character constituting the caption by the code insertion unit 114, the data transmission unit 115 converts the caption to the real-time video data. After insertion, the real-time video data with the subtitle inserted is transmitted to the video playback terminal 161 of the viewer 160 .

For example, as in the above example, when the real-time video data is referred to as 'real-time video data 1', the data transmission unit 115 inserts the subtitle into 'real-time video data 1' and then inserts the subtitle into 'real-time video data 1'. Real-time video data 1' may be transmitted to the video playback terminal 161 of the viewer 160.

At this time, according to an embodiment of the present invention, when the video playback terminal 161 receives the real-time video data from the live streaming server 110, the real-time video data is reproduced, and the subtitles are displayed together with the real-time video on the screen. can display

For example, as in the above example, as the data transmission unit 115 transmits the 'real-time video data 1' with the subtitle inserted therein to the video playback terminal 161, the video playback terminal 161 receives the 'real-time video data 1'. By playing ', the subtitle can be displayed on the screen together with the real-time video.

In relation to this, when 'real-time video data 1' is reproduced in the video reproducing terminal 161, the viewer 160 uses a predetermined mobile terminal 162 possessed by the viewer 160 to display the at least one of the subtitles. When the digital code inserted into the bitmap font image for the first character, which is one of the characters, is recognized, the mobile terminal 162 identifies the digital code, thereby confirming information included in the digital code.

According to one embodiment of the present invention, the live streaming server 110 may further include a secondary identification number extraction unit 116, a token verification unit 117 and a content transmission unit 118.

The auxiliary identification number extraction unit 116 is a bit for a first letter, which is any one of the at least one letter constituting the subtitle of the real-time video displayed as the real-time video data is reproduced in the video reproducing terminal 161. Since the digital code inserted into the map font image is scanned and recognized by the mobile terminal 162 possessed by the viewer 160, the first note included in the digital code is received from the mobile terminal 162. When an identification number and data for the first authentication token are received, and a content data search request based on the first main identification number and the first authentication token is received, the authentication token identical to the first authentication token is received from the content storage. Corresponds to and extracts the stored auxiliary identification number.

The token verification unit 117 is an auxiliary identification number stored in correspondence with the same authentication token as the first authentication token from the content storage by the auxiliary identification number extractor 116, and when the first auxiliary identification number is extracted , The calculation result calculated when the first secondary identification number extracted from the content storage and the first primary identification number received from the mobile terminal 162 are applied as inputs to the token generation function, the first authentication token check whether this is the case.

When the content transfer unit 118 confirms that the calculation result is the first authentication token by the token verification unit 117, the first content stored in correspondence with the first auxiliary identification number from the content storage. After extracting the data, the first content data is transmitted to the mobile terminal 162 .

Hereinafter, operations of the auxiliary identification number extraction unit 116, the token verification unit 117, and the content transmission unit 118 will be described in detail, for example.

First, according to the above example, as the real-time video data is reproduced in the video playback terminal 161, it is assumed that the subtitle of the real-time video is displayed on the screen, and the mobile terminal 162 owned by the viewer 160 ), it is assumed that the digital code inserted into the bitmap font image for the first character, which is any one of the at least one character constituting the subtitle, is scanned and recognized.

In this regard, according to the foregoing example, since the digital code includes data for '123' as the first primary identification number and '42' as the first authentication token, the mobile terminal 162 may use the digital code Data on '123' as the first primary identification number and '42' as the first authentication token may be extracted from the data.

Then, the mobile terminal 162 transmits data on '123' as the first main identification number and '42' as the first authentication token to the live streaming server 110, while the first main identification number and A content data search request based on the first authentication token may be transmitted.

Accordingly, when the live streaming server 110 receives data for '123' as the first primary identification number and '42' as the first authentication token, and the content data search request is received, the secondary identification number The extraction unit 116 may extract an auxiliary identification number stored in correspondence with the same authentication token as '42', which is the first authentication token, from the content storage as shown in Table 1 above.

In relation to this, since the auxiliary identification number stored in correspondence to the same authentication token as '42' is '4567' in the content storage as shown in Table 1, the auxiliary identification number extraction unit 116 is able to extract the contents as shown in Table 1 above. You can extract '4567' from the repository.

Then, the token verification unit 117 applies '4567' extracted from the content storage as shown in Table 1 and '123', the first main identification number received from the mobile terminal 162, to the token generating function as inputs. It is possible to check whether the calculation result obtained when the first authentication torque is '42' is obtained.

As a result, when it is confirmed that the calculation result is '42', the contents transmission unit 118 stores '42' corresponding to the first auxiliary identification number '4567' from the contents storage as shown in Table 1 above. After extracting 'content data 1', 'content data 1' may be transmitted to the mobile terminal 162.

At this time, according to one embodiment of the present invention, the content transmission unit 118 is configured to encrypt the first content data and transmits the encrypted first content data to the video playback terminal 161, and the content encryption unit 119 and the transmission processing unit 120 ) may be included.

When the content encryption unit 119 confirms that the calculation result is the first authentication token by the token confirmation unit 117, the first content stored in correspondence with the first auxiliary identification number from the content storage. Data is extracted, an encryption key for encrypting the first content data is randomly generated, and then the first content data is encrypted with the encryption key.

When the first content data is encrypted by the content encryption unit 119, the transmission processing unit 120 transmits the encrypted first content data to the mobile terminal 162 and simultaneously to the video playback terminal 161. , transmits an exposure command instructing to expose the encryption key on the screen on which the real-time video is displayed.

Hereinafter, operations of the content encryption unit 119 and the transmission processing unit 120 will be described in detail by way of example.

First, as in the above example, it is assumed that the calculation result is confirmed to be '42', which is the first authentication token, by the token verification unit 117.

Then, the content encryption unit 119 may extract 'contents data 1' stored in correspondence to '4567', which is the first auxiliary identification number, from the contents storage as shown in Table 1, and 'contents data 1'. An encryption key for encrypting can be randomly generated.

In this case, when the encryption key is referred to as 'encryption key 1', the content encryption unit 119 may encrypt 'content data 1' with 'encryption key 1'.

Then, the transmission processing unit 120 transmits the encrypted 'contents data 1' to the mobile terminal 162, and at the same time, to the video playback terminal 161, 'encryption key 1' is displayed on the screen on which the real-time video is displayed. may transmit an exposure command instructing to expose '.

At this time, when receiving the exposure command from the live streaming server 110, the video playback terminal 161 may expose 'encryption key 1' on the screen on which the real-time video is displayed in response to the exposure command. .

At this time, according to one embodiment of the present invention, when the mobile terminal 162 receives the encrypted first content data from the live streaming server 110, the video playback terminal 161 for the viewer 160 After displaying an input guide message instructing the mobile terminal 162 to input the encryption key exposed through the screen on which the real-time video is displayed, the viewer 160 sends the encryption key to the mobile terminal 162. , the encrypted first content data can be decrypted with the encryption key and displayed on the screen.

For example, as in the above example, it is assumed that the encrypted 'contents data 1' is received by the mobile terminal 162 as the transmission processing unit 120 transmits the encrypted 'contents data 1' to the mobile terminal 162. let's do it.

Then, the mobile terminal 162 instructs the viewer 160 to input the 'encryption key 1' exposed through the screen on which the real-time video of the video playback terminal 161 is displayed to the mobile terminal 162. An input guide message may be displayed.

In this regard, according to the above-described example, since the 'encryption key 1' is exposed on the screen on which the real-time video of the video playback terminal 161 is displayed, the viewer 160 may view the real-time video of the video playback terminal 161. After confirming 'encryption key 1' through the screen on which the video is displayed, 'encryption key 1' can be input to the mobile terminal 162 according to the input guide message.

At this time, according to the above example, since the encrypted 'contents data 1' is data generated by encrypting 'contents data 1' with 'encryption key 1' by the contents encryption unit 119, the mobile terminal 162 With the 'encryption key 1', the encrypted 'content data 1' can be decrypted and displayed on the screen.

3 is a flowchart illustrating a method of operating a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention.

In step S310, when the first content data is received from the video transmission terminal of the video sender and a processing request instructing to include search information for searching the first content data in the real-time video is received, the inquiry is performed. A main identification number of n (n is a natural number of 2 or more) digits and an auxiliary identification number of k (k is a natural number of 2 or more) digits for use as information are randomly generated.

In step S320, when a first primary identification number is generated as the primary identification number and the first secondary identification number is generated as the secondary identification number, in a preset token generation function for generating an authentication token, the first secondary identification number is generated. A first authentication token is calculated by applying the identification number and the first auxiliary identification number as inputs.

In step S330, when the first authentication token is calculated, the first authentication token and the first auxiliary identification number are stored in a content storage in correspondence with the first content data, and the first main identification number and the first auxiliary identification number are stored. A digital code containing data for the first authentication token is generated.

In step S340, after the digital code is generated, when real-time video data to be live-streamed is received from the video transmission terminal, voice recognition for the real-time video data is performed through a pre-built speech recognition model. By doing so, a subtitle is generated, and the digital code is inserted into each bitmap font image for each of at least one character constituting the subtitle.

In step S350, when the digital code is inserted into each bitmap font image for each of the at least one character constituting the caption, the caption is inserted into the real-time video data, and then the real-time video into which the caption is inserted Data is transmitted to the viewer's video playback terminal.

At this time, according to one embodiment of the present invention, when the first main identification number and the first auxiliary identification number are applied as inputs, the token generation function uses n numbers constituting the first main identification number as components. After generating an operation matrix having a size of n x k by calculating the Kronecker product between an n-dimensional column vector having the first auxiliary identification number and a k-dimensional row vector having k numbers constituting the first auxiliary identification number as components, the operation matrix It may be a function that calculates a Euclid norm as the first authentication token.

At this time, according to one embodiment of the present invention, in step S340, when the subtitle is generated by performing voice recognition on the real-time video data, a bitmap font image for each of the at least one character constituting the subtitle. is divided into t horizontal (t is a natural number equal to or greater than 2) and t vertical divisions to generate t x t divisions, and then pixel values of some of the divisions are preset as the preset pixel values By converting, the digital code can be inserted.

In addition, according to an embodiment of the present invention, when the real-time video data is received from the live streaming server, the video playback terminal reproduces the real-time video data to display the subtitles together with the real-time video on the screen. can

In addition, according to an embodiment of the present invention, the operating method of the live streaming server may include any one of the at least one letter constituting the subtitle of the real-time video displayed as the real-time video data is reproduced in the video playback terminal. Since the digital code inserted into the bitmap font image for the first character, which is one, is scanned and recognized by the mobile terminal possessed by the viewer, the first character included in the digital code is received from the mobile terminal. When a content data search request based on the first main identification number and the first authentication token is received while the data for the primary identification number and the first authentication token are received, the same authentication as the first authentication token is received from the content storage. extracting an auxiliary identification number stored in correspondence with the token, when the first auxiliary identification number is extracted from the content storage as an auxiliary identification number stored in correspondence with the same authentication token as the first authentication token; When the first auxiliary identification number extracted from the content storage and the first primary identification number received from the mobile terminal are applied as inputs to the token generation function, whether the calculation result is the first authentication token The checking step and when it is confirmed that the calculation result is the first authentication token, the first content data stored in correspondence to the first auxiliary identification number is extracted from the content storage, and then the first content data is stored. The step of transmitting to the mobile terminal may be further included.

In this case, in the step of transmitting the first content data to the mobile terminal, when it is confirmed that the calculation result is the first authentication token, the first authentication token stored in correspondence to the first auxiliary identification number from the content storage. Extracting content data, randomly generating an encryption key for encrypting the first content data, then encrypting the first content data with the encryption key, and when the first content data is encrypted, to the mobile terminal , Transmitting the encrypted first content data and, at the same time, transmitting an exposure command instructing the video playback terminal to expose the encryption key on a screen on which the real-time video is displayed. .

At this time, when the mobile terminal receives the encrypted first content data from the live streaming server, the encryption key exposed through the screen on which the real-time video of the video playback terminal is displayed is transmitted to the viewer. After displaying an input guide message instructing input to the mobile terminal, when the encryption key is input to the mobile terminal by the viewer, the encrypted first content data is decrypted with the encryption key and displayed on the screen. can

In the above, with reference to FIG. 3 , a method of operating a live streaming server capable of inserting a digital code used to retrieve content data into a real-time video according to an embodiment of the present invention has been described. Here, the method of operating a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention is the live streaming server 110 described with reference to FIGS. 1 and 2. Since it may correspond to a configuration for an operation, a detailed description thereof will be omitted.

A method of operating a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention is implemented as a computer program stored in a storage medium for execution through combination with a computer. can

In addition, the method of operating a live streaming server capable of inserting a digital code used to search content data into a real-time video according to an embodiment of the present invention is implemented in the form of program commands that can be executed through various computer means. It can be recorded on a readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the medium may be those specially designed and configured for the present invention or those known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, the present invention has been described by specific details such as specific components and limited embodiments and drawings, but these are provided to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. , Those skilled in the art in the field to which the present invention belongs can make various modifications and variations from these descriptions.

Therefore, the spirit of the present invention should not be limited to the described embodiments, and it will be said that not only the claims to be described later, but also all modifications equivalent or equivalent to these claims belong to the scope of the present invention. .

110: live streaming server
111: identification number generation unit 112: authentication token calculation unit
113: code generation unit 114: code insertion unit
115: data transmission unit 116: auxiliary identification number extraction unit
117: token verification unit 118: content transmission unit
119: content encryption unit 120: transmission processing unit
150: video sender
151: video transmission terminal
160: viewer
161: video playback terminal
162: mobile terminal

Claims (14)

In a live streaming server capable of inserting a digital code used to search content data in real-time video,
When the first content data is received from the video transmission terminal of the video sender and a processing request instructing to include search information for searching the first content data in the real-time video is received, for use as the search information an identification number generation unit randomly generating a primary identification number of n (n is a natural number of 2 or more) digits and an auxiliary identification number of k (k is a natural number of 2 or more) digits;
When a first primary identification number is generated as the primary identification number and a first auxiliary identification number is generated as the secondary identification number, a preset token generation function for generating an authentication token is performed using the first primary identification number and the secondary identification number. 1 authentication token calculator for calculating a first authentication token by applying the auxiliary identification number as an input;
When the first authentication token is calculated, the first authentication token and the first auxiliary identification number are stored in a content storage in correspondence with the first content data, and the first main identification number and the first authentication token are stored. a code generating unit that generates a digital code including data about;
After the digital code is generated, when real-time video data, which is a subject of live streaming, is received from the video transmission terminal, subtitles are generated by performing voice recognition on the real-time video data through a pre-built speech recognition model. a code inserting unit inserting the digital code into each bitmap font image for each of at least one character constituting the subtitle; and
When the digital code is inserted into each bitmap font image for each of the at least one character constituting the caption, the caption is inserted into the real-time video data, and then the real-time video data with the caption inserted is converted into a viewer's image. Data transmission unit that transmits data to the playback terminal
Live streaming server that includes. According to claim 1,
The token generating function is
When the first main identification number and the first auxiliary identification number are applied as inputs, an n-dimensional column vector having n numbers constituting the first main identification number as components and the first auxiliary identification number are formed. After generating an operation matrix having a size of nxk by calculating the Kronecker product between k-dimensional row vectors having k numbers of Live streaming server, characterized in that the function calculated as a token. According to claim 1,
The code insertion part
When the subtitle is generated by performing voice recognition on the real-time video data, dividing the bitmap font image for each of the at least one character constituting the subtitle into t (where t is a natural number equal to or greater than 2) horizontally and t vertically Live streaming server characterized by inserting the digital code by dividing into regions, generating txt number of divided regions, and converting pixel values of some of the divided regions into preset pixel values . According to claim 1,
The video playback terminal
The live streaming server, characterized in that, when the real-time video data is received from the live streaming server, the real-time video data is reproduced to display the subtitle along with the real-time video on the screen. According to claim 1,
In the video reproducing terminal, the digital code inserted into the bitmap font image for the first letter, which is any one of the at least one letter constituting the subtitle of the real-time video displayed as the real-time video data is reproduced, Due to being scanned and recognized by the mobile terminal possessed by the viewer, data for the first main identification number and the first authentication token included in the digital code are received from the mobile terminal, and the first a secondary identification number extraction unit extracting a secondary identification number stored in correspondence with the same authentication token as the first authentication token from the content storage when a content data search request based on the primary identification number and the first authentication token is received;
As an auxiliary identification number stored in correspondence with the same authentication token as the first authentication token from the content storage, when the first auxiliary identification number is extracted, the first auxiliary identification number extracted from the content storage and the mobile terminal a token verification unit for confirming whether or not a calculation result calculated when the first primary identification number received from is applied as an input to the token generation function becomes the first authentication token; and
When it is confirmed that the calculation result is the first authentication token, the first content data stored in correspondence to the first auxiliary identification number is extracted from the content storage, and the first content data is transmitted to the mobile terminal. content delivery unit
Live streaming server that further includes. According to claim 5,
The content delivery unit
When it is confirmed that the calculation result is the first authentication token, the first content data stored in correspondence with the first auxiliary identification number is extracted from the content storage, and encryption for encrypting the first content data is performed. a content encryption unit that randomly generates a key and then encrypts the first content data with the encryption key; and
When the first content data is encrypted, transmit the encrypted first content data to the mobile terminal and at the same time expose the encryption key to the video playback terminal on a screen on which the real-time video is displayed. Transmission processing unit that transmits the exposure command to indicate
including,
The mobile terminal
When the encrypted first content data is received from the live streaming server, ask the viewer to input the encryption key exposed through the screen on which the real-time video of the video playback terminal is displayed to the mobile terminal. After displaying an input guide message instructing, when the encryption key is input to the mobile terminal by the viewer, the encrypted first content data is decrypted with the encryption key and displayed on the screen. server. In the operating method of a live streaming server capable of inserting a digital code used to search content data in real-time video,
When the first content data is received from the video transmission terminal of the video sender and a processing request instructing to include search information for searching the first content data in the real-time video is received, for use as the search information randomly generating a main identification number of n (n is a natural number of 2 or more) digits and an auxiliary identification number of k (k is a natural number of 2 or more) digits;
When a first primary identification number is generated as the primary identification number and a first auxiliary identification number is generated as the secondary identification number, a preset token generation function for generating an authentication token is performed using the first primary identification number and the secondary identification number. Calculating a first authentication token by applying 1 auxiliary identification number as an input;
When the first authentication token is calculated, the first authentication token and the first auxiliary identification number are stored in a content storage in correspondence with the first content data, and the first main identification number and the first authentication token are stored. generating a digital code including data about;
After the digital code is generated, when real-time video data, which is a subject of live streaming, is received from the video transmission terminal, subtitles are generated by performing voice recognition on the real-time video data through a pre-built speech recognition model. and inserting the digital code into each bitmap font image for each of at least one character constituting the subtitle; and
When the digital code is inserted into each bitmap font image for each of the at least one character constituting the caption, the caption is inserted into the real-time video data, and then the real-time video data with the caption inserted is converted into a viewer's image. Transmitting to the playback terminal
Method of operation of a live streaming server comprising a. According to claim 7,
The token generating function is
When the first main identification number and the first auxiliary identification number are applied as inputs, an n-dimensional column vector having n numbers constituting the first main identification number as components and the first auxiliary identification number are formed. After generating an operation matrix having a size of nxk by calculating the Kronecker product between k-dimensional row vectors having k numbers of A method of operating a live streaming server, characterized in that the function is calculated as a token. According to claim 7,
Inserting the digital code
When the subtitle is generated by performing voice recognition on the real-time video data, dividing the bitmap font image for each of the at least one character constituting the subtitle into t (where t is a natural number equal to or greater than 2) horizontally and t vertically Live streaming server characterized by inserting the digital code by dividing into regions, generating txt number of divided regions, and converting pixel values of some of the divided regions into preset pixel values how it works. According to claim 7,
The video playback terminal
When the real-time video data is received from the live streaming server, the real-time video data is reproduced to display the subtitle along with the real-time video on the screen. According to claim 7,
In the video reproducing terminal, the digital code inserted into the bitmap font image for the first letter, which is any one of the at least one letter constituting the subtitle of the real-time video displayed as the real-time video data is reproduced, Due to being scanned and recognized by the mobile terminal possessed by the viewer, data for the first main identification number and the first authentication token included in the digital code are received from the mobile terminal, and the first extracting a secondary identification number stored in correspondence with the same authentication token as the first authentication token from the content storage when a content data search request based on the primary identification number and the first authentication token is received;
As an auxiliary identification number stored in correspondence with the same authentication token as the first authentication token from the content storage, when the first auxiliary identification number is extracted, the first auxiliary identification number extracted from the content storage and the mobile terminal checking whether or not a calculation result calculated when the first primary identification number received from is applied as an input to the token generation function becomes the first authentication token; and
When it is confirmed that the calculation result is the first authentication token, the first content data stored in correspondence to the first auxiliary identification number is extracted from the content storage, and the first content data is transmitted to the mobile terminal. steps to transmit
Method of operation of a live streaming server further comprising a. According to claim 11,
Transmitting the first content data to the mobile terminal
When it is confirmed that the calculation result is the first authentication token, the first content data stored in correspondence with the first auxiliary identification number is extracted from the content storage, and encryption for encrypting the first content data is performed. After randomly generating a key, encrypting the first content data with the encryption key; and
When the first content data is encrypted, transmit the encrypted first content data to the mobile terminal and at the same time expose the encryption key to the video playback terminal on a screen on which the real-time video is displayed. Sending an exposure command instructing
including,
The mobile terminal
When the encrypted first content data is received from the live streaming server, ask the viewer to input the encryption key exposed through the screen on which the real-time video of the video playback terminal is displayed to the mobile terminal. After displaying an input guide message instructing, when the encryption key is input to the mobile terminal by the viewer, the encrypted first content data is decrypted with the encryption key and displayed on the screen. How the server works. A computer-readable recording medium recording a computer program for executing the method of any one of claims 7 to 12 through a combination with a computer. A computer program stored in a storage medium for executing the method of any one of claims 7 to 12 through a combination with a computer.

Images