KR102593762B1 - Method and server for generating highlight contents for live internet broadcasting based on voice recognition - Google Patents

Abstract

A server that generates highlight content for voice-based live Internet broadcasting is disclosed. This server may include a server communication unit that communicates with a host terminal and a listener terminal of a voice-based live Internet broadcast, a server memory that stores reinforcement data used in highlight content, and a server processor including one or more cores. The server processor may include a broadcast data management module, a setting module, a section determination module, and a generation module. By providing this server, user convenience can be improved and the effect of increasing profits from live Internet broadcasting can be expected.

Description

Method for generating highlight content for live Internet broadcasting based on voice recognition and server for the same {METHOD AND SERVER FOR GENERATING HIGHLIGHT CONTENTS FOR LIVE INTERNET BROADCASTING BASED ON VOICE RECOGNITION}

This disclosure relates to a voice-based live Internet broadcasting service. More specifically, the present disclosure relates to a method for generating highlight content for a live Internet broadcast based on voice recognition and a server for the same.

The content described below simply provides background information related to this embodiment and does not constitute prior art.

With the recent rapid spread of the Internet in the past terrestrial-centered broadcasting system, viewer demand for platforms that support broadcasting services through the Internet has increased explosively, and Internet-based streaming services are being actively provided, and video-based streaming services are being actively provided. In addition to streaming services, audio-based streaming services are also steadily growing.

In addition, voice-based live Internet broadcasting is being actively conducted in response to the needs of listeners who want to hear the host's voice directly and communicate in real time, as well as broadcasts recorded in audio-based streaming services. In voice-based live Internet broadcasting, the broadcast host broadcasts based on audio, and real-time communication takes place through messenger chatting and multi-party live calls.

Since many broadcast hosts are single or small-scale, it is difficult not only to conduct broadcasts but also to separately produce broadcast-related content.

Registered Patent Publication No. 10-2388670 (Registration date: 2022.04.15)

The purpose of the embodiments disclosed in this disclosure is to provide a method for generating highlight content related to voice-based live Internet broadcasting.

In addition, the purpose of the embodiment disclosed in the present disclosure is to provide a method of generating a plurality of highlight contents related to voice-based live Internet broadcasting, providing them to the broadcast host, and selecting the highlight content according to the broadcast host's selection.

Additionally, the purpose of the embodiment disclosed in the present disclosure is to provide a method of encouraging listeners to participate in live broadcasting by producing highlight content related to their preferred live Internet broadcasting.

The problems to be solved by the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

A server that generates highlight content for voice-based live Internet broadcasting according to the present disclosure to achieve the above-described technical problem includes a server communication unit that communicates with a host terminal and a listener terminal of the voice-based live Internet broadcasting; Server memory to store enrichment data used in highlight content; and a server processor including one or more cores.

The server processor includes a broadcast data management module that divides broadcast data of the live Internet broadcast into predetermined categories and stores them in the server memory; a setting module that sets trigger conditions for generating one or more highlight contents for the live Internet broadcast; a section determination module that determines a broadcast section of the live Internet broadcast that satisfies the set trigger condition; and a creation module that generates the highlight content for the live Internet broadcast by combining the broadcast data of the determined broadcast section and the reinforcement data.

The server processor generates the highlight content and provides it to the host terminal through the server communication unit, and when at least one of the highlight contents is selected by the host terminal, the selected highlight content is transmitted to the listener terminal through the server communication unit. It can be configured to provide through.

The broadcast data management module may be configured to divide broadcast data of the live Internet broadcast into at least one category of the host's voice data, chat data, live call data during broadcast, and sponsorship-related data and store them in the server memory. there is.

The setting module includes the chat progress speed in the live Internet broadcast, the number of sponsored listeners and the sponsorship amount of sponsored listeners, the frequency of use of a specific keyword, a specific emotional state implied by the keyword, the listener's participation level, the host's voice intensity, and the host's voice intensity. It may be configured to set a condition in which at least one of the utterance frequency and ignition time is above a predetermined level as the trigger condition.

The setting module may be configured to set a condition requested by the host terminal as a trigger condition for generating the highlight content.

If there are a plurality of sections satisfying the trigger conditions, the generation module integrates some of the multiple sections into one highlight content, and determines the playback order within the content for each section based on the weight between the trigger conditions in the integrated content. It can be configured to be deployed differently.

The reinforcement data is data that is not actually broadcast or used in the live Internet broadcast, and includes audio data to increase the number of views of the highlight content, video data, language data used by generation or situation, and hot keywords of the live Internet broadcast. It can contain at least one.

In addition, a method of generating highlight content for a voice-based live Internet broadcast performed by a computing device according to the present disclosure to achieve the above-described technical problem includes dividing broadcast data of the live Internet broadcast into predetermined categories. saving; setting trigger conditions for generating one or more highlight contents for the live Internet broadcast; determining a broadcast section of the live Internet broadcast that satisfies the set trigger condition; and generating the highlight content for the live Internet broadcast by combining broadcast data of the determined broadcast section and reinforcement data pre-stored for use in the highlight content.

The generating method includes, after the generating step, generating the highlight content and providing it to a host terminal; and when at least one of the highlight contents is selected by the host terminal, providing the selected highlight contents to the listener terminal.

The setting step includes the chat progress speed in the live Internet broadcast, the number of sponsored listeners and the sponsorship amount of sponsored listeners, the frequency of use of a specific keyword, a specific emotional state implied by the keyword, the listener's participation, the host's voice strength, It may include setting a condition in which at least one of the host's speech frequency and speech time is above a predetermined level as the trigger condition.

In addition to this, a computer program stored in a computer-readable recording medium may be further provided to execute the method for implementing the present disclosure.

In addition, a computer-readable recording medium recording a computer program for executing a method for implementing the present disclosure may be further provided.

According to the means for solving the above-described problem of the present disclosure, the burden on the host for producing highlight content related to voice-based live Internet broadcasting can be reduced, and listeners who are interested in the live Internet broadcast by watching the highlight content can participate in the broadcast and Sponsorship can increase, user convenience can be improved, and profits can be expected to increase through live Internet broadcasting.

The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned may be clearly understood by those skilled in the art from the description below.

1 is a diagram schematically illustrating a voice-based live Internet broadcasting system according to the present disclosure.
Figure 2 is a relative block diagram showing the configurations of a server and a host terminal that generate highlight content according to the present disclosure.
Figure 3 is a sequence diagram showing the processing process of a server that generates highlight content according to the present disclosure.
FIG. 4 is a diagram illustrating a server that generates highlight content that separates and stores data according to the present disclosure.
FIG. 5 is a diagram illustrating trigger conditions for generating highlight content according to the present disclosure.
FIG. 6 is a diagram illustrating a process for sharing highlight content selected by a host terminal of a server generating highlight content according to the present disclosure.

Like reference numerals refer to like elements throughout this disclosure. The present disclosure does not describe all elements of the embodiments, and general content or overlapping content between the embodiments in the technical field to which the present disclosure pertains is omitted. The term 'unit, module, member, block' used in the specification may be implemented as software or hardware, and depending on the embodiment, a plurality of 'unit, module, member, block' may be implemented as a single component, or It is also possible for one 'part, module, member, or block' to include multiple components.

Throughout the specification, when a part is said to be “connected” to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection through a wireless communication network. do.

Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

Throughout the specification, when a member is said to be located “on” another member, this includes not only cases where a member is in contact with another member, but also cases where another member exists between the two members.

Terms such as first and second are used to distinguish one component from another component, and the components are not limited by the above-mentioned terms.

Singular expressions include plural expressions unless the context clearly makes an exception.

The identification code for each step is used for convenience of explanation. The identification code does not explain the order of each step, and each step may be performed differently from the specified order unless a specific order is clearly stated in the context. there is.

Hereinafter, the operating principle and embodiments of the present disclosure will be described with reference to the attached drawings.

In this specification, the 'server that generates highlight content according to the present disclosure' may be implemented not only by a cloud system but also by various devices that can perform computational processing and provide results to the user. For example, a server that generates highlight content according to the present disclosure may include all of a computer, a server device, and a portable terminal, or may be in any one form.

Here, the computer may include, for example, a laptop, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

The server device is a server that processes information by communicating with external devices, and may include an application server, computing server, database server, file server, game server, mail server, proxy server, and web server.

The portable terminal is, for example, a wireless communication device that guarantees portability and mobility, such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), and PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone ), all types of handheld wireless communication devices, and wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-device (HMD). may include.

Functions related to artificial intelligence according to the present disclosure are operated through a processor and memory. The processor may consist of one or multiple processors. At this time, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-specific processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-specific processor such as an NPU. One or more processors control input data to be processed according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, when one or more processors are dedicated artificial intelligence processors, the artificial intelligence dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

Predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, created through learning means that a basic artificial intelligence model is learned using a large number of learning data by a learning algorithm, thereby creating a predefined operation rule or artificial intelligence model set to perform the desired characteristics (or purpose). It means burden. This learning may be accomplished in the device itself that performs artificial intelligence according to the present disclosure, or may be accomplished through a separate server and/or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

An artificial intelligence model may be composed of multiple neural network layers. Each of the plurality of neural network layers has a plurality of weight values, and neural network calculation is performed through calculation between the calculation result of the previous layer and the plurality of weights. Multiple weights of multiple neural network layers can be optimized by the learning results of the artificial intelligence model. For example, a plurality of weights may be updated so that loss or cost values obtained from the artificial intelligence model are reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNN), for example, Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks, etc., but are not limited to the examples described above.

The processor creates a neural network, trains or learns a neural network, performs calculations based on received input data, generates an information signal based on the results, or generates a neural network. The network can be retrained.

Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), and LSTM. (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield) Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network) ), Liquid State Machine (LSM), Extreme Learning Machine (ELM), Echo State Network (ESN), Deep Residual Network (DRN), Differential Neural Computer (DNC), Neural Turning Machine (NTM), Capsule Network (CN), Those skilled in the art will understand that it may include any neural network, including, but not limited to, KN (Kohonen Network) and AN (Attention Network).

According to an exemplary embodiment of the present disclosure, the processor may support a Convolution Neural Network (CNN), a Region with Convolution Neural Network (R-CNN), a Region Proposal Network (RPN), a Recurrent Neural Network (RNN), such as GoogleNet, AlexNet, VGG Network, etc. ), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT for natural language processing, SP-BERT, MRC/QA, Text Analysis, Dialog System, GPT-3 , GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, and Anomaly Detection, Prediction, Time-Series Forecasting, Optimization, Recommendation, and Data Creation for ResNet data intelligence. , but is not limited to this. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

1 is a diagram schematically illustrating a voice-based live Internet broadcasting system 1000 according to the present disclosure.

The live Internet broadcasting system 1000 may include components necessary to provide voice-based live Internet broadcasting. The live Internet broadcasting system 1000 generates highlight content for voice-based live Internet broadcasting. A server (100, hereinafter referred to as 'highlight content creation server'), a host terminal (200) corresponding to the host (H) of the live Internet broadcast, and a listener terminal (300) corresponding to the listener (L) of the live Internet broadcast. It can be included.

The highlight content creation server 100 receives broadcast data for voice-based live Internet broadcasting from the host terminal 200 and provides it to the listener terminal 300, so that the live Internet broadcast can be provided to the listener (L). There is (S11).

In an embodiment, the highlight content creation server 100 may be implemented not only as a server function in a server and client structure, but also as a cloud. In this case, the highlight content creation server 100 may be equipped with storage, servers, virtualization modules/systems, operating systems, middleware, runtime modules, data, and applications. Accordingly, the host terminal 200 and the listener terminal 300 at the client end can use various functions provided by the cloud system as long as they are equipped with a service application, and can be charged according to the amount of usage. For example, the highlight content creation server 100 may be implemented based on SaaS (Server as a software), but the embodiment is not limited to this.

The highlight content creation server 100 may provide a messenger and multi-party Internet call function to the host terminal 200 and the listener terminal 300 so that the host (H) and the listener (L) can communicate (S13, S15).

The highlight content creation server 100 produces (generates) highlight content related to the live Internet broadcast while the host (H) is conducting a voice-based live Internet broadcast or after the broadcast is completed, and provides the host terminal 200 and It can be provided to the listener terminal 300 (S17, S19).

Here, highlight content is not only content actually used in the live broadcast, but also data not used in the live broadcast, audio data, video data, generational or situational data to increase the value of the highlight content, make it interesting and attractive, or increase the number of views. It can be implemented as content that includes a variety of language data, hot keywords from live Internet broadcasts, letters, and symbols. Accordingly, highlight content can be generated based on voice and can also be generated based on video.

FIG. 2 is a relative block diagram showing the configurations of the highlight content creation server 100 and the host terminal 200 according to the present disclosure.

First, the highlight content creation server 100 according to the present disclosure may include a server communication unit 110, a server input unit 120, a server display 130, a server memory 150, and at least one processor 190. . The components of the highlight content creation server 100 shown in FIG. 2 are not essential for implementing the highlight content creation server 100 according to the present disclosure, so the highlight content creation server 100 described above is It may have more or fewer components than those listed.

Among the above components, the server communication unit 110 may include one or more components that enable communication with external devices (including host terminals and listener terminals, etc.), for example, a broadcast reception module and a wired communication module. , it may include at least one of a wireless communication module, a short-range communication module, and a location information module. The server communication unit 110 can communicate with the host terminal 200 and the listener terminal 300 of a voice-based live Internet broadcast.

The server input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user, and includes at least one of at least one camera, at least one microphone, and a user input unit. It can be included. Voice data or image data collected from the server input unit 120 may be analyzed and processed as a user's control command.

The server display 130 displays (outputs) information processed by the highlight content creation server 100. For example, the server display 130 displays execution screen information of an application (eg, an application) running on the highlight content creation server 100, or a user interface (UI) or graphic user interface (GUI) according to such execution screen information. Interface) information can be displayed.

The server memory 150 can store data supporting various functions of the highlight content creation server 100 and a program for the operation of the control unit, and can store input/output data (e.g., music files, still images, video, etc.), a plurality of application programs (application programs or applications) running on the highlight content creation server 100, data for operation of the device, and commands can be stored. At least some of these applications may be downloaded from an external server via wireless communication. In particular, the server memory 150 may store reinforcement data used to create highlight content.

The server memory 150 is of flash memory type, hard disk type, solid state disk type, SDD type (Silicon Disk Drive type), and multimedia card micro type. card micro type), card type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), EEPROM (electrically It may include at least one type of storage medium among erasable programmable read-only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk. In addition, the server memory 150 is separate from the highlight content creation server 100, but may be a database connected by wire or wirelessly, and may be implemented as a database system. The server memory 150 may store the above-described reinforcement data used in highlight content.

The server processor 190 includes at least one core, a memory that stores data for an algorithm for controlling the operation of components in the highlight content creation server 100 or a program that reproduces the algorithm, and data stored in the memory. It can be implemented with at least one processor (not shown) that performs the above-described operations using . At this time, the memory and processor may each be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip.

The server processor 190 includes a broadcast data management module 191 that divides broadcast data of live Internet broadcasts into predetermined categories and stores them in the server memory 150, and trigger conditions for generating one or more highlight contents for live Internet broadcasts. A setting module 193 for setting, a section determination module 195 for determining the broadcast section of the live Internet broadcast that satisfies the set trigger conditions, and a highlight content for the live Internet broadcast by combining the broadcast data and reinforcement data of the determined broadcast section. It may include a generation module 197 that generates.

In addition, the server processor 190 combines any one or a plurality of the components described above in order to implement various embodiments according to the present disclosure described in FIGS. 3 to 6 below on the highlight content creation server 100. You can control it.

The host terminal 200 shown in FIG. 2 may include a terminal communication unit 210, a terminal input unit 220, a terminal display 230, a terminal memory 250, and a terminal processor 290. The components of the host terminal 200 shown in FIG. 2 are not essential for implementing the host terminal 200 according to the present disclosure, so the host terminal 200 described in this specification has more components than the components listed above. It may have many or few components.

When explaining the configurations and differences of the highlight content creation server 100, the terminal input unit 220 may include a microphone for receiving the voice of the host H, but the embodiment is limited to this. That is not the case.

The terminal display 230 can output voice-based messenger chatting for live Internet broadcasting and various interfaces for live Internet broadcasting.

In an embodiment, the terminal processor 290 may include a filtering module that recognizes and filters the content of the host H's speech, and the filtering module recognizes the content of the host H's speech based on a speech to text (STT) technique. By converting the voice into text, filtering target phrases (including, for example, profanity or swear words, etc.) can be highlighted and displayed on the terminal display 230, and the filtering target can also be displayed in the transmitted voice signal of the host (H). The phrase may be processed (for example, there is a method of replacing it with a specific sound or lowering the output, but the embodiment is not limited thereto) and provided to the highlight content creation server 100. At this time, the terminal processor 290 may provide a warning notification through the terminal display 230 or speaker so that the host H can recognize that the filtering target phrase has been used.

At least one component may be added or deleted in response to the performance of the components shown in FIG. 2. Additionally, it will be easily understood by those skilled in the art that the mutual positions of the components may be changed in response to the performance or structure of the system.

Meanwhile, each component shown in FIG. 2 refers to software and/or hardware components such as Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

Figure 3 is a sequence diagram showing the processing process of the highlight content creation server 100 according to the present disclosure. FIG. 4 is a diagram for explaining the highlight content creation server 100 that separates and stores data according to the present disclosure, and FIG. 5 is a diagram for explaining trigger conditions for generating highlight content according to the present disclosure. While explaining FIG. 3, FIGS. 4 and 5 will be described with reference to FIGS. 4 and 5 where necessary.

First, the broadcast data management module 191 of the server processor 190 can store broadcast data of voice-based live Internet broadcasting by dividing it into predetermined categories (S310).

Referring to FIG. 4, the broadcast data management module 191 can separately store broadcast data (BD) and reinforcement data (VD).

The broadcast data management module 191 includes broadcast data (BD) of a live Internet broadcast, voice data (BD1) of the host (H), chat data (BD2), live call data during broadcast (BD3), and sponsorship-related data (BD4). ) can be divided into at least one category and stored in the server memory 150, and the broadcast data management module 191 can separately store various analysis data (BD5) in the server memory 150.

The broadcast data management module 191 may separately store voice data BD1 for each predetermined time section. The broadcast data management module 191 can store chat data (BD2) by dividing it into host (H) and listener (L), and can store non-chat basic templates (phrases not entered by the host (H) or listener (L)). , images, etc.) can be stored separately.

The broadcast data management module 191 can separately store the speech contents of the host (H) and the listener (L) among the live call data (BD3) during the broadcast, and convert them into text based on the STT algorithm to store them in the server memory 150. It can be saved in .

The broadcast data management module 191 can store sponsorship-related data (BD4), including information on the number of sponsored listeners and the sponsorship amount of sponsored listeners.

The broadcast data management module 191 can store various analysis data (BD5) in the server memory 150. The analyzed data (BD5) includes the frequency of use of specific keywords, specific emotional states implied by keywords, and listener participation. , the intensity of the host's voice, the host's speaking frequency and speaking time, etc., but the embodiment is not limited thereto.

In an embodiment, the server processor 190 may analyze a specific emotional state implied by a specific keyword using an artificial intelligence-based analysis model. The analysis model can convert the host (H)'s speech into text, receive phrases from the chat window as input, and analyze the specific emotional state implied by the keyword. The analysis model can analyze and output emotional states such as joy, sadness, boredom, fun, and anger. The server processor 190 may separately store the analyzed specific emotional state in the server memory 150.

Additionally, the broadcast data management module 191 can store augmentation data (VD) separately from broadcast data (BD), which may be data that is not actually broadcast or used in a live Internet broadcast. However, although it is saved as reinforcement data (VD), if it is used redundantly in actual broadcasting, it can be stored redundantly in actual broadcast data (BD).

Reinforcement data may include audio data to increase the number of views of highlight content, video data, language data used by generation or situation, hot keywords of live Internet broadcasts, etc., but the embodiment is not limited thereto.

In an embodiment, in order to create highlight content, the server processor 190 sorts and stores hot keywords used during times of greatest listener participation in order of listener popularity, and selects the hot keywords with high priority when creating highlight content. can be used. At this time, the server processor 190 may adjust the popularity ranking of the hot keyword downward according to the number of uses of the hot keyword that has already been used.

After step S310, the setting module 193 of the server processor 190 may set a trigger condition (TC) for generating one or more highlight contents for a live Internet broadcast (S320).

Referring to FIG. 5, the server processor 190 can generate highlight content when the corresponding trigger condition is satisfied. Trigger conditions (TC) are when the chat progress speed in a live Internet broadcast is above a certain level (TC1), when the number of sponsored listeners is above a certain level, when the sponsorship amount of sponsored listeners is above a certain level (TC2), and when it is a specific time zone. (TC3), when the frequency of use of a specific keyword is above a predetermined level, when the specific emotional state implied by the keyword is above a predetermined level (TC4), when the host's voice intensity is above a predetermined level, when the host's utterance frequency is above a predetermined level It may include at least one of a case where the host's speech time is above a predetermined level (TC5) and a case where the listener's participation level is above a predetermined level (TC6). Additionally, the trigger condition (TC) may also include a setting condition (TC7) requested by the host (H).

Specifically, the server processor 190 sets a trigger condition when the chat progress speed in a live Internet broadcast is above a predetermined level (TC1), for example, when the number of chats per second is 5 or more, and when the change in chat speed (acceleration) is above a predetermined acceleration. It is determined that this is satisfied, and highlight content can be created.

Additionally, the server processor 190 may determine that the trigger condition is satisfied when the number of sponsored listeners is greater than a predetermined level or when the sponsorship amount of the sponsored listeners is greater than a predetermined level (TC2) and generate highlight content. In an embodiment, the server processor 190 may determine that a trigger condition is satisfied in the case of a live Internet broadcast conducted by a popular host and generate highlight content. The standard for a popular host may be determined by the number of listeners participating, the number of listeners sponsoring, and the sponsorship amount, but the embodiment is not limited to this.

Additionally, in the case of a broadcast in a specific time zone (TC3), the server processor 190 may determine that the trigger condition is satisfied and generate highlight content. In an embodiment, the server processor 190 may generate highlight content for a live Internet broadcast at a time set by the host (H) or at a time set by the listener (L).

In an embodiment, if the sponsorship amount is more than a predetermined amount, the server processor 190 may generate highlight content including messenger chat content or Internet call content of the sponsored listener who paid the sponsorship amount only for the sponsor. To this end, the server processor 190 may list sponsors in order of sponsorship amount, synchronize the sponsorship timing and sponsor participation information (messenger chat content, Internet call content, etc.) and store them in the server memory 150.

Additionally, the server processor 190 may determine that the trigger condition is satisfied when the frequency of use of a specific keyword is above a predetermined level (TC4) and generate highlight content. For example, the server processor 190 generates highlight content when the number of emoticons (including 'ㅋ', 'ㅠ', etc.) representing emotions input by the listener (L) or host (H) is more than a predetermined number (TC4). can do. At this time, the server processor 190 may simply count the number of emoticons, but only one emoticon can be reflected per chat.

In addition, the server processor 190 operates when the voice intensity of the host (H) is higher than a predetermined level, the speech frequency of the host is higher than a predetermined level, or the speech time of the host is higher than a predetermined level (TC5), for example, the server processor 190 When the voice intensity of the host H is measured to be above a predetermined decibel or the utterance time/frequency is above a predetermined level, it is determined that the trigger condition is satisfied, and highlight content can be generated.

In addition, when the listener's participation degree is above a predetermined level (TC6), for example, when more than a predetermined number of people participate in a specific live Internet broadcast, the server processor 190 sets the number of chat participation of each listener to a predetermined number. In the case above, it is determined that the trigger condition is satisfied, and highlight content can be generated.

In an embodiment, when only a specific listener inputs excessive chat content, the server processor 190 may reduce and count the input keywords of the specific listener or, in some cases, ignore them.

Additionally, when the host terminal 200 requests creation of highlight content (TC7), the server processor 190 may determine that the trigger condition is satisfied and generate highlight content.

After step S320, the section determination module 195 of the server processor 190 may determine a broadcast section of a live Internet broadcast that satisfies a set trigger condition (S330).

The section determination module 195 can determine a broadcast section that satisfies the trigger condition for each trigger condition in a live Internet broadcast.

After step S330, the generation module 197 of the server processor 190 may generate highlight content for the live Internet broadcast by combining the broadcast data and reinforcement data of the determined broadcast section (S340).

The creation module 197 can generate highlight content according to various criteria, which will be described in detail below.

The creation module 197 may generate highlight content for each section that satisfies the trigger condition. The generation module 197 may combine reinforcement data for each broadcast section in which the trigger condition is satisfied.

The creation module 197 can recreate highlight content by combining data that is actually broadcast or not used in a live Internet broadcast into a broadcast section, and can generate audio data (music, specific voice, and hot keywords) to increase the number of views of highlight content. (corresponding sound, etc.) can be added, avatars corresponding to the host and avatar content corresponding to the listener can be added, and conversations can be reconstructed in chat or Internet calls. At this time, the server processor 190 may reconstruct the conversation and add a comment evaluating the conversation by adding a virtual third party. The creation module 197 can generate highlight content so that the avatar also shows emotions in situations where the host or listener shows emotions in a conversation between the hosts and listeners. At this time, the creation module 197 may use the above-described artificial intelligence-based analysis model that identifies the emotion implied in the keyword.

Additionally, the creation module 197 may generate highlight content by combining language data used by generation or situation, hot keywords of live Internet broadcasting, etc., together in the determined broadcast section. The creation module 197 can insert keywords with high priority among hot keywords at appropriate points in the highlight content based on video or audio.

In addition, when there are multiple sections that satisfy the trigger conditions, the creation module 197 integrates some of the multiple sections into one highlight content, and determines the playback order within the content based on the weight between the trigger conditions in the integrated content. It can be arranged very differently.

In an embodiment, the creation module 197 may separately generate highlight content corresponding to a section with a large sponsorship amount by adding reinforcement data, and may additionally generate separate highlight content. In the case of separate highlight content, the creation module 197 includes a section where the chat progress speed is above a predetermined level, a section where the frequency of use of a specific keyword or a specific emotional state implied by the keyword is above a predetermined level, a section where the listener's participation rate is above a predetermined level, Highlight content can be created by dividing the host into sections where the voice intensity/utterance frequency/utterance time is above a certain level and adding reinforcement data for each section.

In an embodiment, when generating integrated content, the creation module 197 places points that simultaneously satisfy a plurality of trigger conditions in the first half of the highlight content, and sequentially places points that satisfy only one trigger condition in the second half of the highlight content. It can be placed. Additionally, the creation module 197 may place a trigger condition with a high weight in the first half of the integrated highlight content. The weight may be determined according to the selection of the host, but the embodiment is not limited thereto.

In an embodiment, the creation module 197 may generate the integrated content into a plurality of partially integrated contents when the playback time of the integrated content is longer than a predetermined time.

FIG. 6 is a diagram illustrating a process of sharing highlight content selected by the host terminal of the highlight content creation server 100 according to the present disclosure.

First, the server processor 190 may generate a plurality of highlight contents (S410) and provide them to the host terminal 200 (S420).

The host terminal 200 may select highlight content (S430) and provide a selection notification to the highlight content creation server 100 (S440).

In an embodiment, server processor 190 may perform a billing process while providing highlight content to the host.

The server processor 190 can provide the selected highlight content to the listener terminal 300 (S450), and the listener terminal 300 can play the corresponding highlight content (S460).

At this time, the listener terminal 300 does not need to be a listener of the broadcast, and can receive highlight content in real time even without downloading it. Accordingly, the effect of encouraging broadcast listening may occur.

In another embodiment, during a live Internet broadcast, if there is an inquiry from a listener in an Internet call or chat window involving the host, the server processor 190 lists the inquiry and sends it to the host terminal ( 200), and inquiries excluding inquiries that have been answered from the inquiry list may be provided to the host terminal 200 at a predetermined period. At this time, the server processor 190 may convert the inquiry generated from the Internet call into text and provide it to the host terminal 200.

In this case, the server processor 190 merges inquiries that are duplicated or have a predetermined similarity range among the listener's inquiries, sorts the inquiries with the highest frequency of inquiries, and sends them to the host terminal 200 through the server communication unit 110. can be provided.

The server processor 190 may use a similarity detection algorithm that outputs the similarity as a percentage when two inquiries are input in order to determine whether the inquiries overlap or have a certain degree of similarity. At this time, the server processor 190 may correspond to each inquiry with a vector in space, and if the cosine similarity value between the two vectors is 0.8 or more and 1 or less, it may be determined that the inquiries are similar. The server processor 190 can determine the similarity between inquiries based on the noun of each inquiry, and the cosine similarity value can be calculated by the scalar product and size between vectors.

In an embodiment, the host terminal 200 can respond or reject unanswered inquiries, and if rejected, the server processor 190 displays the rejected inquiries so that only the host terminal 200 can see them. This can be provided to the host terminal 200.

In an embodiment, the server processor 190 may provide a warning message to the listener terminal through the server communication unit 110 to listeners who ask difficult questions beyond a predetermined level.

Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium that stores instructions executable by a computer. Instructions may be stored in the form of program code, and when executed by a processor, may create program modules to perform operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

Computer-readable recording media include all types of recording media storing instructions that can be decoded by a computer. For example, there may be Read Only Memory (ROM), Random Access Memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage device, etc.

As described above, the disclosed embodiments have been described with reference to the attached drawings. A person skilled in the art to which this disclosure pertains will understand that the present disclosure may be practiced in forms different from the disclosed embodiments without changing the technical idea or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

100: highlight content creation server, 200: host terminal, 300: listener terminal.

Claims (10)

As a server that generates highlight content for voice-based live Internet broadcasting,
a server communication unit that communicates with a host terminal and a listener terminal of a voice-based live Internet broadcast; Server memory to store enrichment data used in highlight content; and
A server processor including one or more cores,
The server processor is,
a broadcast data management module that divides broadcast data of the live Internet broadcast into predetermined categories and stores them in the server memory;
a setting module that sets trigger conditions for generating one or more highlight contents for the live Internet broadcast;
a section determination module that determines a broadcast section of the live Internet broadcast that satisfies the set trigger condition; and
A generation module that generates the highlight content for the live Internet broadcast by combining the broadcast data of the determined broadcast section and the reinforcement data,
The reinforcement data is,
It is data that is not actually broadcast or used in the live Internet broadcast, and includes at least one of audio data to increase the number of views of the highlight content, video data, language data used by generation or situation, and hot keywords of the live Internet broadcast; ,
The creation module is,
If there are multiple sections that satisfy the trigger conditions, some of the multiple sections are integrated into one highlight content, and the playback order within the content is arranged differently for each section based on the weight between the trigger conditions in the integrated content. ,
The creation module is,
A section that satisfies the trigger condition with a high weight is placed in the first half of the integrated content, a section that simultaneously satisfies a plurality of trigger conditions is placed in the first half of the integrated content, and a section that satisfies only one trigger condition is placed in the first half of the integrated content. A server that is sequentially placed in the second half of the integrated content. According to paragraph 1,
The server processor is,
The highlight content is generated and provided to the host terminal through the server communication unit, and when at least one of the highlight contents is selected by the host terminal, the selected highlight content is provided to the listener terminal through the server communication unit. Being a server. According to paragraph 2,
The broadcast data management module,
A server configured to classify broadcast data of the live Internet broadcast into at least one category of the host's voice data, chat data, live call data during the broadcast, and sponsorship-related data and store them in the server memory. According to paragraph 3,
The setting module is,
In the live Internet broadcast, the speed of chatting, the number of sponsored listeners and the sponsorship amount of sponsored listeners, the frequency of use of specific keywords, the specific emotional state implied by the keyword, listener participation, the intensity of the host's voice, and the frequency and speech of the host. A server configured to set a condition in which at least one of the times is above a predetermined level as the trigger condition. According to paragraph 4,
The setting module is,
A server configured to set a condition requested by the host terminal as a trigger condition for generating the highlight content. delete delete A method of generating highlight content for a voice-based live Internet broadcast performed by a highlight content creation server,
Classifying and storing broadcast data of the live Internet broadcast into predetermined categories;
setting trigger conditions for generating one or more highlight contents for the live Internet broadcast;
determining a broadcast section of the live Internet broadcast that satisfies the set trigger condition; and
Generating, through a generation module, the highlight content for the live Internet broadcast by combining broadcast data of the determined broadcast section and reinforcement data pre-stored for use in the highlight content,
The reinforcement data is,
It is data that is not actually broadcast or used in the live Internet broadcast, and includes at least one of audio data to increase the number of views of the highlight content, video data, language data used by generation or situation, and hot keywords of the live Internet broadcast; ,
The creation module is,
If there are multiple sections that satisfy the trigger conditions, some of the multiple sections are integrated into one highlight content, and the playback order within the content is arranged differently for each section based on the weight between the trigger conditions in the integrated content. ,
The creation module is,
A section that satisfies the trigger condition with a high weight is placed in the first half of the integrated content, a section that simultaneously satisfies a plurality of trigger conditions is placed in the first half of the integrated content, and a section that satisfies only one trigger condition is placed in the first half of the integrated content. A creation method that places sequentially in the second half of integrated content. According to clause 8,
After the above generating step,
generating the highlight content and providing it to a host terminal; and
When at least one of the highlight contents is selected by the host terminal, the method further comprising providing the selected highlight content to a listener terminal. According to clause 9,
The setting steps are:
In the live Internet broadcast, the speed of chatting, the number of sponsored listeners and the sponsorship amount of sponsored listeners, the frequency of use of specific keywords, the specific emotional state implied by the keyword, listener participation, the intensity of the host's voice, and the frequency and speech of the host. A generating method comprising setting a condition in which at least one of the times is above a predetermined level as the trigger condition.

Images