CN117939258A

CN117939258A - Live broadcast on-demand monitoring method and device, equipment and medium thereof

Info

Publication number: CN117939258A
Application number: CN202211268196.3A
Authority: CN
Inventors: 余彬彬; 黄志义; 张聪
Original assignee: Guangzhou Cubesili Information Technology Co Ltd
Current assignee: Guangzhou Cubesili Information Technology Co Ltd
Priority date: 2022-10-17
Filing date: 2022-10-17
Publication date: 2024-04-26

Abstract

The application discloses a live broadcast on-demand monitoring method, a device, equipment and a medium thereof, wherein the method comprises the following steps: responding to a virtual camera calling event, and determining third-party broadcasting software triggering the virtual camera calling event; acquiring software information of the third-party playing software, and intercepting a current graphical user interface to acquire an interface screenshot; identifying third-party live broadcast characteristic information contained in the interface screenshot; and correspondingly storing the software information, the interface screenshot and the third-party live broadcast characteristic information into a platform software protection library. The method monitors the third party software which calls the virtual camera of the first party software to conduct live broadcast, and protects the software authority and the software security of the first party software by collecting the software information and the live broadcast characteristic information of the third party software.

Description

Live broadcast on-demand monitoring method and device, equipment and medium thereof

Technical Field

The application relates to the field of copyright protection and network live broadcast, in particular to a live broadcast on-demand monitoring method, and further relates to a device, equipment and a nonvolatile storage medium corresponding to the method.

Background

The existing live broadcast opening software applied to the live broadcast service has various live broadcast functions, so that the live broadcast service of a host user is assisted, the host user can use functions such as virtual animation special effects or beautifying filters in a live broadcast room where the host user is located, live broadcast pictures in the live broadcast room are beautified, and the live broadcast atmosphere in the live broadcast room is improved, but part of third-party live broadcast platforms can call the live broadcast functions of the live broadcast software developed by other live broadcast platforms to use through technical means, so that the live broadcast functions of the live broadcast software developed by other live broadcast platforms are greatly disturbed, part of the third-party live broadcast platforms do not use the functions or material content of the live broadcast software developed by other live broadcast platforms through permission of other live broadcast platforms, the software security of the live broadcast software developed by other live broadcast platforms is infringed, and live broadcast service development profit is carried out by the live broadcast software developed by the third-party live broadcast platforms by other live broadcast platforms, so that the live broadcast platform is quite unfair to other live broadcast platforms.

In view of the problems of the existing third-party live broadcast platform using live broadcast on-air software of other live broadcast platforms, the inventor makes corresponding exploration in consideration of solving the problems.

Disclosure of Invention

The application aims to meet the requirements of users and provides a live broadcast monitoring method, and further relates to a corresponding device, equipment, a nonvolatile storage medium and a computer program product of the method.

In order to achieve the purpose of the application, the following technical scheme is adopted:

The application provides a live broadcast on-stream monitoring method which is suitable for the purpose of the application, and comprises the following steps:

Responding to a virtual camera calling event, and determining third-party broadcasting software triggering the virtual camera calling event;

acquiring software information of the third-party playing software, and intercepting a current graphical user interface to acquire an interface screenshot;

Identifying third-party live broadcast characteristic information contained in the interface screenshot;

and correspondingly storing the software information, the interface screenshot and the third-party live broadcast characteristic information into a platform software protection library.

In a further embodiment, the step of responding to the virtual camera call event and determining the third party playing software triggering the virtual camera call event includes the following steps:

responding to the virtual camera calling event, and acquiring a calling party software name of calling party broadcasting software triggering the virtual camera calling event;

Judging whether the calling party software name is a preset first party playing software name, if not, determining the calling party playing software as third party playing software.

In a further embodiment, the step of obtaining the software information of the third party playing software and intercepting the current gui to obtain the screenshot includes the following steps:

Responding to a first party live broadcast function use event, and determining a first party live broadcast function corresponding to the first party live broadcast function use event;

Acquiring function use information corresponding to the first party live broadcast function, and correspondingly storing the function use information to a platform software protection library;

And acquiring software information of the third-party playing software, and performing screenshot processing on a graphical user interface triggering the first-party live broadcast function event to acquire a corresponding interface screenshot.

In a further embodiment, the step of identifying the third party live feature information included in the screenshot includes the following steps:

performing image preprocessing on the interface screenshot, and extracting an image feature vector of the interface screenshot;

identifying a plurality of text messages displayed in the interface screenshot according to the image feature vector;

and calling a text classifier, extracting text information which accords with a third-party live broadcast feature text matching rule from each piece of text information, determining a text feature type corresponding to each piece of text information, and packaging the text information and the text feature type as a third-party live broadcast feature text to obtain the third-party live broadcast feature information.

according to the image feature vector, identifying a face image contained in the interface screenshot;

And packaging the face image as a main broadcasting face image into the third-party live broadcast characteristic information.

In a further embodiment, the step of storing the software information, the screenshot and the third-party live broadcast feature information in a platform software protection library includes the following steps:

Acquiring third-party live broadcast platform information and a third-party anchor user name contained in the third-party live broadcast characteristic information;

Inquiring a target third party platform data table group corresponding to the third party live broadcast platform information in the platform software protection library;

And inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the target third-party platform data group, and storing the software information, the interface screenshot and the third-party live broadcast characteristic information into the target third-party anchor data table.

In a further embodiment, after the step of storing the software information, the screenshot and the third-party live feature information in the platform software protection library, the method includes:

Acquiring a user identifier of a host player logged in first-party multicast software of a called virtual camera;

inquiring target user account information corresponding to the anchor user identification in a user database corresponding to the first party multicast software;

and inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the platform software protection library according to the third-party anchor user name contained in the third-party live broadcast characteristic information.

And storing the target user account information to the target third-party anchor data table.

and updating the account information of the target user so as to modify the anchor user corresponding to the anchor user identifier into a forbidden anchor user.

The application provides a live broadcast monitoring device, which comprises:

the calling event response module is used for responding to the virtual camera calling event and determining third-party broadcasting software triggering the virtual camera calling event;

The interface screenshot obtaining module is used for obtaining the software information of the third-party playing software and intercepting the current graphical user interface to obtain an interface screenshot;

the interface screenshot identifying module is used for identifying the third-party live broadcast characteristic information contained in the interface screenshot;

And the third party information storage module is used for correspondingly storing the software information, the interface screenshot and the third party live broadcast characteristic information into a platform software protection library.

In a further embodiment, the calling event response module includes:

The software name acquisition sub-module is used for responding to the virtual camera calling event and acquiring the calling party software name of calling party multicast software triggering the virtual camera calling event;

and the software name judging sub-module is used for judging whether the calling party software name is a preset first party playing software name, and if not, determining the calling party playing software as third party playing software.

In a further embodiment, the interface screenshot obtaining module includes:

the live broadcast function determining submodule is used for responding to the first party live broadcast function use event and determining a first party live broadcast function corresponding to the first party live broadcast function use event;

the use information acquisition sub-module is used for acquiring function use information corresponding to the first party live broadcast function so as to correspondingly store the function use information into a platform software protection library;

And the interface screenshot obtaining sub-module is used for obtaining the software information of the third-party playing software, and carrying out screenshot processing on the graphical user interface triggering the first-party live broadcast function event to obtain a corresponding interface screenshot.

In a further embodiment, the interface screenshot identifying module includes:

The feature vector acquisition sub-module is used for carrying out image preprocessing on the interface screenshot and extracting the image feature vector of the interface screenshot;

The text information identification sub-module is used for identifying a plurality of text information displayed in the interface screenshot according to the image feature vector;

And the text information classification sub-module is used for calling a text classifier, extracting text information which accords with a third-party live broadcast feature text matching rule from each text information, determining a text feature type corresponding to each text information, and packaging the text information and the text feature type as a third-party live broadcast feature text into the third-party live broadcast feature information.

In a preferred embodiment, the interface screenshot identifying module further includes:

the face image recognition sub-module is used for recognizing a face image contained in the interface screenshot according to the image feature vector;

and the face image packaging sub-module is used for packaging the face image serving as the anchor face image into the third-party live broadcast characteristic information.

In a further embodiment, the third party information storage module includes:

The characteristic information analysis sub-module is used for acquiring the third-party live broadcast platform information and the third-party anchor user name contained in the third-party live broadcast characteristic information;

The data table group inquiring sub-module is used for inquiring a target third party platform data table group corresponding to the third party live broadcast platform information in the platform software protection library;

And the information storage sub-module is used for inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the target third-party platform data group and storing the software information, the interface screenshot and the third-party live broadcast characteristic information into the target third-party anchor data table.

In order to solve the above technical problem, an embodiment of the present application further provides a computer device, including a memory and a processor, where the memory stores computer readable instructions, and when the computer readable instructions are executed by the processor, the processor is caused to execute the steps of the live broadcast monitoring method.

To solve the above technical problem, an embodiment of the present application further provides a storage medium storing computer readable instructions, where the computer readable instructions when executed by one or more processors cause the one or more processors to execute the steps of the live broadcast monitoring method.

In order to solve the above technical problem, an embodiment of the present application further provides a computer program product, which includes a computer program and computer instructions, where the computer program and the computer instructions, when executed by a processor, cause the processor to execute the steps of the live broadcast monitoring method.

Compared with the prior art, the application has the following advantages:

the application monitors the third party broadcasting software of the virtual camera module of the first party broadcasting software in violation call to reserve files for the software, the platform and the anchor information of the third party broadcasting software, thereby protecting the software security and the software use right of the first party broadcasting software; when the first party broadcasting software monitors that the virtual camera is called, the monitoring third party broadcasting software acquires the software information of the third party broadcasting software, and through a graphical user interface when the terminal equipment operates the third party broadcasting software, characteristic information of a third party live broadcasting platform contained in the intercepted and acquired interface screenshot is identified by using a neural network model, such as a user name, a main broadcasting face image, a live broadcasting room name and a room number of a main broadcasting user using the third party broadcasting software and the third party live broadcasting platform to which the third party broadcasting software belongs, and further, the software information, the live broadcasting characteristic information and the interface screenshot of the third party broadcasting software are classified and stored into a platform software protection library, so that the live broadcasting platform of the first party broadcasting software performs account blocking and other stopping actions on the main broadcasting user who violates the use rule of the third party broadcasting software through the platform software protection library, or legal right is performed on the third party live broadcasting platform through the platform software protection library, the software use right and the software safety of the third party broadcasting software are protected, the competition environment of each live broadcasting platform in the network legal industry is protected, and the live broadcasting platform cooperation is promoted.

Drawings

The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic diagram of a typical network deployment architecture according to an embodiment of the present application;

FIG. 2 is a flow chart of an exemplary embodiment of a live broadcast monitoring method of the present application;

FIG. 3 is a diagram of a graphical user interface showing third party conferencing software in accordance with the present application;

FIG. 4 is a flowchart of a method for determining a caller's software for playing a virtual camera according to an embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for recording a live broadcast function of a third party playing software using a first party playing software according to an embodiment of the present application;

FIG. 6 is a flowchart illustrating a method for identifying text information of an interface screenshot using an image recognition model according to an embodiment of the present application;

FIG. 7 is a flowchart illustrating a specific embodiment of a face image with an interface screenshot identified by an image recognition model according to the present application;

FIG. 8 is a flowchart illustrating the method for classifying and storing software information, third-party live feature information, and interface screenshots into a platform software protection library according to an embodiment of the present application;

FIG. 9 is a flowchart illustrating a specific embodiment of storing user account information of a host user who invokes a virtual camera module to a platform software protection library;

FIG. 10 is a flow chart of an embodiment of the present application for modifying a host user who is illegal to invoke a virtual camera module to disable the host user;

FIG. 11 is a functional block diagram of an exemplary embodiment of a live broadcast monitoring device of the present application;

fig. 12 is a basic structural block diagram of a computer device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.

It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

As used herein, "client," "terminal device," and "terminal device" are understood by those skilled in the art to include both devices that include only wireless signal receivers without transmitting capabilities and devices that include receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device such as a personal computer, tablet, or the like, having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service, personal communications System) that may combine voice, data processing, facsimile and/or data communications capabilities; PDA (Personal DIGITAL ASSISTANT ) that may include a radio frequency receiver, pager, internet/intranet access, web browser, notepad, calendar and/or GPS (Global Positioning System ) receiver; a conventional laptop and/or palmtop computer or other appliance that has and/or includes a radio frequency receiver. As used herein, "client," "terminal device" may be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or adapted and/or configured to operate locally and/or in a distributed fashion, at any other location(s) on earth and/or in space. As used herein, a "client," "terminal device," or "terminal device" may also be a communication terminal, an internet terminal, or a music/video playing terminal, for example, may be a PDA, a MID (Mobile INTERNET DEVICE ), and/or a Mobile phone with a music/video playing function, or may also be a device such as a smart tv, a set top box, or the like.

The application refers to hardware such as a server, a client, a working node and the like, which essentially is an electronic device with personal computer and other functions, and is a hardware device with necessary components disclosed by von neumann principles such as a central processing unit (comprising an arithmetic unit and a controller), a memory, an input device, an output device and the like, wherein a computer program is stored in the memory, and the central processing unit calls the program stored in the external memory to run, executes instructions in the program, and interacts with the input and output devices, thereby completing specific functions.

It should be noted that the concept of the present application, called "server", is equally applicable to the case of server clusters. The servers should be logically partitioned, physically separate from each other but interface-callable, or integrated into a physical computer or group of computers, according to network deployment principles understood by those skilled in the art. Those skilled in the art will appreciate this variation and should not be construed as limiting the implementation of the network deployment approach of the present application.

Referring to fig. 1, the hardware base required for implementing the related technical solution of the present application may be deployed according to the architecture shown in the figure. The server 80 of the present application is deployed at the cloud as an online server, and may be responsible for further connecting to related data servers and other servers providing related support, so as to form a logically related service cluster, to provide services for related terminal devices, such as a smart phone 81 and a personal computer 82 shown in the figure, or a third party server (not shown). The smart phone and the personal computer can access the internet through a well-known network access mode, and establish a data communication link with the cloud server 80 so as to run a terminal application program related to the service provided by the server.

For the server, the application program is usually constructed as a service process, and a corresponding program interface is opened for remote call of the application program running on various terminal devices.

The application program refers to an application program running on a server or terminal equipment, the application program adopts a programming mode to realize the related technical scheme of the application, the program codes of the application program can be stored in a nonvolatile storage medium which can be identified by a computer in the form of computer executable instructions, and the program codes are called by a central processing unit to run in a memory, and the related device of the application is constructed by the running of the application program on the computer.

Those skilled in the art will appreciate that: although the various methods of the present application are described based on the same concepts so as to be common to each other, the methods may be performed independently of each other unless specifically indicated otherwise. Similarly, for the various embodiments disclosed herein, all concepts described herein are presented based on the same general inventive concept, and thus, concepts described herein with respect to the same general inventive concept, and concepts that are merely convenient and appropriately modified, although different, should be interpreted as equivalents.

Referring to fig. 2, in an exemplary embodiment of the present application, a live broadcast monitoring method includes the following steps:

Step S11, responding to a virtual camera calling event, and determining third party playing software triggering the virtual camera calling event:

The virtual camera invoking event refers to an event of invoking a virtual camera of the first party's opening software by the third party's opening software, and the third party's opening software invokes the virtual camera module of the first party's opening software to adapt to a live broadcast picture beautifying function of the first party's opening software, for example, a face beautifying function, a live broadcast animation special effect function, a face recognition animation special effect function, and the like, so that it can be understood that the third party's opening software invokes the virtual camera module of the first party's opening software to use the live broadcast picture beautifying function developed by the live broadcast platform of the first party's opening software, thereby providing a live broadcast picture beautifying function of the first party's opening software for an anchor user of the third party's opening software, but the third party's opening software invokes the event of the virtual camera module of the first party's opening software, and does not provide for the third party's opening software, therefore, the virtual camera triggered by the third party's opening software is not invoked for the third party's opening software, and the third party's opening software is triggered by the live broadcast picture beautifying function, and the third party's opening software is triggered by the virtual camera triggering information, and the third party's opening software triggering safety feature.

The first party playing software responds to the virtual camera calling event, and particularly, the first party playing software responds to the virtual camera calling event, acquires the calling party software name of the calling party playing software triggering the virtual camera calling event so as to judge whether the calling party software name is a preset first party playing software name, if yes, the calling party playing software is characterized as the first party playing software, if not, the calling party playing software is characterized as the third party playing software, and the calling party playing software is determined as the third party playing software so as to monitor the calling party playing software.

The virtual camera is a module of the first party playing software, specifically, the first party playing software applies for registering the virtual camera module with a live broadcast picture beautifying function to a running system running on a terminal device as a camera interface supported by the running system, when the first party playing software or the third party playing software needs to run the camera to start a live broadcast service, the virtual camera module registered to the running system is called, and then the live broadcast picture beautifying function of the virtual camera is used, such as a face beautifying function, a live broadcast animation special effect function, a face recognition animation special effect function and the like are added to a live broadcast room, live broadcast pictures of the live broadcast room are beautified, and the live broadcast atmosphere of the live broadcast room is promoted.

Step S12, obtaining the software information of the third-party playing software, and intercepting the current graphical user interface to obtain an interface screenshot:

The software information of the third-party playing software comprises the software name of the third-party playing software, the software installation path of the third-party playing software at the current terminal equipment, the hardware occupation condition information of the third-party playing software, the file signature of the third-party playing software and the like.

After responding to the virtual camera invoking event, intercepting a current graphical user interface displayed in screen hardware of the current terminal equipment so as to acquire the interface screenshot with the image information of the current graphical user interface.

Referring to fig. 3, fig. 3 is a schematic diagram of a current gui, the software window 301 shown in fig. 3 is a software window of the third party playing software triggering the virtual camera invoking event, the software window 301 includes a live broadcasting room information window 302 and a live broadcasting stream collecting and playing window 303, the interface screenshot is obtained by intercepting the current gui shown in fig. 3, so that the information displayed in the software window 301 shown in fig. 3 is identified later, such as the live broadcasting platform name, the anchor user name, the live broadcasting room name and the live broadcasting room number of the third party playing software platform displayed in the software window 301, and the face image and the animation special effect image in the live broadcasting stream collecting and playing window 303 are identified.

Step S13, identifying third-party live broadcast feature information contained in the interface screenshot:

After the interface screenshot is obtained, carrying out image recognition processing on the interface screenshot through the image recognition model constructed based on the neural network so as to obtain the third-party live broadcast characteristic information about the third-party multicast software displayed in the interface screenshot.

The image recognition model is a neural network model trained to be converged, image preprocessing, such as image scaling, image clipping or image normalization, is performed on the interface screenshot, after the image preprocessing of the interface screenshot is completed, image feature vectors of the interface screenshot are extracted, and then the image feature vectors are input into a full-connection layer or a convolution layer for processing, so that text information related to third-party broadcasting software or face image information of a host user and the like displayed in the interface screenshot are recognized, and the information is packaged into the third-party live broadcast feature information.

And specifically, carrying out image preprocessing on the interface screenshot through the image recognition model, extracting image feature vectors of the interface screenshot, recognizing a plurality of text messages displayed in the interface screenshot according to the image feature vectors, calling a text classifier, extracting text messages which accord with a third-party live broadcast feature text matching rule from the text messages, determining text feature types corresponding to the text messages, and packaging the text messages and the text feature types as third-party live broadcast feature texts into the third-party live broadcast feature information.

The image recognition model can be deployed to a service server of the first party playing software besides the current terminal equipment which is responsible for calling and executing by the first party playing software, and after the first party playing software obtains the interface screenshot, the interface screenshot is pushed to the service server so as to drive the service server to call the image recognition model, and the third party live broadcast characteristic information in the interface screenshot is recognized.

And specifically, the image recognition model is used for carrying out image preprocessing on the interface screenshot, extracting the image feature vector of the interface screenshot, recognizing the face image contained in the interface screenshot according to the image feature vector, and further packaging the face image as the anchor face image into the third-party live broadcast feature information.

The third-party live broadcast feature information generally comprises user and platform information which are identified from the interface screenshot and used for storing the third-party live broadcast platform information, a third-party host user name, a third-party live broadcast room number, a face image of a host user and the like, and the third-party live broadcast feature information is stored as the user and platform information which violate the user use rule of the first-party playing software by identifying the information from the interface screenshot, so that the intellectual property and software security of the first-party playing software which is called a virtual camera module are protected.

Step S14, storing the software information, the interface screenshot and the third-party live broadcast feature information correspondingly into a platform software protection library:

and after the third-party live broadcast characteristic information is identified from the interface screenshot, storing the third-party live broadcast characteristic information, the software information and the interface screenshot into the platform software protection library as mapping relation data.

The platform software protection library generally stores the third-party live broadcast feature information, the software information and the interface screenshot in a classified manner based on a third-party platform type and a host user type, specifically obtains the third-party live broadcast platform information and a third-party host user name contained in the third-party live broadcast feature information, so as to inquire out a target third-party platform data table group corresponding to the third-party live broadcast platform information in the platform software protection library, further inquire out a target third-party host data table corresponding to the third-party host user name in the target third-party platform data group, and stores the software information, the interface screenshot and the third-party live broadcast feature information into the target third-party host data table; it can be understood that the platform software protection library is provided with a plurality of anchor data tables, and each anchor data table is deployed in groups by using the type of the third party platform to which the anchor data table belongs to form each third party platform data table group, and further, a third party platform is used as a father node, and the storage structure of each anchor user to which the anchor user belongs as a child node correspondingly stores the third party live broadcast characteristic information, the software information and the interface screenshot corresponding to each anchor user.

After the software information, the interface screenshot and the third-party live characteristic information are correspondingly stored in the platform software protection library, the platform to which the first-party live software belongs can determine that the third-party live software of the third-party live software is used by violations according to the software information, the interface screenshot or the third-party live characteristic information, a third-party platform live broadcasting room in which the third-party live broadcasting software belongs is located, the live broadcasting function or material content of the third-party live broadcasting software is used by violations in the network live broadcasting platform to which the third-party live broadcasting software belongs is further located, for example, the third-party live broadcasting platform information contained in the third-party live broadcasting characteristic information is used for determining that the third-party live broadcasting software belongs to the third-party live broadcasting platform, and a third-party user name contained in the third-party live broadcasting characteristic information is used for determining that a third-party live broadcasting platform is used by violations in the third-party live broadcasting platform, and further querying the third-party live broadcasting platform by using the third-party live broadcasting software is provided for the third-party live broadcasting platform by the third-party live broadcasting platform or the third-party live broadcasting software, and the third-party live broadcasting software is further determined to be queried by the third-party live broadcasting platform on the third-party live broadcasting platform through the third-party live broadcasting software.

In an embodiment, the platform of the first party's open broadcasting software may use information related to the third party's open broadcasting software stored in the platform software protection library to construct a live broadcasting room recording evidence obtaining service acting on a third party's live broadcasting room using the first party's open broadcasting software in a violation manner, specifically, according to the software information or the third party's live broadcasting feature information, for example, first determining a third party's network live broadcasting platform to which the third party's open broadcasting software belongs according to the third party's live broadcasting platform information included in the third party's live broadcasting feature information, then locating a third party's live broadcasting room using the third party's open broadcasting software in the third party's live broadcasting platform according to a third party's live broadcasting user name in the third party's live broadcasting feature information, marking the third party's live broadcasting room as an infringement live broadcasting room, and entering into a live broadcasting room recording the corresponding third party's live broadcasting application program through the software information or through a browser, and recording a live broadcasting video interface in real time on the third party's live broadcasting room, and taking the live broadcasting content of the third party's open broadcasting software as the live broadcasting material.

The live broadcast platform to which the first party's playing software belongs can carry out non-compliance use playing software marking and account blocking processing on the anchor users belonging to the live broadcast platform through the platform software protection library, specifically, for the implementation mode of marking non-compliance use playing software, the target user account information corresponding to the anchor user identification in the user database corresponding to the first party's playing software is inquired through obtaining the anchor user identification currently logged in the first party's playing software called by the virtual camera, and then a target third party anchor account data table corresponding to the third party anchor user name in the platform software protection library is inquired according to the third party anchor user name contained in the currently obtained third party live broadcast feature information, so that the target user account information is stored in the target third party anchor data table, and the anchor user currently logged in the first party's playing software is recorded to use the first party's playing software in a non-compliance mode.

The live broadcast platform to which the first party's playing software belongs can carry out account blocking processing on the anchor user belonging to the live broadcast platform through the platform software protection library, specifically, the anchor user identification logged in the first party's playing software called with the virtual camera is obtained, so as to inquire out the target user account information corresponding to the anchor user identification in the user database corresponding to the first party's playing software, further, the target user account information is updated, so that the anchor user corresponding to the anchor user identification is modified to be a blocking anchor user, the anchor user is forbidden to log in and use the first party's playing software, the anchor user is prevented from calling the virtual camera module of the first party's playing software, the live broadcast service is carried out in the third party's playing software by using the functions of the virtual camera module, and the software use right of the first party's playing software is protected.

The platform software protection library can be deployed in a service server of a live broadcast platform to which the first party's playing software belongs, and the current terminal equipment pushes the third party live broadcast feature information, the software information and the interface screenshot into the service server so as to correspondingly store the third party live broadcast feature information, the software information and the interface screenshot into the platform software protection library by the service server, and simultaneously acquire the anchor user information logging in the first party's playing software, so that the corresponding anchor user is subjected to non-compliance use record and account blocking processing according to the anchor user information and the platform software protection library.

According to the method, the third party playing software of the virtual camera module of the first party playing software is monitored and illegally called, so that the software, the platform and the anchor information of the third party playing software are subjected to file-reserving backup, and the software safety and the software use right of the first party playing software are protected; when the first party broadcasting software monitors that the virtual camera is called, the monitoring third party broadcasting software acquires the software information of the third party broadcasting software, and through a graphical user interface when the terminal equipment operates the third party broadcasting software, characteristic information of a third party live broadcasting platform contained in the intercepted and acquired interface screenshot is identified by using a neural network model, such as a user name, a main broadcasting face image, a live broadcasting room name and a room number of a main broadcasting user using the third party broadcasting software and the third party live broadcasting platform to which the third party broadcasting software belongs, and further, the software information, the live broadcasting characteristic information and the interface screenshot of the third party broadcasting software are classified and stored into a platform software protection library, so that the live broadcasting platform of the first party broadcasting software performs account blocking and other stopping actions on the main broadcasting user who violates the use rule of the third party broadcasting software through the platform software protection library, or legal right is performed on the third party live broadcasting platform through the platform software protection library, the software use right and the software safety of the third party broadcasting software are protected, the competition environment of each live broadcasting platform in the network legal industry is protected, and the live broadcasting platform cooperation is promoted.

The above exemplary embodiments and variations thereof fully disclose embodiments of the live broadcast monitoring method of the present application, but various variations of the method may still be deduced by transforming and augmenting some technical means, as follows outline other embodiments:

In one embodiment, please refer to fig. 4, in the step of determining the third party playing software triggering the virtual camera call event in response to the virtual camera call event, the method includes the following steps:

Step S111, responding to the virtual camera call event, and obtaining a calling party software name of calling party playing software triggering the virtual camera call event:

When the virtual camera module of the first party playing software is called to trigger response to the virtual camera calling event, a calling party software name of the calling party playing software calling the virtual camera module is obtained, specifically, the first party playing software determines the calling party playing software calling the virtual camera module through the running memory of the current terminal equipment so as to obtain the calling party software name of the calling party playing software from the running memory.

Step S112, judging whether the calling party software name is a preset first party playing software name, if not, determining the calling party playing software as a third party playing software:

After the calling party software name is obtained, whether the calling party software name is the preset first party playing software name is judged, the first party playing software name is the software name of the first party playing software called the virtual camera module, if the software names are the same, the virtual camera module is characterized as being called by the first party playing software, if the software names are different, the calling party playing software called the virtual camera module is characterized as third party playing software, and at the moment, the calling party playing software is determined as the third party playing software.

In this embodiment, through the comparison and verification of the convenient software names, whether the calling party playing software of the virtual camera module calling the first party playing software is the first party playing software or the third party playing software is determined efficiently, so that the monitoring efficiency of the third party playing software is improved, and the software use right and the software safety of the first party playing software are protected.

In one embodiment, please refer to fig. 3 and 5, the step of obtaining the software information of the third party playing software and intercepting the current gui obtaining screenshot includes the following steps:

Step S121, in response to the first-party live function usage event, determines a first-party live function corresponding to the first-party live function usage event:

When it is determined that the third party playing software invokes the first party playing software as a virtual camera, it will monitor whether the third party playing software uses the live function of the first party playing software, when the third party playing software uses any one of the first party live functions of the first party playing software, the first party function use event acting on the first party live function used by the third party playing software will be triggered, for example, please refer to fig. 3, an animation effect 307 displayed in a live stream playing window 303 shown in fig. 3, the animation effect 307 is the first party live function of the first party playing software about an animation effect, and when the third party playing software uses the first party live function about an animation effect to display the animation effect 307 in a live stream of the third party playing software, the shown function use event acting on the third party live function will be triggered.

Step S122, obtaining function usage information corresponding to the first party live broadcast function, so as to store the function usage information to a platform software protection library correspondingly:

The function use information is a use record of the third party playing software to use the first party live broadcast function, for example, the use times of the first party live broadcast function, the use time of the first party live broadcast function, and the like.

And acquiring the function use information of the third-party broadcast software using the first-party live broadcast function, so that the function use information, the software information, the interface screenshot and the third-party live broadcast characteristic information are stored in the platform software protection library as mapping relation data.

Step S123, obtaining software information of the third party playing software, and performing screenshot processing on a graphical user interface triggering the first party live broadcast function event to obtain a corresponding interface screenshot:

After responding to the first party live broadcast function use event, the software information of the third party live broadcast software is acquired, and a graphical user interface of the third party live broadcast function event used by the third party live broadcast software is subjected to screenshot so as to acquire an interface screenshot of the first party live broadcast function event used by the third party live broadcast software, as shown in fig. 3, the graphical user interface shown in fig. 3 is intercepted, and the animation special effects 307 of the first party live broadcast software are used by the third party live broadcast software in the graphical user interface.

In this embodiment, when the first-party live broadcast function of the first-party broadcast software is used by the third-party broadcast software, the function service condition of the third-party broadcast software using the first-party live broadcast function is recorded, and software information recording and interface screenshot processing are performed to protect the first-party live broadcast function used by the third-party broadcast software, and further protect the software security and the software use right of the first-party broadcast software.

In an embodiment, referring to fig. 3 and fig. 6, the step of identifying the third party live feature information included in the interface screenshot includes the following steps:

step S131, performing image preprocessing on the interface screenshot, and extracting an image feature vector of the interface screenshot:

Referring to fig. 3, the graphical user interface of the terminal device shown in fig. 3 is the interface screenshot, and after the interface screenshot shown in fig. 3 is obtained, image preprocessing is performed on the interface screenshot, so that the interface screenshot accords with an image input format of the image recognition model, and then the image feature vector of the interface screenshot is extracted.

Step S132, identifying a plurality of text messages displayed in the interface screenshot according to the image feature vector:

referring to fig. 3, in the text information in the interface screenshot shown in fig. 3, the software window name "third party playing software platform" of the software window 301 is generally a live broadcast platform to which the third party playing software belongs, and the host user name, the live broadcast room name and the live broadcast room number of the third party playing software are displayed in the live broadcast room information window 302, and besides identifying the text information in the software window 301, other windows and software names of software icons in the interface screenshot shown in fig. 3 may be identified generally, and therefore, in order to obtain the text information conforming to the third party live broadcast feature as the third party live broadcast feature information, a text classifier is required to be used to screen out the corresponding text information, as the software names of the software icons 304 shown in fig. 3 and the text information of the small windows in the menu bar 305.

Step S133, calling a text classifier, extracting text information meeting a third-party live broadcast feature text matching rule from each piece of text information, determining a text feature type corresponding to each piece of text information, and packaging the text information and the text feature type as a third-party live broadcast feature text into the third-party live broadcast feature information:

After identifying a plurality of text messages in the interface screenshot shown in fig. 3, invoking the text classifier, extracting text messages meeting a third-party live broadcast feature text matching rule from each text message, for example, screening a live broadcast platform name, a host user name, a live broadcast room name and a live broadcast room number of the third-party live broadcast software shown in fig. 3 through the text classifier, determining text feature types of the screened text messages, for example, screening a third-party live broadcast software platform, a host a, a live broadcast room a and 111 shown in fig. 3, wherein the text feature type of the third-party live broadcast software platform is the third-party live broadcast platform name, the text feature type of the host a is the third-party host user name, the text feature type of the live broadcast room a is the third-party live broadcast room name, and the text feature type of the 111 is the third-party live broadcast room number.

In this embodiment, the screenshot of the interface of the third party playing software for calling the virtual camera is identified and run through the image identification model, so as to identify text information in the graphical user interface for screening, and further, a live broadcast platform, a live broadcast user and a live broadcast room of the third party playing software for calling the virtual camera are accurately determined, so that the text information is kept in files, and the first party playing software for calling the virtual camera is protected.

In an embodiment, referring to fig. 3 and fig. 7, the step of identifying the third party live feature information included in the interface screenshot includes the following steps:

Step S131', performing image preprocessing on the interface screenshot, and extracting an image feature vector of the interface screenshot:

Step S132', identifying a face image included in the screenshot according to the image feature vector:

Referring to fig. 3, a live stream picture acquired by the third party playing software is displayed in a live stream acquisition and playing window 303 in the software window 301 shown in fig. 3, a face image of a host user is displayed in the live stream picture, and the face image displayed in the live stream picture played in the live stream acquisition and playing window 303 is identified through the image identification model.

Step S133', packaging the face image as a anchor face image into the third party live broadcast feature information:

and after the face image is identified from the interface screenshot, taking the face image as the anchor face image of a third-party live broadcast platform, and taking the anchor face image as the third-party live broadcast characteristic information.

In this embodiment, the interface screenshot is identified through the image identification model to identify the face image in the live broadcast stream picture displayed in the interface screenshot, and the face image is used as the live broadcast face image of the live broadcast user who performs the live broadcast service in the third-party live broadcast platform to be used as the third-party live broadcast feature information for storage in a file, so as to protect the first-party on-air software of the called virtual camera.

In one embodiment, please refer to fig. 8, the step of storing the software information, the screenshot and the third-party live feature information in a platform software protection library includes the following steps:

Step S141, obtaining third-party live broadcast platform information and a third-party anchor user name included in the third-party live broadcast feature information:

Because the platform software protection library is provided with a plurality of the anchor data tables, and each anchor data table is grouped and deployed by the type of the third party platform to which the anchor data table belongs to form each third party platform data table group, and then a third party platform is used as a father node, and the storage structure of each anchor user to which the anchor data table belongs is used as a child node correspondingly stores the third party live broadcast characteristic information, the software information and the interface screenshot corresponding to each anchor user, the third party live broadcast characteristic information needs to be analyzed first, and the third party live broadcast platform information and the third party anchor user name protected in the third party live broadcast characteristic information are obtained.

Step S142, inquiring a target third party platform data table group corresponding to the third party live broadcast platform information in the platform software protection library:

And according to the third-party live broadcast platform information obtained from the third-party live broadcast characteristic information in a resolving way, a target third-party platform data table group corresponding to the third-party live broadcast platform information is queried from the platform software protection library, and the third-party platform data table group comprises a plurality of third-party anchor data tables of anchor users to which the platform belongs.

Step S143, a target third party anchor data table corresponding to the third party anchor user name in the target third party platform data group is queried, and the software information, the interface screenshot and the third party live broadcast feature information are stored in the target third party anchor data table:

And inquiring the target third-party platform data table group, and inquiring a target third-party anchor data table corresponding to the third-party anchor user name from the target third-party platform data table, so as to store the software information, the interface screenshot and the third-party live broadcast characteristic information into the target third-party anchor data table.

In this embodiment, the platform software protection library uses the third-party live broadcast platform as a father node, and the anchor user uses the child node to store the software information, the interface screenshot and the third-party live broadcast feature information in a classified manner, so as to classify the software information, the interface screenshot and the third-party live broadcast feature information of the anchor user under different third-party live broadcast platforms, and facilitate the platform to query and call the information of the third-party broadcast software of the virtual camera module.

In one embodiment, referring to fig. 9, after the step of storing the software information, the screenshot and the third-party live feature information in the platform software protection library, the method includes:

step S15, obtaining a user identifier of a host player logged in first-party multicast software of a called virtual camera:

The first-party multicast software is generally available only after a host user logs in, so that when the virtual camera module of the first-party multicast software is monitored to be called, the host user identification of the host user logged in the first-party multicast software is acquired.

Step S16, inquiring target user account information corresponding to the anchor user identification in a user database corresponding to the first party multicast software:

The user database is a database of user account information of each platform user in the live broadcast platform to which the first broadcast software belongs, and the target user account information corresponding to the target anchor user identification is queried from the user database according to the anchor user identification.

Step S17, inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the platform software protection library according to the third-party anchor user name contained in the third-party live broadcast feature information:

After inquiring the target user account information of the anchor platform of the anchor user on the first party for opening the anchor software from the user database, inquiring a target third party anchor data table corresponding to the anchor user from the platform software protection library, identifying the third party live broadcast characteristic information from the interface screenshot through the image identification model, and inquiring the target third party anchor data table corresponding to the third party anchor user name from the platform software protection library according to the third party anchor user name protected by the third party live broadcast characteristic information.

Step S18, storing the target user account information into the target third party anchor data table:

And storing the account information of the target user into the target third-party anchor data table to determine the third-party anchor user who illegally calls the anchor user of the first-party anchor software in the live broadcast platform to which the third-party anchor software belongs, so as to be convenient for recording the behavior of the anchor user for using the first-party anchor software in an illegal manner.

In this embodiment, user account information of a platform anchor user of a virtual camera that illicitly invokes first-party anchor software is obtained to store the user account information into a third-party anchor data table in a platform software protection library, and the first-party platform anchor user and the third-party platform anchor user are mapped to perform non-compliance use software behavior recording.

In one embodiment, referring to fig. 10, after the step of storing the software information, the screenshot and the third-party live feature information in the platform software protection library, the method includes:

Step S15', obtaining a user identifier of a host player logged in first-party multicast software of a called virtual camera:

Step S16', inquiring the target user account information corresponding to the anchor user identification in the user database corresponding to the first party multicast software:

Step S17', updating the account information of the target user to modify the anchor user corresponding to the anchor user identifier into a forbidden anchor user:

And adding a blocking user identifier or modifying a blocking state to update the account information of the target user so as to modify a host user of the virtual camera module which does not call the first party playing software in a compliance manner into a blocking host user, thereby prohibiting the host user from logging in to use the first party playing software.

In this embodiment, the anchor user who does not call the virtual camera module of the first party's playing software in compliance is determined as the blocking anchor user, so that the anchor user is prevented from logging in to use the first party's playing software, and the software security and the software use right of the first party's playing software are protected.

Further, by performing the functionalization of each step in the method disclosed in the foregoing embodiments, a live broadcast monitoring device of the present application may be constructed, and according to this concept, please refer to fig. 11, in an exemplary embodiment, the device includes: the calling event response module 11 is used for responding to a virtual camera calling event and determining third-party broadcasting software triggering the virtual camera calling event; the interface screenshot obtaining module 12 is configured to obtain software information of the third party playing software, and intercept a current graphical user interface to obtain an interface screenshot; the interface screenshot identifying module 13 is used for identifying the third-party live broadcast characteristic information contained in the interface screenshot; and the third party information storage module 14 is used for correspondingly storing the software information, the interface screenshot and the third party live broadcast characteristic information into a platform software protection library.

In one embodiment, the calling event response module 11 includes: the software name acquisition sub-module is used for responding to the virtual camera calling event and acquiring the calling party software name of calling party multicast software triggering the virtual camera calling event; and the software name judging sub-module is used for judging whether the calling party software name is a preset first party playing software name, and if not, determining the calling party playing software as third party playing software.

In one embodiment, the interface screenshot capture module 12 includes: the live broadcast function determining submodule is used for responding to the first party live broadcast function use event and determining a first party live broadcast function corresponding to the first party live broadcast function use event; the use information acquisition sub-module is used for acquiring function use information corresponding to the first party live broadcast function so as to correspondingly store the function use information into a platform software protection library; and the interface screenshot obtaining sub-module is used for obtaining the software information of the third-party playing software, and carrying out screenshot processing on the graphical user interface triggering the first-party live broadcast function event to obtain a corresponding interface screenshot.

In one embodiment, the interface screenshot identifying module 13 includes: the feature vector acquisition sub-module is used for carrying out image preprocessing on the interface screenshot and extracting the image feature vector of the interface screenshot; the text information identification sub-module is used for identifying a plurality of text information displayed in the interface screenshot according to the image feature vector; and the text information classification sub-module is used for calling a text classifier, extracting text information which accords with a third-party live broadcast feature text matching rule from each text information, determining a text feature type corresponding to each text information, and packaging the text information and the text feature type as a third-party live broadcast feature text into the third-party live broadcast feature information.

In another embodiment, the interface screenshot identifying module 13 further includes: the feature vector acquisition sub-module is used for carrying out image preprocessing on the interface screenshot and extracting the image feature vector of the interface screenshot; the face image recognition sub-module is used for recognizing a face image contained in the interface screenshot according to the image feature vector; and the face image packaging sub-module is used for packaging the face image serving as the anchor face image into the third-party live broadcast characteristic information.

In one embodiment, the third party information storage module 14 includes: the characteristic information analysis sub-module is used for acquiring the third-party live broadcast platform information and the third-party anchor user name contained in the third-party live broadcast characteristic information; the data table group inquiring sub-module is used for inquiring a target third party platform data table group corresponding to the third party live broadcast platform information in the platform software protection library; and the information storage sub-module is used for inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the target third-party platform data group and storing the software information, the interface screenshot and the third-party live broadcast characteristic information into the target third-party anchor data table.

In order to solve the technical problem, the embodiment of the application also provides a computer device for running a computer program realized according to the live broadcast monitoring method. Referring specifically to fig. 12, fig. 12 is a basic structural block diagram of a computer device according to the present embodiment.

As shown in fig. 12, the internal structure of the computer device is schematically shown. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store a control information sequence, and when the computer readable instructions are executed by a processor, the processor can realize a live broadcast monitoring method. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may have stored therein computer readable instructions that, when executed by the processor, cause the processor to perform a live broadcast monitoring method. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in FIG. 12 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In this embodiment, the processor is configured to execute specific functions of each module/sub-module in the live broadcast monitoring device of the present application, and the memory stores program codes and various data required for executing the modules. The network interface is used for data transmission between the user terminal or the server. The memory in this embodiment stores program codes and data required for executing all modules/sub-modules in the live broadcast monitoring apparatus, and the server can call the program codes and data of the server to execute the functions of all sub-modules.

The present application also provides a non-volatile storage medium, wherein the live broadcast monitoring method is written as a computer program, and the computer program is stored in the storage medium in the form of computer readable instructions, and when the computer readable instructions are executed by one or more processors, the program means that the program runs in a computer, so that the one or more processors execute the steps of the live broadcast monitoring method in any embodiment.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).

In summary, the method monitors the third party software which calls the virtual camera of the first party software to perform live broadcast, and protects the software authority and the software security of the first party software by collecting the software information and the live broadcast characteristic information of the third party software.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

Those of skill in the art will appreciate that the various operations, methods, steps in the flow, acts, schemes, and alternatives discussed in the present application may be alternated, altered, combined, or eliminated. Further, other steps, means, or steps in a process having various operations, methods, or procedures discussed herein may be alternated, altered, rearranged, disassembled, combined, or eliminated. Further, steps, measures, schemes in the prior art with various operations, methods, flows disclosed in the present application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.

Claims

1. The live broadcast on-demand monitoring method is characterized by comprising the following steps of:

2. The method of claim 1, wherein the step of determining third party multicast software for triggering the virtual camera invocation event in response to the virtual camera invocation event comprises the steps of:

3. The method according to claim 1, wherein the step of obtaining software information of the third party playing software and intercepting the current gui obtaining screenshot comprises the steps of:

4. The method according to claim 1, wherein the step of identifying the third party live feature information contained in the interface screenshot includes the steps of:

5. The method according to claim 1, wherein the step of identifying the third party live feature information contained in the interface screenshot includes the steps of:

6. The method according to claim 1, wherein the step of storing the software information, the screenshot and the third live feature information in a platform software protection library includes the steps of:

7. The method of claim 1, wherein after the step of storing the software information, the interface screenshot, and the third party live feature information in the platform software protection library, the method further comprises:

Inquiring a target third-party anchor data table corresponding to the third-party anchor user name in the platform software protection library according to the third-party anchor user name contained in the third-party live broadcast feature information;

8. The method of claim 1, wherein after the step of storing the software information, the interface screenshot, and the third party live feature information in the platform software protection library, the method further comprises:

9. A live broadcast on-demand monitoring device, comprising:

10. An electronic device comprising a central processor and a memory, characterized in that the central processor is adapted to invoke a computer program stored in the memory for performing the steps of the method according to any of claims 1 to 8.

11. A non-volatile storage medium, characterized in that it stores in the form of computer readable instructions a computer program implemented according to the method of any one of claims 1 to 8, which when invoked by a computer, performs the steps comprised by the method.