CN112165638A - Video live broadcast processing method based on cloud computing and server - Google Patents
Video live broadcast processing method based on cloud computing and server Download PDFInfo
- Publication number
- CN112165638A CN112165638A CN202011025998.2A CN202011025998A CN112165638A CN 112165638 A CN112165638 A CN 112165638A CN 202011025998 A CN202011025998 A CN 202011025998A CN 112165638 A CN112165638 A CN 112165638A
- Authority
- CN
- China
- Prior art keywords
- video
- verification code
- dynamic
- video output
- live
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2407—Monitoring of transmitted content, e.g. distribution time, number of downloads
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25808—Management of client data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/845—Structuring of content, e.g. decomposing content into time segments
- H04N21/8456—Structuring of content, e.g. decomposing content into time segments by decomposing the content in the time domain, e.g. in time segments
Abstract
The application relates to a video live broadcast processing method and a server based on cloud computing. When the scheme is applied, the first dynamic verification code, the second dynamic verification code and the third dynamic verification code of the video output device can be determined through the distribution device under different conditions, the corresponding current live video stream is obtained through the distribution device when the video output device carries out live broadcasting based on the target live video stream, and then the current live video stream is segmented to obtain a plurality of video segments. And then verifying each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video verification result. And finally, determining whether the video output equipment has abnormal output behaviors or not based on the video verification result. Therefore, abnormal output behaviors of the video output equipment can be detected according to the dynamic verification codes of the video output equipment at different moments.
Description
Technical Field
The application relates to the technical field of live video, in particular to a live video processing method and a server based on cloud computing.
Background
With the rapid development of online video live broadcast, the field related to the live broadcast industry is more and more extensive. People can participate in the live broadcast interaction through a mobile phone, a tablet or other intelligent equipment, and convenience of information exchange and cultural communication among different regions is greatly improved. In the field of live video, along with the continuous expansion of the scale of a live video network, more and more intelligent devices can be accessed to the live video network to watch live video. However, some illegal viewing behaviors (such as recording, hacking and video stream stealing) can have negative effects on normal video live broadcasting. Therefore, how to detect the intelligent device at the video output end to avoid the illegal watching behavior is a technical problem to be solved urgently at the present stage.
Disclosure of Invention
The application provides a video live broadcast processing method and a server based on cloud computing, so as to solve the technical problems in the prior art.
In view of the first aspect of the embodiments of the present invention, there is provided a video live broadcast processing method based on cloud computing, which is applied to a server, where the server is connected to a streaming device in a communication manner, and the streaming device is further connected to a video output device in a communication manner, and the method includes at least:
when a live video request uploaded by the video output device is received through the distribution device, determining a first dynamic verification code of the video output device through the distribution device, and issuing a target live video stream to the distribution device based on the live video request, so that the distribution device issues the target live video stream to the video output device;
when the video output equipment carries out video live broadcasting based on the target live video stream, periodically determining a second dynamic verification code of the video output equipment through the shunting equipment;
detecting whether a target operation behavior exists in the video output device when video live broadcasting is carried out on the basis of the target live video stream through the shunting device; when the target operation behavior is detected to exist in the video output device during video live broadcasting through the distribution device, determining a third dynamic verification code of the video output device during generation of the target operation behavior through the distribution device; wherein the target operation behavior is generated by the video output device based on an operation instruction input by a user;
acquiring a current live video stream corresponding to the video output device during live video broadcasting through the distribution device, and segmenting the current live video stream to obtain a plurality of video segments; verifying each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video verification result corresponding to each video clip;
and determining whether the video output equipment has abnormal output behaviors or not according to each video check result.
In view of the second aspect of the embodiments of the present invention, there is also provided a server, including: the system comprises a processor, a memory and a network interface, wherein the memory and the network interface are connected with the processor; the network interface is connected with a nonvolatile memory in the server; when the processor is operated, the computer program is called from the nonvolatile memory through the network interface, and the computer program is operated through the memory so as to execute the method.
In view of the third aspect of the embodiment of the present invention, a readable storage medium applied to a computer is further provided, and a computer program is burned on the readable storage medium, and when the computer program runs in a memory of a server, the method is implemented.
When the cloud-computing-based live video processing method and the server are applied, the first dynamic verification code, the second dynamic verification code and the third dynamic verification code of the video output device can be determined through the distribution device under different conditions, the corresponding current live video stream of the video output device is obtained through the distribution device when the video output device is live broadcast based on the target live video stream, and then the current live video stream is segmented to obtain a plurality of video segments. And then verifying each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video verification result, and finally determining whether the video output equipment has abnormal output behaviors or not based on the video verification result. Therefore, abnormal output behaviors of the video output equipment can be detected according to the dynamic verification codes of the video output equipment at different moments.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a schematic diagram of a video live broadcast processing system based on cloud computing according to an exemplary embodiment of the present application.
Fig. 2 is a flowchart illustrating a video live broadcast processing method based on cloud computing according to an exemplary embodiment of the present application.
Fig. 3 is a block diagram illustrating an embodiment of a video live broadcast processing apparatus based on cloud computing according to an exemplary embodiment.
Fig. 4 is a hardware structure diagram of a server where the cloud computing-based live video processing apparatus of the present application is located.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
In order to better understand the cloud computing-based live video processing method and the server disclosed in the present application, an application scenario used in the present application embodiment is first described, as shown in fig. 1, where fig. 1 is a schematic view of a communication architecture of a cloud computing-based live video processing system 10 according to an exemplary embodiment of the present application. In fig. 1, the live video processing system 10 may include a server 20, a streaming device 30, and a video output device 40.
In one possible example, the server 20 may be the hub of the entire live video processing system 10 for controlling and scheduling the entire live video network. The distribution device 30 is configured to distribute the video stream delivered by the server 20 to different video output devices 40, and it can be understood that the distribution device 30 can share the work of delivering the video stream for the server 20, so that the server 20 can vacate a time slice resource to perform online video output detection on the video output device 40, and the timeliness of the online video output detection is ensured. Video output devices 40 include, but are not limited to, cell phones, tablets, laptops, and other smart devices capable of live interaction.
In another possible example, the server 20 may communicate with a plurality of splitting devices 30, and each splitting device 30 may communicate with a plurality of video output devices 40. In this way, when the server 20 distributes the video stream, only the communication interaction with the distribution device 30 is required, and the workload of the server 20 is greatly reduced.
On the basis, please refer to fig. 2 in combination, which is a schematic flowchart illustrating steps of a video live broadcast processing method based on cloud computing according to the present invention, where the method may be applied to the server 20 in fig. 1, and specifically may include the content described in the following steps.
In the following steps, the video output device 40 may generate different dynamic verification codes at different times, and the server 20 may detect the output behavior of the video output device 40 according to the dynamic verification codes. The dynamic verification code may be a character string encoded according to a set encoding format, for example, a binary character string, or a hexadecimal character string, which is not limited herein.
Step 210, when a live video request uploaded by the video output device is received through the distribution device, determining, through the distribution device, a first dynamic verification code of the video output device and issuing a target live video stream to the distribution device based on the live video request, so that the distribution device issues the target live video stream to the video output device.
Step 220, when the video output device performs video live broadcast based on the target live broadcast video stream, periodically determining, by the streaming device, a second dynamic verification code of the video output device.
In step 220, the distribution device 30 can determine the second dynamic verification code of the video output device 40 at regular time. For example, the distribution device 30 may determine the second dynamic verification code of the video output device 40 every ten seconds, or may determine the second dynamic verification code of the video output device 40 every thirty seconds. The specific time interval can be adjusted according to actual conditions, and is not further described herein.
Step 230, detecting, by the streaming device, whether a target operation behavior exists when the video output device performs video live broadcasting based on the target live broadcast video stream; when the target operation behavior is detected to exist in the video output device during video live broadcasting through the distribution device, determining a third dynamic verification code of the video output device during generation of the target operation behavior through the distribution device; wherein the target operation behavior is generated by the video output device based on an operation instruction input by a user.
Step 240, obtaining a current live video stream corresponding to the video output device during live video broadcast through the distribution device, and segmenting the current live video stream to obtain a plurality of video segments; and checking each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video checking result corresponding to each video clip.
In step 240, the current live video stream and the target live video stream may be the same or different, and in order to detect the output behavior of the video output device 40, the current live video stream of the video output device 40 needs to be analyzed.
And step 250, determining whether the video output equipment has abnormal output behaviors according to each video check result.
In step 250, the video verification result includes the matching degree between the current live video stream and the target live video stream, the bullet screen first keyword feature of the current live video stream, the device log parameter feature of the video output device 40 corresponding to the current live video stream, and the like. By analyzing each video verification result, whether the video output equipment has abnormal output information such as recorded broadcast, confusion, video stream stealing and the like can be determined. Wherein, the abnormal output behavior can be understood as abnormal live video behavior.
When the method described in the above step 210 to step 250 is applied, the first dynamic verification code, the second dynamic verification code, and the third dynamic verification code of the video output device can be determined through the distribution device under different conditions, and the current live video stream corresponding to the video output device when the video output device performs live broadcasting based on the target live video stream is obtained through the distribution device, so that the current live video stream is segmented to obtain a plurality of video segments. And then verifying each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video verification result, and finally determining whether the video output equipment has abnormal output behaviors or not based on the video verification result. Therefore, abnormal output behaviors of the video output equipment can be detected according to the dynamic verification codes of the video output equipment at different moments.
Therefore, corresponding measures can be taken in time when abnormal output behaviors are detected, and leakage and loss of video information or user information of a live broadcast system are avoided.
In a possible implementation manner, in step 240, the step of verifying each video segment based on the first dynamic verification code, the second dynamic verification code, and the third dynamic verification code to obtain a video verification result corresponding to each video segment may specifically include the following steps.
(1) And respectively determining a first dynamic weight corresponding to the first dynamic verification code, a second dynamic weight corresponding to the second dynamic verification code and a third dynamic weight corresponding to the third dynamic verification code.
(2) And calculating a weight difference value between the first dynamic weight, the second dynamic weight and the third dynamic weight.
(3) And correcting the first dynamic verification code and the second dynamic verification code according to the weight difference, and comparing the character consistency between the first dynamic verification code and the second dynamic verification code before and after the third dynamic verification code is corrected to obtain a comparison result.
(4) And determining a verification condition corresponding to each video clip on the basis of the comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code by adopting the verification condition to obtain verification logic information.
(5) And verifying the video coding set corresponding to each video segment based on the verification logic information to obtain a video verification result corresponding to each video segment.
Further, the specific implementation steps of step (1) to step (5) can be implemented by the methods described in step 2401 to step 2408.
Step 2401, determining a first weight difference between a first dynamic weight corresponding to the first dynamic verification code and a second dynamic weight corresponding to the second dynamic verification code, and a second weight difference between a second dynamic weight corresponding to the second dynamic verification code and a third dynamic weight corresponding to the third dynamic verification code; and the dynamic weight is used for representing the aging coefficient of different dynamic verification codes relative to the current moment.
It can be understood that the larger the dynamic weight is, the larger the aging factor characterizing the corresponding dynamic verification code relative to the current time is, and the closer the generation time of the corresponding dynamic verification code is to the current time is.
Step 2402, for the first dynamic verification code, correcting the first dynamic verification code by taking the first dynamic weight as a reference according to the first weight difference to obtain a fourth dynamic verification code; and for the second dynamic verification code, correcting the second dynamic verification code by taking the second dynamic weight as a reference according to the second weight difference value to obtain a fifth dynamic verification code.
Step 2403, respectively performing character consistency comparison on the first dynamic verification code and the second dynamic verification code, the first dynamic verification code and the fourth dynamic verification code, the second dynamic verification code and the third dynamic verification code, and the second dynamic verification code and the fifth dynamic verification code to obtain a first comparison result, a second comparison result, a third comparison result, and a fourth comparison result.
Step 2404, determining a first dissimilarity weight between the first comparison result and the second comparison result and a second dissimilarity weight between the third comparison result and the fourth comparison result; wherein the difference weight is used for representing the similarity between different comparison results.
Step 2405, determining whether the first difference weight and the second difference weight both fall within a preset weight interval.
Step 2406, if yes, determining a verification condition for verifying each video segment according to the first comparison result and the third comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code, and the third dynamic verification code according to the verification condition corresponding to each video segment to obtain verification logic information.
Step 2407, if not, respectively determining a first difference value and a second difference value between the first difference weight and the preset weight interval and the second difference weight and the preset weight interval; comparing the magnitude of the first difference and the second difference; when the first difference is smaller than the second difference, determining a verification condition for verifying each video segment according to the first comparison result and the second comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code according to the verification condition corresponding to each video segment to obtain verification logic information; when the first difference is larger than the second difference, determining a verification condition for verifying each video segment according to the third comparison result and the fourth comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code according to the verification condition corresponding to each video segment to obtain verification logic information.
Step 2408, verifying the video coding set corresponding to each video segment based on the verification logic information to obtain a video verification result corresponding to each video segment.
It can be understood that, by the method described in steps 2401 to 2408, the dynamic weights corresponding to the first dynamic verification code, the second dynamic verification code, and the third dynamic verification code can be analyzed, so as to take the aging factors of different dynamic verification codes into account, and then the verification logic information corresponding to each video segment under different conditions is determined through the above determination process, so that the video coding set corresponding to each video segment is verified based on the verification logic information, so as to accurately obtain the video verification result corresponding to each video segment.
In one possible implementation, the determining, by the streaming device, the first dynamic verification code of the video output device in step 210 specifically includes the following steps.
Step 2111, sending a first dynamic random number for instructing the video output device to perform cyclic redundancy check to the shunting device; wherein the first dynamic random number is randomly generated in the server according to the current time.
In this embodiment, a Cyclic Redundancy Check (CRC) is a hash function that generates a short fixed parity check code according to data such as a data packet or a computer file on a network, and is mainly used to detect or check errors that may occur after data transmission or storage. It is understood that whether the video output apparatus is in a normal operation state can be determined through the CRC check.
Step 2112, the first dynamic random number is issued to the video output device through the shunting device, so that the video output device performs cyclic redundancy check according to the first dynamic random number to obtain a first dynamic verification code.
Step 2113, acquiring the first dynamic verification code fed back by the video output device through the shunting device.
It can be understood that the active verification of the video output device is realized by the first dynamic random number randomly generated based on the current time, and the confidence of the obtained first dynamic random number can be improved.
In a possible example, in order to improve the rate at which the server issues the target live video stream to the streaming device, the server may determine the data packet of the target video stream in a multi-thread search manner based on the live video request, and for this reason, in step 210, the step of issuing the target live video stream to the streaming device based on the live video request may specifically include the content described in the following steps.
And 2121, determining a second occupancy rate of the remaining time slice resources in the server according to the first occupancy rate of the time slice resources of the pre-started video output detection thread.
In the present embodiment, the time slice resource is used to represent the computation amount of the server in a unit time (per second). Accordingly, the occupancy rate of the time slice resource is used for characterizing the operation efficiency of different threads of the server when the server is started.
Step 2122, acquiring first identification information of each data packet in a prestored database; the database is used for storing data packets of different video streams, the data packets are updated in the database in real time, and the first identification information corresponds to the data packets in the database one to one.
Step 2123, acquiring second identification information carried in the live video request; the second identification information is used for representing the identification information of the data packet corresponding to the target live broadcast video stream requested to be acquired by the video output device.
2124, starting N identification query threads according to the second occupancy rate, dividing the first identification information in the database into N groups of information sets, and introducing each group of information sets into one identification query thread; and simultaneously starting the N identification query threads and querying whether target identification information consistent with the second identification information exists through each identification query thread, wherein the target identification information is one of all the first identification information in the database, and N is a positive integer.
Step 2125, when the target identification information is found by any one of the N identification query threads, closing the N identification query threads; and sending the data packet corresponding to the target identification information to the distribution equipment.
It can be understood that, in specific implementation, time slice resources of the server are determined, and then a plurality of identification query threads are started on the premise of meeting the requirement of the remaining time slice resources, so that parallel query of target identification information is realized, time for querying the target identification information consistent with the second identification information can be greatly shortened, and further, the rate of issuing the target live broadcast video stream to the distribution equipment by the server is improved.
The step of periodically determining, by the shunting device, the second dynamic verification code of the video output device in step S220 may be determined by the following sub-steps.
And step 221, determining bullet screen information acquired by the video output device during video live broadcast through the shunting device.
Step 222, for the current bullet screen information in the bullet screen information, determining a bullet screen quality level of the current bullet screen information in a set time period based on a first keyword carried by the current bullet screen information in the set time period and a second keyword carried by each bullet screen information in the set time period; and the quality grade of the bullet screen is used for representing the safety of the bullet screen information received by the video output equipment.
Step 223, determining a bullet screen quality change coefficient of the current bullet screen information between two adjacent set time periods according to the bullet screen quality grades of the current bullet screen information in the two adjacent set time periods, and determining whether the current bullet screen information is abnormal bullet screen information based on the bullet screen quality change coefficient.
Step 224, if yes, shortening the preset timing duration to obtain the current timing duration; if not, prolonging the preset timing duration to obtain the current timing duration; and periodically determining a second dynamic verification code of the video output equipment through the shunting equipment according to the current timing duration, and returning barrage information obtained when the video output equipment is determined to carry out video live broadcast through the shunting equipment when the second dynamic verification code is obtained every time.
In specific implementation, through the content described in the above steps 221 to 224, keyword extraction can be performed on the barrage information received by the video output device, and whether the video output device receives an abnormal barrage can be determined. In this way, the preset set time length can be adjusted based on the judgment result, so that the self-adaptive acquisition of the second dynamic verification code is realized. By the method, the real-time performance and the reliability of the acquired second dynamic verification code can be ensured.
In one example, the current live video stream may be determined by: and enabling the shunting equipment to acquire the video signal output by the video output equipment, enabling the shunting equipment to transcode the video signal into multi-frame video codes, and acquiring the multi-frame video codes sent by the shunting equipment.
It can be understood that the server can be ensured to quickly acquire the multi-frame video code for representing the current live video stream by transcoding the video signal output by the video output device through the shunting device.
On the basis, segmenting the current live video stream to obtain a plurality of video segments may specifically include the content described in the following steps.
(1) And determining the correlation coefficient of every two adjacent frames of video coding in the multi-frame video coding.
In this embodiment, the correlation coefficient may be obtained by the similarity between each two adjacent frames of video coding. Further, the correlation coefficient can be expressed by a value between 0 and 1.
(2) And integrating a plurality of continuous video codes with the correlation coefficient larger than the set value into one video segment.
For example, the video in the present embodiment is encoded into 10 frames. The video clips obtained by the segmentation are 3 segments, and each video clip sequentially comprises: the first to the second frames 1 to 4, 5 to 6, 6 to 9 and 10.
By analyzing the size of the correlation coefficient of each two adjacent frames of video coding, the accuracy of video segmentation and the continuity and integrity of each video segment can be ensured.
In a more specific embodiment, the step of determining whether the video output device has abnormal output behavior according to each video verification result described in step 250 specifically includes the following steps.
Step 251, determining a first number of video verification results for representing that the video verification result is abnormal and a second number for representing that the video verification result is normal.
Step 252, determining whether the first number is greater than the second number; if so, determining that the video output equipment has abnormal output behaviors, and if not, determining that the video output equipment does not have the abnormal output behaviors.
When the contents described in the above steps 251 to 252 are applied, each video verification result can be analyzed, so that the problem of low accuracy caused by a cutting judgment mode is avoided. It is understood that, when the number of video verification results is larger, the confidence of determining whether the video output device has abnormal output behavior through the above steps is higher. Generally, live video online usually outputs a live video stream for a long time, so that the number of video clips is large, and in this case, by the above method, whether the video output device has abnormal output behavior can be determined more accurately and reliably.
On the basis of the above, please refer to fig. 3 in combination, a video live broadcast processing apparatus 200 based on cloud computing is provided, and the apparatus is described in detail as follows.
A1. The utility model provides a live video processing apparatus based on cloud calculates, is applied to the server, server and shunting device communication connection, shunting device still with video output device communication connection, the device includes at least:
the video issuing module 201 is configured to, when a live video request uploaded by the video output device is received through the distribution device, determine, through the distribution device, a first dynamic verification code of the video output device and issue a target live video stream to the distribution device based on the live video request, so that the distribution device issues the target live video stream to the video output device.
A verification code obtaining module 202, configured to periodically determine, by the streaming device, a second dynamic verification code of the video output device when the video output device performs video live broadcast based on the target live broadcast video stream.
An operation detection module 203, configured to detect, by the streaming device, whether a target operation behavior exists when the video output device performs video live broadcasting based on the target live video stream; when the target operation behavior is detected to exist in the video output device during video live broadcasting through the distribution device, determining a third dynamic verification code of the video output device during generation of the target operation behavior through the distribution device; wherein the target operation behavior is generated by the video output device based on an operation instruction input by a user.
The video verification module 204 is configured to obtain, through the distribution device, a current live video stream corresponding to the video output device during live video broadcasting, and segment the current live video stream to obtain a plurality of video segments; and checking each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video checking result corresponding to each video clip.
And the output detection module 205 is configured to determine whether the video output device has an abnormal output behavior according to each video verification result.
A2. The apparatus of a1, the video verification module 204, configured to: and enabling the shunting equipment to acquire the video signal output by the video output equipment, enabling the shunting equipment to transcode the video signal into multi-frame video codes, and acquiring the multi-frame video codes sent by the shunting equipment.
A3. The apparatus of a2, the video verification module 204, configured to: determining the correlation coefficient of every two adjacent frames of video coding in the multi-frame video coding; and integrating a plurality of continuous video codes with the correlation coefficient larger than the set value into one video segment.
A4. The apparatus of any of a1-3, the output detection module 205 to: determining a first number of video verification results for representing that the video verification results are abnormal and a second number for representing that the video verification results are normal; judging whether the first number is larger than the second number; if so, determining that the video output equipment has abnormal output behaviors, and if not, determining that the video output equipment does not have the abnormal output behaviors.
A5. The apparatus of a1, the video verification module 204, configured to:
respectively determining a first dynamic weight corresponding to the first dynamic verification code, a second dynamic weight corresponding to the second dynamic verification code and a third dynamic weight corresponding to the third dynamic verification code;
calculating a weight difference value between the first dynamic weight, the second dynamic weight and the third dynamic weight;
correcting the first dynamic verification code and the second dynamic verification code according to the weight difference, and comparing the character consistency between the first dynamic verification code and the second dynamic verification code before and after the third dynamic verification code is corrected to obtain a comparison result;
determining a verification condition corresponding to each video clip on the basis of the comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code by adopting the verification condition to obtain verification logic information;
and verifying the video coding set corresponding to each video segment based on the verification logic information to obtain a video verification result corresponding to each video segment.
A6. According to the apparatus of a1, the video delivery module 201 is configured to:
sending a first dynamic random number used for indicating the video output equipment to carry out cyclic redundancy check to the shunting equipment; wherein the first dynamic random number is randomly generated in the server according to the current moment;
issuing the first dynamic random number to the video output equipment through the shunting equipment, and enabling the video output equipment to perform cyclic redundancy check according to the first dynamic random number to obtain a first dynamic verification code;
and acquiring a first dynamic verification code fed back by the video output equipment through the shunting equipment.
A7. According to the apparatus of a6, the video delivery module 201 is configured to:
determining a second occupancy rate of the residual time slice resources in the server according to the first occupancy rate of the time slice resources of the pre-started video output detection thread;
acquiring first identification information of each data packet in a prestored database; the database is used for storing data packets of different video streams, the data packets are updated in the database in real time, and the first identification information corresponds to the data packets in the database one by one;
acquiring second identification information carried in the live video request; the second identification information is used for representing identification information of a data packet corresponding to a target live video stream requested to be acquired by the video output device;
starting N identification query threads according to the second occupancy rate, dividing the first identification information in the database into N groups of information sets, and introducing each group of information sets into one identification query thread; starting the N identification query threads simultaneously and querying whether target identification information consistent with the second identification information exists through each identification query thread, wherein the target identification information is one of all first identification information in the database, and N is a positive integer;
when the target identification information exists through any one identification query thread in the N identification query threads, the N identification query threads are closed; and sending the data packet corresponding to the target identification information to the distribution equipment.
A8. According to the method of a1, the verification code obtaining module 202 is configured to:
determining bullet screen information acquired by the video output device during video live broadcast through the shunting device;
determining the bullet screen quality grade of the current bullet screen information in a set time period based on a first keyword carried by the current bullet screen information in the set time period and a second keyword carried by each bullet screen information in the set time period aiming at the current bullet screen information in the bullet screen information; the quality grade of the bullet screen is used for representing the safety of bullet screen information received by the video output equipment;
determining a bullet screen quality change coefficient of the current bullet screen information between two adjacent set time periods according to the bullet screen quality grades of the current bullet screen information in the two adjacent set time periods, and determining whether the current bullet screen information is abnormal bullet screen information or not based on the bullet screen quality change coefficient;
if so, shortening the preset timing duration to obtain the current timing duration; if not, prolonging the preset timing duration to obtain the current timing duration; and periodically determining a second dynamic verification code of the video output equipment through the shunting equipment according to the current timing duration, and returning barrage information obtained when the video output equipment is determined to carry out video live broadcast through the shunting equipment when the second dynamic verification code is obtained every time.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, wherein the modules described as separate parts may or may not be physically separate, and the parts displayed as modules may or may not be physical modules, may be located in one place, or may be distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
On the basis of the above, please refer to fig. 4 in combination, there is also provided a server 20, including: a processor 21, and a memory 22 and a network interface 23 connected to the processor 21, wherein the network interface 23 is connected to a nonvolatile memory 24 in the server 21; the processor 20, when running, retrieves a computer program from the non-volatile memory 24 via the network interface 23 and runs the computer program via the memory 22 to perform the method described above.
On the basis of the above, a readable storage medium applied to a computer is also provided, and the readable storage medium is burned with a computer program, and the computer program realizes the above method when running in the memory 22 of the server 20.
Further, the embodiment of the present invention also provides the following contents.
B1. The utility model provides a live video processing system based on cloud calculates, the system includes server, splitter and video output device communication connection, splitter respectively with the server with video output device communication, the server in with splitter is the one-to-many connection, splitter with video output device is the one-to-many connection.
And the distribution equipment is used for forwarding the live video request to the server when receiving the live video request uploaded by the video output equipment.
And the server is used for determining a first dynamic verification code of the video output equipment through the distribution equipment and issuing a target live video stream to the distribution equipment based on the live video request.
And the distribution equipment is used for sending the target live video stream to the video output equipment.
The server is used for periodically determining a second dynamic verification code of the video output device through the shunting device when the video output device carries out video live broadcasting based on the target live video stream.
The server is used for detecting whether a target operation behavior exists in the video output device when video live broadcasting is carried out on the basis of the target live broadcast video stream through the shunting device; when the target operation behavior is detected to exist in the video output device during video live broadcasting through the distribution device, determining a third dynamic verification code of the video output device during generation of the target operation behavior through the distribution device; wherein the target operation behavior is generated by the video output device based on an operation instruction input by a user.
The server is used for acquiring a current live broadcast video stream corresponding to the video output device during video live broadcast through the shunting device, and segmenting the current live broadcast video stream to obtain a plurality of video segments; and checking each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video checking result corresponding to each video clip.
And the server is used for determining whether the video output equipment has abnormal output behaviors or not according to each video verification result.
B2. According to the system in accordance with B1, the streaming device is configured to obtain a video signal output by the video output device, and transcode the video signal into a multi-frame video code; and the server is used for acquiring the multi-frame video codes sent by the shunting equipment.
B3. The system of B2, the server, to: determining the correlation coefficient of every two adjacent frames of video coding in the multi-frame video coding; and integrating a plurality of continuous video codes with the correlation coefficient larger than the set value into one video segment.
B4. The system of any one of B1-B3, the server to:
determining a first number of video verification results for representing that the video verification results are abnormal and a second number for representing that the video verification results are normal;
judging whether the first number is larger than the second number; if so, determining that the video output equipment has abnormal output behaviors, and if not, determining that the video output equipment does not have the abnormal output behaviors.
B5. The system of B1, the server, to:
respectively determining a first dynamic weight corresponding to the first dynamic verification code, a second dynamic weight corresponding to the second dynamic verification code and a third dynamic weight corresponding to the third dynamic verification code;
calculating a weight difference value between the first dynamic weight, the second dynamic weight and the third dynamic weight;
correcting the first dynamic verification code and the second dynamic verification code according to the weight difference, and comparing the character consistency between the first dynamic verification code and the second dynamic verification code before and after the third dynamic verification code is corrected to obtain a comparison result;
determining a verification condition corresponding to each video clip on the basis of the comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code by adopting the verification condition to obtain verification logic information;
and verifying the video coding set corresponding to each video segment based on the verification logic information to obtain a video verification result corresponding to each video segment.
B6. The system of B1, the server, to:
sending a first dynamic random number used for indicating the video output equipment to carry out cyclic redundancy check to the shunting equipment; wherein the first dynamic random number is randomly generated in the server according to the current moment;
issuing the first dynamic random number to the video output equipment through the shunting equipment, and enabling the video output equipment to perform cyclic redundancy check according to the first dynamic random number to obtain a first dynamic verification code;
and acquiring a first dynamic verification code fed back by the video output equipment through the shunting equipment.
B7. The system of B6, the server, to:
determining a second occupancy rate of the residual time slice resources in the server according to the first occupancy rate of the time slice resources of the pre-started video output detection thread;
acquiring first identification information of each data packet in a prestored database; the database is used for storing data packets of different video streams, the data packets are updated in the database in real time, and the first identification information corresponds to the data packets in the database one by one;
acquiring second identification information carried in the live video request; the second identification information is used for representing identification information of a data packet corresponding to a target live video stream requested to be acquired by the video output device;
starting N identification query threads according to the second occupancy rate, dividing the first identification information in the database into N groups of information sets, and introducing each group of information sets into one identification query thread; starting the N identification query threads simultaneously and querying whether target identification information consistent with the second identification information exists through each identification query thread, wherein the target identification information is one of all first identification information in the database, and N is a positive integer;
when the target identification information exists through any one identification query thread in the N identification query threads, the N identification query threads are closed; and sending the data packet corresponding to the target identification information to the distribution equipment.
B8. The system of B1, the server, to:
determining bullet screen information acquired by the video output device during video live broadcast through the shunting device;
determining the bullet screen quality grade of the current bullet screen information in a set time period based on a first keyword carried by the current bullet screen information in the set time period and a second keyword carried by each bullet screen information in the set time period aiming at the current bullet screen information in the bullet screen information; the quality grade of the bullet screen is used for representing the safety of bullet screen information received by the video output equipment;
determining a bullet screen quality change coefficient of the current bullet screen information between two adjacent set time periods according to the bullet screen quality grades of the current bullet screen information in the two adjacent set time periods, and determining whether the current bullet screen information is abnormal bullet screen information or not based on the bullet screen quality change coefficient;
if so, shortening the preset timing duration to obtain the current timing duration; if not, prolonging the preset timing duration to obtain the current timing duration; and periodically determining a second dynamic verification code of the video output equipment through the shunting equipment according to the current timing duration, and returning barrage information obtained when the video output equipment is determined to carry out video live broadcast through the shunting equipment when the second dynamic verification code is obtained every time.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (9)
1. A video live broadcast processing method based on cloud computing is applied to a server, the server is in communication connection with a distribution device, the distribution device is also in communication connection with a video output device, and the method at least comprises the following steps:
when a live video request uploaded by the video output device is received through the distribution device, determining a first dynamic verification code of the video output device through the distribution device, and issuing a target live video stream to the distribution device based on the live video request, so that the distribution device issues the target live video stream to the video output device;
when the video output equipment carries out video live broadcasting based on the target live video stream, periodically determining a second dynamic verification code of the video output equipment through the shunting equipment;
detecting whether a target operation behavior exists in the video output device when video live broadcasting is carried out on the basis of the target live video stream through the shunting device; when the target operation behavior is detected to exist in the video output device during video live broadcasting through the distribution device, determining a third dynamic verification code of the video output device during generation of the target operation behavior through the distribution device; wherein the target operation behavior is generated by the video output device based on an operation instruction input by a user;
acquiring a current live video stream corresponding to the video output device during live video broadcasting through the distribution device, and segmenting the current live video stream to obtain a plurality of video segments; verifying each video clip based on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code to obtain a video verification result corresponding to each video clip;
determining whether the video output equipment has abnormal output behaviors or not according to each video check result; wherein the video verification result comprises: the matching degree of the current live broadcast video stream and the target live broadcast video stream, the bullet screen first keyword characteristics of the current live broadcast video stream and the device log parameter characteristics of the video output device corresponding to the current live broadcast video stream.
2. The method according to claim 1, wherein the step of obtaining, by the streaming device, a current live video stream corresponding to the video output device during video live broadcasting specifically includes: and enabling the shunting equipment to acquire the video signal output by the video output equipment, enabling the shunting equipment to transcode the video signal into multi-frame video codes, and acquiring the multi-frame video codes sent by the shunting equipment.
3. The method according to claim 2, wherein the step of segmenting the current live video stream to obtain a plurality of video segments specifically comprises:
determining the correlation coefficient of every two adjacent frames of video coding in the multi-frame video coding;
and integrating a plurality of continuous video codes with the correlation coefficient larger than the set value into one video segment.
4. The method according to any one of claims 1 to 3, wherein the step of determining whether the video output device has an abnormal output behavior according to each video verification result specifically comprises:
determining a first number of video verification results for representing that the video verification results are abnormal and a second number for representing that the video verification results are normal;
judging whether the first number is larger than the second number; if so, determining that the video output equipment has abnormal output behaviors, and if not, determining that the video output equipment does not have the abnormal output behaviors.
5. The method according to claim 1, wherein the step of verifying each video segment based on the first dynamic verification code, the second dynamic verification code, and the third dynamic verification code to obtain a video verification result corresponding to each video segment specifically includes:
respectively determining a first dynamic weight corresponding to the first dynamic verification code, a second dynamic weight corresponding to the second dynamic verification code and a third dynamic weight corresponding to the third dynamic verification code;
calculating a weight difference value between the first dynamic weight, the second dynamic weight and the third dynamic weight;
correcting the first dynamic verification code and the second dynamic verification code according to the weight difference, and comparing the character consistency between the first dynamic verification code and the second dynamic verification code before and after the third dynamic verification code is corrected to obtain a comparison result;
determining a verification condition corresponding to each video clip on the basis of the comparison result, and performing verification code character conversion on the first dynamic verification code, the second dynamic verification code and the third dynamic verification code by adopting the verification condition to obtain verification logic information;
and verifying the video coding set corresponding to each video segment based on the verification logic information to obtain a video verification result corresponding to each video segment.
6. The method according to claim 1, wherein the step of determining, by the streaming device, the first dynamic authentication code of the video output device specifically includes:
sending a first dynamic random number used for indicating the video output equipment to carry out cyclic redundancy check to the shunting equipment; wherein the first dynamic random number is randomly generated in the server according to the current moment;
issuing the first dynamic random number to the video output equipment through the shunting equipment, and enabling the video output equipment to perform cyclic redundancy check according to the first dynamic random number to obtain a first dynamic verification code;
and acquiring a first dynamic verification code fed back by the video output equipment through the shunting equipment.
7. The method according to claim 6, wherein the step of issuing a target live video stream to the streaming device based on the live video request specifically includes:
determining a second occupancy rate of the residual time slice resources in the server according to the first occupancy rate of the time slice resources of the pre-started video output detection thread;
acquiring first identification information of each data packet in a prestored database; the database is used for storing data packets of different video streams, the data packets are updated in the database in real time, and the first identification information corresponds to the data packets in the database one by one;
acquiring second identification information carried in the live video request; the second identification information is used for representing identification information of a data packet corresponding to a target live video stream requested to be acquired by the video output device;
starting N identification query threads according to the second occupancy rate, dividing the first identification information in the database into N groups of information sets, and introducing each group of information sets into one identification query thread; starting the N identification query threads simultaneously and querying whether target identification information consistent with the second identification information exists through each identification query thread, wherein the target identification information is one of all first identification information in the database, and N is a positive integer;
when the target identification information exists through any one identification query thread in the N identification query threads, the N identification query threads are closed; and sending the data packet corresponding to the target identification information to the distribution equipment.
8. The method according to claim 1, wherein the step of periodically determining, by the streaming device, the second dynamic authentication code of the video output device specifically includes:
determining bullet screen information acquired by the video output device during video live broadcast through the shunting device;
determining the bullet screen quality grade of the current bullet screen information in a set time period based on a first keyword carried by the current bullet screen information in the set time period and a second keyword carried by each bullet screen information in the set time period aiming at the current bullet screen information in the bullet screen information; the quality grade of the bullet screen is used for representing the safety of bullet screen information received by the video output equipment;
determining a bullet screen quality change coefficient of the current bullet screen information between two adjacent set time periods according to the bullet screen quality grades of the current bullet screen information in the two adjacent set time periods, and determining whether the current bullet screen information is abnormal bullet screen information or not based on the bullet screen quality change coefficient;
if so, shortening the preset timing duration to obtain the current timing duration; if not, prolonging the preset timing duration to obtain the current timing duration; and periodically determining a second dynamic verification code of the video output equipment through the shunting equipment according to the current timing duration, and returning barrage information obtained when the video output equipment is determined to carry out video live broadcast through the shunting equipment when the second dynamic verification code is obtained every time.
9. A server, comprising:
a processor, and
a memory and a network interface connected with the processor;
the network interface is connected with a nonvolatile memory in the server;
the processor, when running, retrieves a computer program from the non-volatile memory via the network interface and runs the computer program via the memory to perform the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011025998.2A CN112165638A (en) | 2020-04-10 | 2020-04-10 | Video live broadcast processing method based on cloud computing and server |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010276497.5A CN111464835B (en) | 2020-04-10 | 2020-04-10 | Online video output detection method based on dynamic verification and server |
| CN202011025998.2A CN112165638A (en) | 2020-04-10 | 2020-04-10 | Video live broadcast processing method based on cloud computing and server |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010276497.5A Division CN111464835B (en) | 2020-04-10 | 2020-04-10 | Online video output detection method based on dynamic verification and server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112165638A true CN112165638A (en) | 2021-01-01 |
Family
ID=71682217
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010276497.5A Active CN111464835B (en) | 2020-04-10 | 2020-04-10 | Online video output detection method based on dynamic verification and server |
| CN202011025998.2A Withdrawn CN112165638A (en) | 2020-04-10 | 2020-04-10 | Video live broadcast processing method based on cloud computing and server |
| CN202011025997.8A Withdrawn CN112165637A (en) | 2020-04-10 | 2020-04-10 | Video stream detection device and method applied to video live broadcast |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010276497.5A Active CN111464835B (en) | 2020-04-10 | 2020-04-10 | Online video output detection method based on dynamic verification and server |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011025997.8A Withdrawn CN112165637A (en) | 2020-04-10 | 2020-04-10 | Video stream detection device and method applied to video live broadcast |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN111464835B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113365096A (en) * | 2021-07-05 | 2021-09-07 | 湖南快乐阳光互动娱乐传媒有限公司 | Live broadcast room warm-up method and device, storage medium and electronic equipment |
| CN113434561A (en) * | 2021-06-24 | 2021-09-24 | 北京金山云网络技术有限公司 | Live broadcast data verification method and system, electronic device and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9612777B2 (en) * | 2013-06-12 | 2017-04-04 | Samsung Electronics Co., Ltd. | Pull printing method, apparatus and system |
| CN107665300A (en) * | 2017-09-18 | 2018-02-06 | 联想(北京)有限公司 | Output intent, server and outlet terminal |
-
2020
- 2020-04-10 CN CN202010276497.5A patent/CN111464835B/en active Active
- 2020-04-10 CN CN202011025998.2A patent/CN112165638A/en not_active Withdrawn
- 2020-04-10 CN CN202011025997.8A patent/CN112165637A/en not_active Withdrawn
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113434561A (en) * | 2021-06-24 | 2021-09-24 | 北京金山云网络技术有限公司 | Live broadcast data verification method and system, electronic device and storage medium |
| CN113365096A (en) * | 2021-07-05 | 2021-09-07 | 湖南快乐阳光互动娱乐传媒有限公司 | Live broadcast room warm-up method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111464835B (en) | 2021-01-15 |
| CN112165637A (en) | 2021-01-01 |
| CN111464835A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109756746B (en) | Video auditing method, device, server and storage medium | |
| CN111770355B (en) | Media server determination method, device, server and storage medium | |
| CN111464835B (en) | Online video output detection method based on dynamic verification and server | |
| CN107426253B (en) | Data verification method and client | |
| US20210120292A1 (en) | Video playback quality detection method and apparatus | |
| CN108260015B (en) | Voting data processing method and device and electronic equipment | |
| CN104348578B (en) | The method and device of data processing | |
| US10972703B2 (en) | Method, device, and storage medium for processing webcam data | |
| CN112311902B (en) | File sending method and device based on micro-service | |
| CN110889096A (en) | Login method, login device, computer equipment and storage medium | |
| CN114647698A (en) | Data synchronization method and device and computer storage medium | |
| CN112131002A (en) | Data management method and device | |
| CN112969172B (en) | Communication flow control method based on cloud mobile phone | |
| US11328095B2 (en) | Peceptual video fingerprinting | |
| CN115190083B (en) | Data transmission method and device applied to intelligent equipment | |
| CN115664992A (en) | Network operation data processing method and device, electronic equipment and medium | |
| CN111432271B (en) | Multi-screen interaction method and system | |
| CN113852835A (en) | Live broadcast audio processing method and device, electronic equipment and storage medium | |
| CN111435346A (en) | Offline data processing method, device and equipment | |
| CN115209188B (en) | Detection method, device, server and storage medium for simultaneous live broadcast of multiple accounts | |
| CN112822515B (en) | Video file replacement method and device | |
| CN112948408B (en) | Data processing method and device | |
| CN110674139B (en) | Information processing method, system, resource management system and storage medium | |
| US20220256238A1 (en) | Content-modification system with user experience analysis feature | |
| CN114741323A (en) | Block chain performance testing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210101 |