CN112233196B - Live broadcasting room green screen detection method, device, equipment and storage medium - Google Patents
Live broadcasting room green screen detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112233196B CN112233196B CN202011218258.0A CN202011218258A CN112233196B CN 112233196 B CN112233196 B CN 112233196B CN 202011218258 A CN202011218258 A CN 202011218258A CN 112233196 B CN112233196 B CN 112233196B
- Authority
- CN
- China
- Prior art keywords
- live
- target
- information
- green screen
- broadcast
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 71
- 238000000034 method Methods 0.000 claims abstract description 68
- 238000012545 processing Methods 0.000 claims abstract description 36
- 230000008569 process Effects 0.000 claims abstract description 33
- 238000000605 extraction Methods 0.000 claims abstract description 10
- 238000012360 testing method Methods 0.000 claims description 77
- 230000008439 repair process Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 230000006978 adaptation Effects 0.000 claims description 7
- 238000009432 framing Methods 0.000 claims 1
- 230000001976 improved effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 16
- 230000006870 function Effects 0.000 description 14
- 238000005516 engineering process Methods 0.000 description 8
- 230000005540 biological transmission Effects 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 241000209140 Triticum Species 0.000 description 2
- 235000021307 Triticum Nutrition 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000011835 investigation Methods 0.000 description 2
- 230000007935 neutral effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000008093 supporting effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The application relates to a method, a device, equipment and a storage medium for detecting a green screen in a live broadcasting room. The method comprises the following steps: acquiring a live broadcast picture image in a live broadcast process of a target live broadcast room; performing HSB color mode extraction processing on the live image to obtain first HSB information corresponding to the pixel points of the live image; determining a first target pixel point of which the H value is in a preset numerical range in first HSB information from pixel points of live-broadcast picture images; and if the duty ratio information of the first target pixel point and the pixel point of the live image meets the preset condition, determining that the live image is a green screen image. The green screen detection efficiency of the live broadcasting room is improved.
Description
Technical Field
The present application relates to the field of video processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting a green screen in a live broadcast room.
Background
The existing green screen detection of the video is generally carried out by a tester, if the green screen phenomenon is encountered in the test process, the green screen phenomenon cannot be reserved timely, so that the investigation of the green screen problem cannot be effectively carried out, the green screen repair cannot be effectively solved timely, and the application scene of the green screen detection in a manual mode is limited. The existing method for automatically identifying the green screen image does not relate to a live scene, and whether the image is green is generally judged through an RGB color mode, and the RGB color mode needs three-channel color synthesis to be used for judging the color, so that the real-time performance is poor.
In addition, at present, the application of machine learning is wider, the machine learning can realize the high efficiency of green screen detection, but the training cost of the machine learning is higher.
Disclosure of Invention
In view of the above technical problems, the application provides a method, a device, equipment and a storage medium for detecting a green screen in a live broadcasting room.
According to one aspect of the application, there is provided a green screen detection method for a live broadcast room, including:
acquiring a live broadcast picture image in a live broadcast process of a target live broadcast room;
performing hue-saturation-brightness HSB color mode extraction processing on the live-broadcast picture image to obtain first HSB information corresponding to a pixel point of the live-broadcast picture image;
determining a first target pixel point of which the H value is in a preset numerical range in the first HSB information from pixel points of the live broadcast picture image;
and if the duty ratio information of the first target pixel point and the pixel point of the live broadcast picture image meets a preset condition, determining that the live broadcast picture image is a green screen image.
According to another aspect of the present application, there is provided a green screen detection apparatus for a live broadcasting room, including:
the live broadcast picture image acquisition module is used for acquiring live broadcast picture images in the live broadcast process of the target live broadcast room;
The first HSB information acquisition module is used for extracting hue-saturation-brightness HSB color modes of the live broadcast picture image to obtain first HSB information corresponding to pixel points of the live broadcast picture image;
the first target pixel point determining module is used for determining a first target pixel point of which the H value in the first HSB information is in a preset numerical range from the pixel points of the live broadcast picture image;
and the green screen image determining module is used for determining that the live broadcast picture image is a green screen image if the duty ratio information of the first target pixel point and the pixel point of the live broadcast picture image meets the preset condition.
According to another aspect of the present application, there is provided a green screen detection apparatus for a live broadcasting room, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above method.
According to another aspect of the present application there is provided a non-transitory computer readable storage medium having stored thereon computer program instructions, wherein the computer program instructions when executed by a processor implement the above-described method.
By acquiring live broadcast picture images in the live broadcast process of the live broadcast room and detecting the live broadcast picture images through the green screen, the automation of the green screen detection of the live broadcast room is realized, and the green screen detection efficiency of the live broadcast room can be improved. And determining whether the live-broadcast picture image is a green screen image or not according to the H value in the first HSB information corresponding to the pixel point of the live-broadcast picture image, and not needing to carry out color channel synthesis, thereby further improving the green screen detection efficiency and providing a guarantee for the timeliness of the green screen restoration of the live-broadcast room.
Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments, features and aspects of the application and together with the description, serve to explain the principles of the application.
Fig. 1 shows a schematic diagram of an application system according to an embodiment of the application.
Fig. 2 shows a flowchart of a live room green screen detection method according to an embodiment of the application.
Fig. 3 shows a flowchart of a live room green screen detection method according to an embodiment of the application.
Fig. 4 is a flowchart of a method for performing a green screen repair process in a live room based on live type information and log information according to an embodiment of the present application.
Fig. 5 shows a flowchart of a live room green screen detection method according to an embodiment of the present application.
Fig. 6 shows a flowchart of a live room green screen detection method according to an embodiment of the present application.
Fig. 7 shows a block diagram of a live room green screen detection apparatus according to an embodiment of the present application.
Fig. 8 is a block diagram illustrating a green screen detection apparatus 800 for a live room according to an example embodiment.
Detailed Description
Various exemplary embodiments, features and aspects of the application will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
In addition, numerous specific details are set forth in the following description in order to provide a better illustration of the application. It will be understood by those skilled in the art that the present application may be practiced without some of these specific details. In some instances, well known methods, procedures, components, and circuits have not been described in detail so as not to obscure the present application.
Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
In recent years, with research and progress of artificial intelligence technology, the artificial intelligence technology is widely applied in a plurality of fields, and the scheme provided by the embodiment of the application relates to technologies such as computer vision, and is specifically described by the following embodiments:
referring to fig. 1, fig. 1 is a schematic diagram of an application system according to an embodiment of the application. The application system can be used for the live broadcasting room green screen detection method. As shown in fig. 1, the application system may include at least a server 01 and a terminal 02.
In the embodiment of the present application, the server 01 may include an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network ), and basic cloud computing services such as big data and an artificial intelligence platform.
In the embodiment of the present application, the terminal 02 may include a smart phone, a desktop computer, a tablet computer, a notebook computer, a smart speaker, a digital assistant, an augmented reality (augmented reality, AR)/Virtual Reality (VR) device, an intelligent wearable device, and other types of entity devices. The physical device may also include software, such as an application, running in the physical device. The operating system running on the terminal 02 in the embodiment of the present application may include, but is not limited to, an android system, an IOS system, linux, windows, and the like.
In the embodiment of the present disclosure, the terminal 02 and the server 01 may be directly or indirectly connected through a wired or wireless communication method, which is not limited to the present disclosure.
The terminal 02 may be configured to provide a user-oriented green screen detection process in a live room. When a user watches the live video of the live broadcasting room on the terminal 02, the terminal can acquire the live video watched by the user through a recording screen; or acquiring live picture images in live video watched by a user in a screenshot mode, so that green screen detection processing of a live broadcasting room can be performed. The manner in which the terminal 02 provides the user-oriented green screen detection process in the live broadcasting room may include, but is not limited to, an application manner, a web page manner, and the like. The live video may be an electronic bidding live video, which is not limited in this application.
It should be noted that in the embodiment of the present application, the method for detecting the green screen of the live broadcasting room may be executed by the server 01, and preferably, the method for detecting the green screen of the live broadcasting room is implemented in the server 01. So as to relieve the data processing pressure of the terminal 02 and improve the device performance of the user oriented terminal. When the server 01 executes the live broadcast room green screen detection method, the terminal can send live broadcast video watched by the user or live broadcast picture images in the live broadcast video watched by the user to the server 02 in real time, so that the server 02 can perform live broadcast room green screen detection processing. Optionally, when the server 02 detects a green screen, the green screen repair process of the live broadcasting room can be performed, and through the timely repair process, the probability of the green screen appearing in the subsequent live broadcasting video of the live broadcasting room can be reduced.
In a specific embodiment, when the server 02 is a distributed system, the distributed system may be a blockchain system, and when the distributed system is a blockchain system, the distributed system may be formed by a plurality of nodes (any form of computing device in an access network, such as a server and a user terminal), and a Peer-To-Peer (P2P, peer To Peer) network is formed between the nodes, where the P2P protocol is an application layer protocol running on top of a transmission control protocol (TCP, transmission Control Protocol) protocol. In a distributed system, any machine, such as a server, a terminal, may join to become a node, including a hardware layer, an intermediate layer, an operating system layer, and an application layer. Specifically, the functions of each node in the blockchain system may include:
1) The routing, the node has basic functions for supporting communication between nodes.
Besides the routing function, the node can also have the following functions:
2) The application is used for being deployed in a block chain to realize specific service according to actual service requirements, recording data related to the realization function to form recorded data, carrying a digital signature in the recorded data to represent the source of task data, sending the recorded data to other nodes in the block chain system, and adding the recorded data into a temporary block when the source and the integrity of the recorded data are verified by the other nodes.
It should be noted that, a possible sequence of steps is shown in the following figures, and is not limited to the strict order of the sequence. Some steps may be performed in parallel without mutual dependency.
In particular, fig. 2 shows a flowchart of a live room green screen detection method according to an embodiment of the present application. As shown in fig. 2, the method may include:
s201, acquiring live broadcast picture images in a live broadcast process of a target live broadcast room.
In the embodiment of the present specification, the target live room may refer to a live room that is performing a preset live type of live. The preset live broadcast type may be a live broadcast scene type, for example, non-PK live broadcast with wheat, non-PK live broadcast with wheat may include live broadcast with event game, etc., which is not limited in this application, and may be set in advance according to actual requirements. The live image can refer to a live interface image or an image of a live window area in the live interface, and when the live image is the image of the live window area in the live interface, the complexity and the processing time of acquiring the live image and subsequently carrying out green screen detection on the live image can be reduced, so that the green screen detection is more efficient.
In the live broadcast process of the target live broadcast room, a user can watch live broadcast of the target live broadcast room on the terminal, and live broadcast picture images in the live broadcast process of the target live broadcast room on the side of the terminal can be obtained. For example, the terminal can intercept live broadcast picture images in the live broadcast process of the target live broadcast room in real time, and can send the live broadcast picture images in the live broadcast process of the target live broadcast room to the server, and accordingly, the server can acquire the live broadcast picture images in the live broadcast process of the target live broadcast room.
In one possible implementation, S201 may include: receiving a live video stream in a live broadcasting process of a target live broadcasting room, wherein the live video stream is sent by a terminal in real time; and carrying out frame division processing on the live video stream to obtain live broadcast picture images in the live broadcast process of the target live broadcast room.
In the embodiment of the specification, the terminal can acquire the live video stream in the live broadcast process of the target live broadcast room in a screen recording mode, and can send the live video stream to the server. The server receives the live video stream, and can perform frame division processing on the live video stream to acquire live image images in the live broadcasting process of the target live broadcasting room. As each frame of live image can be obtained, green screen detection can be carried out on the live image without omission.
S203, performing hue-saturation-brightness HSB color mode extraction processing on the live-broadcast picture image to obtain first HSB information corresponding to the pixel point of the live-broadcast picture image;
s205, determining a first target pixel point with a hue H value within a preset numerical range in the first HSB information from the pixel points of the live-broadcast picture image.
In this embodiment of the present disclosure, the preset value range may refer to a value range in which an H (hues) value represents green, and the preset value range may be determined through experiments, for example, the preset value range may be (35,77).
In this embodiment of the present disclosure, an HSB (hue-saturation-brightness) color mode extraction process may be performed on a live-broadcast picture image to obtain first HSB information corresponding to pixels of the live-broadcast picture image, that is, obtain first HSB information, for example, an HSB value, corresponding to each pixel in the live-broadcast picture image. Therefore, a first target pixel point of which the H value is in a preset numerical range in the first HSB information can be determined from the pixel points of the live-broadcast picture image, namely, a green pixel point in the live-broadcast slow image is determined.
It should be noted that, in view of the very low probability that the neutral live picture image occurs in the live broadcast room during the live broadcast, the neutral color may include black, gray, and white. Based on this situation, in order to adapt to the real-time requirement of the green screen detection in the live broadcasting room in the embodiment of the present disclosure, the green pixel point is determined based on the H value within the preset value range. However, if the live image appears as a black image, a gray image or a white image, the determined green image may include the black image, the gray image or the white image in a manner of determining the recording image based on the H value within a preset value range. Based on this, optionally, in order to improve the accuracy of green screen detection, an initial target pixel point of the hue H value in the first HSB information within a preset numerical range may be determined from the pixel points of the live-broadcast picture image; an initial target pixel point with an S value within a first numerical range and a B value within a second numerical range may be obtained as a first target pixel point. The first and second ranges of values may be determined experimentally, e.g., the first range of values is (43,255); the second range of values is (46,255). The first target pixel point determined in this way will not include a black pixel point, a gray pixel point and a white pixel point, and green screen detection is more accurate.
S207, if the duty ratio information of the first target pixel point and the pixel point of the live image meets the preset condition, determining that the live image is a green screen image.
In this embodiment of the present disclosure, the ratio information of the first target pixel points to the pixels of the live-broadcast picture image may refer to a ratio of the number of the first target pixel points to the number of the pixels of the live-broadcast picture image. The preset condition may refer to a preset ratio, such as 80%. If the duty ratio information reaches the preset proportion, the live image can be determined to be a green screen image, and accordingly the green screen image can be determined to be detected.
By acquiring live broadcast picture images in the live broadcast process of the live broadcast room and detecting the live broadcast picture images through the green screen, the automation of the green screen detection of the live broadcast room is realized, and the green screen detection efficiency of the live broadcast room can be improved. And determining whether the live-broadcast picture image is a green screen image or not according to the H value in the first HSB information corresponding to the pixel point of the live-broadcast picture image, and not needing to carry out color channel synthesis, thereby further improving the green screen detection efficiency and providing a guarantee for the timeliness of the green screen restoration of the live-broadcast room.
The green screen detection in the live broadcasting room can also carry out the reason analysis and repair processing of the green screen in the live broadcasting room when the green screen image is detected, so that the probability of the green screen in the follow-up live broadcasting can be reduced. Fig. 3 shows a flowchart of a live room green screen detection method according to an embodiment of the application. In one possible implementation, as shown in fig. 3, after determining that the live image is a green screen image, i.e., after S207, the method may further include:
S301, acquiring live broadcast type information of a target live broadcast room and log information in a target time range; the target time range may characterize the point in time information at which the green screen image appears.
In the embodiment of the present disclosure, the live broadcast type information may represent a live broadcast type of a live broadcast room; the log information may be log information of the terminal side. After determining that the live image is a green screen image, the server can acquire a time point when the green screen image appears and a preset time rule; the target time range can thus be determined based on the point in time at which the green screen image appears and the preset time rule. As an example, the preset time rule may be to set a preset time period before a time point at which the green screen image appears and a time point at which the green screen image appears as a target time range, where the preset time period may be set according to actual needs, and the present application is not limited. After determining the target time range, log information within the target time range may be requested from the terminal. And the live broadcast type of the target live broadcast room at the time point when the green screen image appears can be acquired.
Alternatively, a green screen alert may be triggered when a green screen image is detected, and step S301 may be performed when the green screen alert is detected.
S303, performing live broadcasting room green screen restoration processing based on the live broadcasting type information and the log information.
In the embodiment of the present disclosure, the live broadcast type may be determined according to live broadcast type information. And the problem of the green screen of the live broadcasting room can be checked based on the live broadcasting type and the log information, so that the green screen repairing process of the live broadcasting room can be performed based on the checking reason.
In one possible implementation, the log information may include terminal device information as well as network information within a target time range. The terminal equipment information may include terminal equipment vendor information, operating system information, terminal hardware information. The network information may include a network identification, a network type, and corresponding network state information, and the network type may include a cellular network, a WIFI network, and the like; the network state information may include good network state, bad network state. The application is not limited in this regard.
Fig. 4 is a flowchart illustrating a method for performing a green screen repair process in a live broadcasting room based on live broadcasting type information and log information according to an embodiment of the present application when the log information includes terminal device information and network information within a target time range. As shown in fig. 4, this step S303 may include:
S401, if the network state information in the network information is normal, setting a first direct broadcast test environment and a second direct broadcast test environment based on the terminal equipment information. The test terminal equipment information in the first direct broadcast test environment is terminal equipment information; the test terminal device information in the second live test environment is different from the terminal device information. That is, the virtual terminal may be set for the reason checking of the subsequent green screen image, the first live test environment may be equivalent to the virtual terminal of the target terminal corresponding to the green screen image, and the second live test environment may be a virtual terminal having information different from that of the terminal device. The target terminal may be a terminal indicating that a green screen image is present. Wherein, the fact that the network state information in the network information is normal may mean that the network state information is good in network state.
Optionally, if the network state in the network information is abnormal, a target network corresponding to the network identifier in the network information may be acquired, and the target network may be repaired based on the network type and the network state information in the network information. Wherein, the network state information in the network information being abnormal may mean that the network state information is a network state difference.
S403, acquiring a target live video corresponding to the target live picture image.
In the embodiment of the present disclosure, the green screen detection of the live broadcasting room is performed in real time, so the target live broadcasting picture image herein may be a live broadcasting picture image from the target live broadcasting room to the green screen image, that is, the target live broadcasting picture image may refer to a live broadcasting picture image from the target live broadcasting room to a time point when the green screen image appears. Therefore, the live video corresponding to the target live image can be obtained as the target live video, and the target live video can be played in the first live test environment and the second live test environment respectively for green screen problem investigation.
S405, determining an object code corresponding to the live broadcast type information.
In the embodiment of the present specification, the platform supporting the live broadcast has codes for implementing various functions, and the codes may be divided according to live broadcast types. The target code corresponding to the live broadcast type information can be determined, so that the target code can be quickly positioned when the code is repaired.
S407, if the target live video is played in the first live test environment and the target live video is not played in the second live test environment, repairing the compatibility code of the terminal equipment in the target code.
In the embodiment of the specification, if the target live video is played in the first live test environment and the target live video is not played in the second live test environment, it can be determined that the occurrence of the green screen phenomenon is caused by the compatibility of the terminal equipment, so that the terminal equipment compatibility code in the target code can be repaired, and the repaired terminal equipment compatibility code can be compatible with the terminal equipment information of the green screen image.
S409, if the target live video is played in the first live test environment and the target live video is played in the second live test environment, repairing the live type adaptation code in the target code.
In the embodiment of the specification, if the target live video is displayed in the first live test environment and the live video is displayed in the second live test environment, it may be determined that the target code itself does not support the live type corresponding to the live type information, and at this time, the live type adaptation code in the target code may be repaired, so that the repaired live type adaptation code may support the live type corresponding to the live type information.
Through the restoration of the target code, the green screen problem caused by the compatibility of terminal equipment and the adaptation of the live broadcast type in all live broadcast rooms can be restored simultaneously.
Optionally, in the embodiment of the present disclosure, the repaired target code may be verified, and after the verification is passed, the repaired target code is used to replace the target code, so that the replaced code may be ensured to be better. Based on this, as shown in fig. 5, the method may further include:
s501, performing green screen test processing based on the repaired target code, the target live video, the first direct broadcast test environment and the second direct broadcast test environment to obtain a green screen test result.
In the embodiment of the present disclosure, the virtual live platform may be set based on the repaired object code. Therefore, the target live video can be played in the first live test environment and the second live test environment based on the virtual live platform, and green screen detection is performed in the playing process of the target live video to obtain a green screen test result, for example, a green screen phenomenon appears and a green screen phenomenon does not appear.
And S503, if the green screen test result shows that the green screen phenomenon does not occur, replacing the target code by the repaired target code. That is, the repaired object code can solve the green screen problem, and the object code is replaced with the repaired object code.
Optionally, if the green screen test result indicates that a green screen phenomenon occurs, indicating that the green screen problem is not solved, continuing to perform the green screen repairing process of the live broadcasting room based on the live broadcasting type information and the log information.
In one possible implementation, the method may further include: if the network state information in the network information is normal, and the green screen phenomenon does not occur when the live video is played in the first live test environment and the live video is played in the second live test environment, repairing the green screen detection code. That is, when the green screen false detection is determined, the green screen detection code can be repaired, and the subsequent green screen detection can be more accurate through dynamic repair of the green screen detection code, so that the false detection rate of the green screen detection is reduced.
In the embodiment of the present disclosure, in order to further improve the green screen detection efficiency of the live broadcast room, the live broadcast picture image may be divided into at least two sub-areas, and accordingly, as shown in fig. 6, after acquiring the live broadcast picture image in the live broadcast process of the target live broadcast room at the terminal side, the method may further include:
S601, dividing the live picture image into a preset number of subareas.
In the embodiment of the present disclosure, the preset number may be at least two, which is not limited in the present disclosure. The areas of the preset number of sub-areas may be the same or different, and the present application is not limited thereto.
S603, determining the detection sequence of the subareas.
In the embodiment of the present disclosure, after obtaining the sub-region, the detection sequence of the sub-region may be randomly determined; the order of detection of the sub-areas may also be determined in a left to right, top to bottom order. The application is not limited in this regard.
S605, based on the detection sequence, HSB color mode extraction processing is carried out on the sub-region, and second HSB information corresponding to the pixel points of the sub-region is obtained.
In this embodiment of the present disclosure, green screen detection may be sequentially performed on sub-regions based on the detection sequence, and when a sub-region is detected as a non-green screen region, green screen detection may be performed on the next sub-region. And stopping green screen detection when the subarea is detected to be a green screen area, and directly determining the live broadcast picture image as a green screen image. The green screen detection efficiency is higher. Specifically, in the step S605, the sub-area is subjected to HSB color mode extraction processing to obtain second HSB information corresponding to the pixel point of the sub-area, which may be referred to in step S203 and will not be described herein.
S607, determining a second target pixel point with an H value in a preset numerical range in second HSB information from the pixel points of the subareas;
s609, if the duty ratio information of the second target pixel point and the pixel point of the sub-region meets the preset condition, determining that the live-broadcast picture image is a green screen image.
In the embodiment of the present disclosure, steps S607 to S609 may refer to steps S205 to S207, which are not described herein.
Fig. 7 shows a block diagram of a live room green screen detection apparatus according to an embodiment of the present application. As shown in fig. 7, the apparatus may include:
a live broadcast picture image acquisition module 701, configured to acquire a live broadcast picture image in a live broadcast process in a target live broadcast room;
a first HSB information obtaining module 703, configured to perform hue-saturation-brightness HSB color mode extraction processing on the live-broadcast picture image, to obtain first HSB information corresponding to a pixel point of the live-broadcast picture image;
a first target pixel determining module 705, configured to determine, from among the pixels of the live-broadcast picture image, a first target pixel in which an H value in the first HSB information is within a preset numerical range;
and the green screen image determining module 707 is configured to determine that the live broadcast picture image is a green screen image if the duty ratio information of the first target pixel point and the pixel point of the live broadcast picture image meets a preset condition.
By acquiring live broadcast picture images in the live broadcast process of the live broadcast room and detecting the live broadcast picture images through the green screen, the automation of the green screen detection of the live broadcast room is realized, and the green screen detection efficiency of the live broadcast room can be improved. And determining whether the live-broadcast picture image is a green screen image or not according to the H value in the first HSB information corresponding to the pixel point of the live-broadcast picture image, and not needing to carry out color channel synthesis, thereby further improving the green screen detection efficiency and providing a guarantee for the timeliness of the green screen restoration of the live-broadcast room.
In one possible implementation, the apparatus may further include:
the live broadcast type and log acquisition module is used for acquiring live broadcast type information of the target live broadcast room and log information in a target time range; the target time range represents time point information of the green screen image;
and the green screen repair module is used for performing green screen repair processing on the live broadcasting room based on the live broadcasting type information and the log information.
In one possible implementation, the log information includes terminal device information and network information within the target time range; the green screen repair module may include:
the test environment setting unit is used for setting a first direct broadcast test environment and a second direct broadcast test environment based on the terminal equipment information if the network state information in the network information is normal, wherein the test terminal equipment information in the first direct broadcast test environment is the terminal equipment information; the test terminal equipment information in the second live broadcast test environment is different from the terminal equipment information;
The target live video acquisition unit is used for acquiring target live video corresponding to target live picture images, wherein the target live picture images are live picture images starting to be live in the target live room and are the green screen images;
the target code determining unit is used for determining target codes corresponding to the live broadcast type information;
the first repairing unit is used for repairing the terminal equipment compatibility code in the target code if the target live video is displayed in the first live test environment and the target live video is not displayed in the second live test environment;
and the second repairing unit is used for repairing the live broadcast type adaptation code in the target code if the target live broadcast video is displayed in the first live broadcast test environment and the live broadcast video is displayed in the second live broadcast test environment.
In one possible implementation, the apparatus may further include:
and the green screen detection code repairing module is used for repairing the green screen detection code if the network state information in the network information is normal, and the green screen phenomenon does not occur when the live video is played in the first direct broadcast test environment and the live video is played in the second direct broadcast test environment.
In one possible implementation, the apparatus may further include:
the green screen test module is used for carrying out green screen test processing based on the repaired target code, the target live video, the first direct broadcast test environment and the second direct broadcast test environment to obtain a green screen test result;
and the object code updating module is used for replacing the object code by the repaired object code if the green screen test result shows that the green screen phenomenon does not occur.
In one possible implementation, the live view image acquisition module 701 may include:
the live video stream receiving unit is used for receiving live video streams in a live broadcasting process of a target live broadcasting room, which are sent in real time by the terminal;
and the live broadcast picture image acquisition unit is used for carrying out frame division processing on the live broadcast video stream to acquire live broadcast picture images in the live broadcast process of the target live broadcast room.
In one possible implementation, the apparatus may further include:
the subarea dividing module is used for dividing the live broadcast picture image into a preset number of subareas;
the detection sequence determining module is used for determining the detection sequence of the subareas;
the second HSB information acquisition module is used for carrying out HSB color mode extraction processing on the subareas based on the detection sequence to obtain second HSB information corresponding to the pixel points of the subareas;
The second target pixel point determining module is used for determining a second target pixel point of which the H value in the second HSB information is in a preset numerical range from the pixel points of the subareas;
the green screen image determining module 707 is further configured to determine that the live broadcast picture image is a green screen image if the duty ratio information of the second target pixel point and the pixel point of the sub-region meets a preset condition.
The specific manner in which the individual modules and units perform the operations in relation to the apparatus of the above embodiments has been described in detail in relation to the embodiments of the method and will not be described in detail here.
In another aspect, the present application provides a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the live room green screen detection method provided in the various alternative implementations described above.
Fig. 8 is a block diagram illustrating a green screen detection apparatus 800 for a live room according to an example embodiment. For example, the apparatus 800 may be provided as a server. Referring to fig. 8, apparatus 800 includes a processing component 822 that further includes one or more processors and memory resources, represented by memory 832, for storing instructions, such as application programs, executable by processing component 822. The application programs stored in memory 832 may include one or more modules each corresponding to a set of instructions. Further, the processing component 822 is configured to execute instructions to perform the above-described methods.
The apparatus 800 may also include a power component 826 configured to perform power management of the apparatus 800, a wired or wireless network interface 850 configured to connect the apparatus 800 to a network, and an input/output (I/O) interface 858. The device 800 may operate based on an operating system stored in memory 832, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, such as memory 832 including computer program instructions executable by processing component 822 of apparatus 800 to perform the above-described method.
The present application may be a system, method, and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement aspects of the present application.
The computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: portable computer disks, hard disks, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static Random Access Memory (SRAM), portable compact disk read-only memory (CD-ROM), digital Versatile Disks (DVD), memory sticks, floppy disks, mechanical coding devices, punch cards or in-groove structures such as punch cards or grooves having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media, as used herein, are not to be construed as transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses through fiber optic cables), or electrical signals transmitted through wires.
The computer readable program instructions described herein may be downloaded from a computer readable storage medium to a respective computing/processing device or to an external computer or external storage device over a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers and/or edge servers. The network interface card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in the respective computing/processing device.
Computer program instructions for carrying out operations of the present application may be assembly instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, c++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present application are implemented by personalizing electronic circuitry, such as programmable logic circuitry, field Programmable Gate Arrays (FPGAs), or Programmable Logic Arrays (PLAs), with state information for computer readable program instructions, which can execute the computer readable program instructions.
Various aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of embodiments of the application has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the improvement of technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (8)
1. A method for detecting a green screen in a live broadcast room, the method comprising:
acquiring a live broadcast picture image in a live broadcast process of a target live broadcast room;
performing hue-saturation-brightness HSB color mode extraction processing on the live-broadcast picture image to obtain first HSB information corresponding to a pixel point of the live-broadcast picture image;
determining a first target pixel point of which the hue H value is in a preset numerical range in the first HSB information from pixel points of the live broadcast picture image;
if the duty ratio information of the first target pixel point and the pixel point of the live broadcast picture image meets a preset condition, determining that the live broadcast picture image is a green screen image;
Acquiring live broadcast type information of the target live broadcast room and log information in a target time range; the target time range represents time point information of the green screen image; the log information comprises terminal equipment information and network information in the target time range;
if the network state information in the network information is normal, setting a first direct broadcast test environment and a second direct broadcast test environment based on the terminal equipment information, wherein the test terminal equipment information in the first direct broadcast test environment is the terminal equipment information; the test terminal equipment information in the second live broadcast test environment is different from the terminal equipment information;
acquiring a target live video corresponding to a target live picture image, wherein the target live picture image is a live picture image of the target live room starting live broadcast to the green screen image;
determining an object code corresponding to the live broadcast type information;
if the target live video is played in the first live test environment and the target live video is not played in the second live test environment, repairing a terminal equipment compatibility code in the target code;
If the target live video is played in the first live test environment and the live video is played in the second live test environment, repairing the live type adaptation code in the target code.
2. The method as recited in claim 1, further comprising:
and if the network state information in the network information is normal, and the green screen phenomenon does not occur when the live video is played in the first live test environment and the live video is played in the second live test environment, repairing a green screen detection code.
3. The method as recited in claim 1, further comprising:
performing green screen test processing based on the repaired target code, the target live video, the first direct broadcast test environment and the second direct broadcast test environment to obtain a green screen test result;
and if the green screen test result shows that the green screen phenomenon does not occur, replacing the target code by the repaired target code.
4. The method according to claim 1, wherein the acquiring live view images in the live broadcast process of the target live room comprises:
Receiving a live video stream in a live broadcasting process of a target live broadcasting room, wherein the live video stream is sent by a terminal in real time;
and carrying out framing treatment on the live video stream to obtain live image images in the live broadcasting process of the target live broadcasting room.
5. The method according to claim 1, further comprising, after the acquiring the live view image in the live broadcast process of the target live room:
dividing the live broadcast picture image into a preset number of subareas;
determining the detection sequence of the subareas;
based on the detection sequence, performing HSB color mode extraction processing on the subareas to obtain second HSB information corresponding to pixel points of the subareas;
determining a second target pixel point of which the H value is in a preset numerical range in the second HSB information from the pixel points of the subareas;
and if the duty ratio information of the second target pixel point and the pixel point of the sub-region meets a preset condition, determining that the live-broadcast picture image is a green screen image.
6. A live broadcast room green screen detection device, characterized by comprising:
the live broadcast picture image acquisition module is used for acquiring live broadcast picture images in the live broadcast process of the target live broadcast room;
the first HSB information acquisition module is used for extracting hue-saturation-brightness HSB color modes of the live broadcast picture image to obtain first HSB information corresponding to pixel points of the live broadcast picture image;
The first target pixel point determining module is used for determining a first target pixel point with a hue H value within a preset numerical range in the first HSB information from pixel points of the live broadcast picture image;
the green screen image determining module is used for determining that the live broadcast picture image is a green screen image if the duty ratio information of the first target pixel point and the pixel point of the live broadcast picture image meets a preset condition;
the live broadcast type and log acquisition module is used for acquiring live broadcast type information of the target live broadcast room and log information in a target time range; the target time range represents time point information of the green screen image;
the green screen repair module is used for performing green screen repair processing of the live broadcasting room based on the live broadcasting type information and the log information;
the log information comprises terminal equipment information and network information in the target time range; the green screen repair module may include:
the test environment setting unit is used for setting a first direct broadcast test environment and a second direct broadcast test environment based on the terminal equipment information if the network state information in the network information is normal, wherein the test terminal equipment information in the first direct broadcast test environment is the terminal equipment information; the test terminal equipment information in the second live broadcast test environment is different from the terminal equipment information;
The target live video acquisition unit is used for acquiring target live video corresponding to target live picture images, wherein the target live picture images are live picture images starting to be live in the target live room and are the green screen images;
the target code determining unit is used for determining target codes corresponding to the live broadcast type information;
the first repairing unit is used for repairing the terminal equipment compatibility code in the target code if the target live video is displayed in the first live test environment and the target live video is not displayed in the second live test environment;
and the second repairing unit is used for repairing the live broadcast type adaptation code in the target code if the target live broadcast video is displayed in the first live broadcast test environment and the live broadcast video is displayed in the second live broadcast test environment.
7. A live room green screen detection device, comprising:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to execute the executable instructions to implement the method of any one of claims 1 to 5.
8. A non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218258.0A CN112233196B (en) | 2020-11-04 | 2020-11-04 | Live broadcasting room green screen detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011218258.0A CN112233196B (en) | 2020-11-04 | 2020-11-04 | Live broadcasting room green screen detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112233196A CN112233196A (en) | 2021-01-15 |
| CN112233196B true CN112233196B (en) | 2023-12-12 |
Family
ID=74121473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011218258.0A Active CN112233196B (en) | 2020-11-04 | 2020-11-04 | Live broadcasting room green screen detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112233196B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113099220B (en) * | 2021-06-07 | 2021-08-13 | 全时云商务服务股份有限公司 | Video conference video image green screen detection method, system and storage medium |
| CN113490009B (en) * | 2021-07-06 | 2023-04-21 | 广州虎牙科技有限公司 | Content information implantation method, device, server and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780634A (en) * | 2016-12-27 | 2017-05-31 | 努比亚技术有限公司 | Picture dominant tone extracting method and device |
| CN110708476A (en) * | 2019-09-17 | 2020-01-17 | 广州秀美网络文化传媒有限公司 | Real-time image processing method and device |
| CN110827364A (en) * | 2018-08-07 | 2020-02-21 | 北京优酷科技有限公司 | Method and device for detecting green screen image |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130151970A1 (en) * | 2011-06-03 | 2013-06-13 | Maha Achour | System and Methods for Distributed Multimedia Production |
| US11023707B2 (en) * | 2017-10-27 | 2021-06-01 | Avigilon Corporation | System and method for selecting a part of a video image for a face detection operation |
-
2020
- 2020-11-04 CN CN202011218258.0A patent/CN112233196B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780634A (en) * | 2016-12-27 | 2017-05-31 | 努比亚技术有限公司 | Picture dominant tone extracting method and device |
| CN110827364A (en) * | 2018-08-07 | 2020-02-21 | 北京优酷科技有限公司 | Method and device for detecting green screen image |
| CN110708476A (en) * | 2019-09-17 | 2020-01-17 | 广州秀美网络文化传媒有限公司 | Real-time image processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112233196A (en) | 2021-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108900776B (en) | Method and apparatus for determining response time | |
| CN112333524B (en) | Dynamic video overlay | |
| WO2019242222A1 (en) | Method and device for use in generating information | |
| CN112233196B (en) | Live broadcasting room green screen detection method, device, equipment and storage medium | |
| US20200118257A1 (en) | Image Processing Method, Terminal, and Non-Transitory Computer-Readable Storage Medium | |
| CN108337505B (en) | Information acquisition method and device | |
| CN105302413B (en) | UI (user interface) testing method and system for control | |
| CN110062157B (en) | Method and device for rendering image, electronic equipment and computer readable storage medium | |
| CN111105392A (en) | Display performance testing method and device and storage medium | |
| CN115311178A (en) | Image splicing method, device, equipment and medium | |
| CN110636331B (en) | Method and apparatus for processing video | |
| CN110582021B (en) | Information processing method and device, electronic equipment and storage medium | |
| WO2020135097A1 (en) | Method and apparatus for channel switch detection of display terminal | |
| CN105227597A (en) | A kind of method, Apparatus and system obtaining terminal equipment screen-picture | |
| CN109033264B (en) | Video analysis method and device, electronic equipment and storage medium | |
| CN115934179B (en) | Service function control method and equipment | |
| CN115019138A (en) | Video subtitle erasing, model training and interaction method, device and storage medium | |
| CN114185784A (en) | Barrage rendering test method and device | |
| CN112306462B (en) | Building block type object variable block construction method, device, storage medium and processor | |
| WO2020011066A1 (en) | Terminal display method and device, terminal and readable medium | |
| CN112634460B (en) | Outdoor panorama generation method and device based on Haar-like features | |
| US20200117444A1 (en) | Intelligent Visual Regression System | |
| CN117592974B (en) | Meta universe-based device maintenance method and device, electronic device and readable medium | |
| CN111339367A (en) | Video processing method and device, electronic equipment and computer readable storage medium | |
| CN111010562A (en) | Fault monitoring method and device, computer readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |