CN113076566B - Display content detection method, apparatus, computer program product, and storage medium - Google Patents
Display content detection method, apparatus, computer program product, and storage medium Download PDFInfo
- Publication number
- CN113076566B CN113076566B CN202110462630.0A CN202110462630A CN113076566B CN 113076566 B CN113076566 B CN 113076566B CN 202110462630 A CN202110462630 A CN 202110462630A CN 113076566 B CN113076566 B CN 113076566B
- Authority
- CN
- China
- Prior art keywords
- multimedia data
- data
- display
- display content
- content detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 61
- 238000004590 computer program Methods 0.000 title claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 10
- 230000003068 static effect Effects 0.000 claims description 6
- 210000001525 retina Anatomy 0.000 claims description 3
- 238000004422 calculation algorithm Methods 0.000 claims description 2
- 230000009471 action Effects 0.000 abstract description 10
- 230000000694 effects Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000004891 communication Methods 0.000 description 7
- 230000004048 modification Effects 0.000 description 6
- 238000012986 modification Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000004075 alteration Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/70—Protecting specific internal or peripheral components, in which the protection of a component leads to protection of the entire computer
- G06F21/82—Protecting input, output or interconnection devices
- G06F21/84—Protecting input, output or interconnection devices output devices, e.g. displays or monitors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Computer Security & Cryptography (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a display content detection method, a device, a computer program product and a storage medium, wherein the method comprises the following steps: acquiring multimedia data sent by a cloud; controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time; when the data type of the multimedia data is monitored to be illegal, a preset display protection action is executed, the problem of inaccurate display of the display content in the prior art is solved, and the effect of safe display of the display content is realized.
Description
Technical Field
The present invention relates to the field of video monitoring technology, and in particular, to a display content detection method, apparatus, computer program product, and storage medium.
Background
At present, the wisdom pole has integrated multiple functions, such as illumination, environmental monitoring or video put in the function such as, among the prior art, and the wisdom pole adopts "lamp pole + edge calculation gateway + cloud platform" mode generally, and at the in-process of putting in video, edit the program and detect the program content at the cloud platform, send safe program content to edge calculation gateway, the rethread edge calculation gateway shows the program content on the lamp pole screen, in the edge calculation gateway, when the program content is attacked by maliciously, the program content does not obtain in time audit just to show on the lamp pole screen.
Disclosure of Invention
The invention mainly aims to provide a display content detection method, a device, a computer program product and a storage medium, and aims to solve the problem of inaccurate display of display content.
In order to achieve the above object, the present invention provides a display content detection method; in one embodiment, the display content detection method includes the steps of:
acquiring multimedia data sent by a cloud;
controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time;
and when the data type of the multimedia data is detected to be illegal, executing a preset display protection action.
In an embodiment, the step of controlling the display device to display the multimedia data and monitoring the data type of the multimedia data in real time includes:
preprocessing the multimedia data, wherein the preprocessing comprises gray scale processing and binarization processing;
determining a predicted object contained in the preprocessed multimedia data through a target detection model;
and determining the data type corresponding to the multimedia data according to the prediction object contained in the multimedia data.
In an embodiment, the step of determining the data type corresponding to the multimedia data according to the prediction object included in the multimedia data includes:
obtaining a tensor value corresponding to the predicted object, wherein the tensor value comprises the frame size of the predicted object and the corresponding data type probability;
and determining the data type of the multimedia data according to the frame size of the predicted object and the data type probability.
In an embodiment, after the step of determining the data type of the multimedia data according to the frame size of the prediction object and the data type probability, the method further includes:
judging whether the probability of the data type corresponding to the prediction object contained in the multimedia data is larger than a preset threshold value or not;
and when the data type probability is larger than a preset threshold value, judging that the data type of the multimedia data is illegal.
In an embodiment, the object detection model includes at least one of YOLO, SSD, retina, and the multimedia data includes at least one of dynamic graphics data, static graphics data.
In an embodiment, the preset protection action includes at least one of:
stopping displaying the multimedia data; and/or the number of the groups of groups,
and displaying corresponding prompt information.
In an embodiment, the step of obtaining the multimedia data sent by the cloud includes:
receiving a video stream sent by a cloud end, and acquiring a decryption key;
and analyzing the video stream by adopting the decryption key to obtain multimedia data.
To achieve the above object, the present invention also provides a computer program product comprising a display content detection program which, when executed by the processor, implements the respective steps of the display content detection method as described above.
In order to achieve the above object, the present invention also provides an intelligent device including a memory, a processor, and a display content detection program stored in the memory and executable on the processor, which when executed by the processor, implements the respective steps of the display content detection method as described above.
In order to achieve the above object, the present invention also provides a storage medium storing a display content detection program which, when executed by a processor, implements the respective steps of the display content detection method described above.
The invention provides a display content detection method, a device, a computer program product and a storage medium, wherein multimedia data sent by a cloud are obtained; controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time; when the data type of the multimedia data is monitored to be illegal, a preset display protection action is executed, the problem of inaccurate display of the display content in the prior art is solved, and the effect of safe display of the display content is realized.
Drawings
FIG. 1 is a schematic diagram of an intelligent device structure according to an embodiment of the present invention;
FIG. 2 is a flowchart showing a first embodiment of a content detection method according to the present invention;
FIG. 3 is a flowchart showing a content detection method according to a second embodiment of the present invention;
FIG. 4 is a flowchart illustrating a third embodiment of a content detection method according to the present invention;
FIG. 5 is a flowchart of a fourth embodiment of a content detection method according to the present invention;
FIG. 6 is a flowchart showing a fifth embodiment of a content detection method according to the present invention;
the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The method and the device solve the problem of inaccurate display of the display content in the prior art, and acquire multimedia data sent by a cloud; controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time; and when the data type of the multimedia data is monitored to be illegal, executing a preset technical scheme of display protection action, and realizing the effect of safely displaying the display content.
In order to better understand the above technical solution, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware running environment according to an embodiment of the present invention.
It should be noted that fig. 1 may be a schematic architecture diagram of a hardware running environment of an intelligent device.
As shown in fig. 1, the smart device may include: a processor 1001, such as a CPU, memory 1005, user interface 1003, network interface 1004, communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the smart device architecture shown in fig. 1 is not limiting of the smart device, and that the smart device may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a display content detection program may be included in the memory 1005 as one type of storage medium. The operating system is a program for managing and controlling intelligent device hardware and software resources, and displays the running of content detection programs and other software or programs.
In the smart device shown in fig. 1, the user interface 1003 is mainly used for connecting a terminal, and performing data communication with the terminal; the network interface 1004 is mainly used for a background server and is in data communication with the background server; the processor 1001 may be used to invoke a display content detection program stored in the memory 1005.
In this embodiment, the smart device includes: a memory 1005, a processor 1001, and a display content detection program stored on the memory and executable on the processor, wherein:
in an embodiment of the present application, the processor 1001 may be configured to invoke the display content detection program stored in the memory 1005, and perform the following operations:
acquiring multimedia data sent by a cloud;
controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time;
and when the data type of the multimedia data is detected to be illegal, executing a preset display protection action.
In an embodiment of the present application, the processor 1001 may be configured to invoke the display content detection program stored in the memory 1005, and perform the following operations:
preprocessing the multimedia data, wherein the preprocessing comprises gray scale processing and binarization processing;
determining a predicted object contained in the preprocessed multimedia data through a target detection model;
and determining the data type corresponding to the multimedia data according to the prediction object contained in the multimedia data.
In an embodiment of the present application, the processor 1001 may be configured to invoke the display content detection program stored in the memory 1005, and perform the following operations:
obtaining a tensor value corresponding to the predicted object, wherein the tensor value comprises the frame size of the predicted object and the corresponding data type probability;
and determining the data type of the multimedia data according to the frame size of the predicted object and the data type probability.
In an embodiment of the present application, the processor 1001 may be configured to invoke the display content detection program stored in the memory 1005, and perform the following operations:
judging whether the probability of the data type corresponding to the prediction object contained in the multimedia data is larger than a preset threshold value or not;
and when the data type probability is larger than a preset threshold value, judging that the data type of the multimedia data is illegal.
In an embodiment of the present application, the processor 1001 may be configured to invoke the display content detection program stored in the memory 1005, and perform the following operations:
receiving a video stream sent by a cloud end, and acquiring a decryption key;
and analyzing the video stream by adopting the decryption key to obtain multimedia data.
Because the intelligent device provided in the embodiments of the present application is an intelligent device used for implementing the method in the embodiments of the present application, based on the method described in the embodiments of the present application, a person skilled in the art can understand the specific structure and the deformation of the intelligent device, and therefore, the description thereof is omitted herein. All intelligent devices adopted by the method of the embodiment of the application belong to the scope of protection required by the application. The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
For a software implementation, the techniques described in embodiments of the present invention may be implemented by modules (e.g., procedures, functions, and so on) that perform the functions described in embodiments of the present invention. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
Based on the above structure, the embodiment of the present invention is provided, and the operating system to which the method for detecting display content is applied in the present application includes, but is not limited to, linux, android or Windows7, and the like, and the method for detecting display content can be applied to intelligent terminals, for example, intelligent devices such as smart bars.
Referring to fig. 2, fig. 2 is a flowchart illustrating a first embodiment of a content detection method according to the present invention, including the following steps:
step S110, acquiring multimedia data sent by a cloud;
step S120, controlling a display device to display the multimedia data, and monitoring the data type of the multimedia data in real time;
step S130, when the data type of the multimedia data is detected to be illegal, a preset display protection action is executed.
In this embodiment, the multimedia data may be displayed on a smart pole, where the smart pole is also called a multifunctional smart pole, is a public infrastructure integrating functions of smart lighting, video delivery, environment detection and the like, the smart pole can mount a 5G communication base station and other devices, the smart pole mainly adopts a construction mode of a cloud + edge computing gateway + a lamp pole intelligent terminal, and the lamp pole intelligent terminal includes a display device for displaying the multimedia data, so that the display device can normally display content under the conditions of offline and network bandwidth limitation and the like, edits the multimedia data at a cloud, and performs auditing of the multimedia data, and the auditing can preview the multimedia data by a staff; and a target detection model can be set at the cloud end and used for detecting whether the multimedia data has illegal contents or not, if the multimedia data does not have illegal contents, the multimedia data is sent to an edge computing gateway, and if the multimedia data has illegal contents, alarm information is generated to prompt staff to detect and process the multimedia data in time.
In this embodiment, after receiving the multimedia data sent by the cloud, the multimedia data is sent to an edge computing gateway, in the edge computing gateway, when a malicious program attacks the edge computing gateway through a network interface or through a USB interface or the like, the multimedia data may be tampered, so that a security problem exists in the multimedia data, and the multimedia data is displayed without being detected safely, thus causing a serious public security problem.
In this embodiment, the multimedia data includes at least one of dynamic graphic data and static graphic data, the dynamic graphic data refers to a graphic that changes its form with time, so the dynamic graphic data is composed of multi-frame graphic data, and the data format of the dynamic graphic data may be at least one of GIF, MPG or DAT, because the predicted object in the dynamic graphic data has motion characteristics, the sources of which include deformation of the predicted object itself, motion of the predicted object, and so on, so the process of detecting the dynamic graphic data is essentially to detect the motion state of the predicted object and the data type of the predicted object; the static graphic data refers to an image which flows along with time and does not change the form, the static graphic data consists of single-frame graphic data, and the data format of the static graphic data can be at least one of BMP, JPG or PNG; after the multimedia data sent by the cloud is obtained, a display device of the intelligent terminal of the intelligent rod is controlled to display the multimedia data, the display device can display the multimedia data comprising data formats such as MP3 and MPEG, before the multimedia data are displayed, a player is adopted to decode the multimedia data, the player is built by adopting a gstreamer multimedia framework, after the player decodes the multimedia data, the multimedia data are divided into two paths to be sent, one path of the multimedia data is sent to a display device of the intelligent rod to be displayed, the other path of the multimedia data is monitored in real time through an edge computing gateway mounted in the intelligent rod, specifically, the edge computing gateway monitors the multimedia data through a target detection model, the target detection model is used for detecting whether the data type of the multimedia data has an illegal type, when the illegal type exists, a preset protection action is executed, and the preset protection action z at least comprises: the display device is controlled to stop displaying the multimedia data, corresponding prompt information can be displayed, alarm information can be generated to indicate that the multimedia data is illegal, workers are reminded of modification, corresponding solving measures can be displayed, and the intelligent rod is automatically modified according to the solving measures or provides the solving measures for the workers to modify.
In the technical scheme of the embodiment, multimedia data sent by a cloud is obtained; controlling a display device to display the multimedia data and monitoring the data type of the multimedia data in real time; when the data type of the multimedia data is monitored to be illegal, a preset display protection action is executed, the problem of inaccurate display of the display content in the prior art is solved, and the effect of safe display of the display content is realized.
Referring to fig. 3, fig. 3 is a flowchart showing a second embodiment of the content detection method according to the present invention, and steps S121 to S123 in the second embodiment are refinement steps of step S120 in the first embodiment, and the second implementation includes the following steps:
step S121, preprocessing the multimedia data;
step S122, determining a predicted object contained in the preprocessed multimedia data through a target detection model;
step S123, determining a data type corresponding to the multimedia data according to a prediction object included in the multimedia data.
In this embodiment, the target detection algorithm adopted by the target detection model includes at least one of YOLO, SSD, retina, and the target detection model adopted at the cloud end and the target detection model adopted at the edge computing gateway may be the same or different, and in this application, a YOLO target detection model is adopted at the cloud end and the edge computing gateway for example for expansion and discussion; before the multimedia data is input into a target detection model, training the target detection model, specifically training the target detection model by adopting illegal type data with labels or tags, such as illegal administrative-related and terrorist images, and training after labeling the illegal type data in the images; and collecting or sorting a plurality of sections of illegal types of videos, intercepting various scenes and multi-mesoscale illegal types of images from the videos, marking the images by using an image marking tool, thereby constructing an illegal type data set, and training the target detection model by adopting the illegal type data set.
In this embodiment, after the multimedia data is obtained from the cloud, noise data may exist in the multimedia data or a part of the multimedia data may be lost, so before the multimedia data is input into the target detection model, the noise data in the multimedia data needs to be screened out or the missing multimedia data needs to be complemented, the screened out or complemented multimedia data is preprocessed, the preprocessing includes gray processing and binarization processing, and the preprocessing further includes equalization processing, when the multimedia data is more, normalization processing may be performed on the multimedia data, and the multimedia data after the preprocessing is input into the target detection model, so as to obtain a more accurate prediction object; when the preprocessed multimedia data is input into the target detection model, because the number of predicted objects in the input multimedia data is uncertain, the target detection model can detect a plurality of predicted objects in the multimedia data, and the target detection model can identify the data types of the plurality of predicted objects in an image and give specific positions, each specific position corresponds to one predicted object feature map, and after the plurality of predicted object feature maps are obtained, the predicted object feature maps need to be classified, namely the predicted objects are classified, and the types of the multimedia data are determined according to classification results.
In the technical scheme of the embodiment, the multimedia data is preprocessed, the preprocessed multimedia data is input into a target detection model to obtain a prediction object, and the technical scheme of the data type corresponding to the multimedia data is determined, so that the data type corresponding to the multimedia data is determined.
Referring to fig. 4, fig. 4 is a flowchart showing a third embodiment of the content detection method according to the present invention, and steps S1231 to S1232 in the third embodiment are refinement steps of step S123 in the second embodiment, and the third implementation includes the following steps:
step S1231, obtaining a tensor value corresponding to the predicted object, wherein the tensor value comprises the frame size of the predicted object and the corresponding data type probability;
step S1232, determining the data type of the multimedia data according to the frame size of the prediction object and the data type probability.
In this embodiment, the predicted objects are labeled by using a minimum feature map, specifically, frames with different sizes or different sizes are adopted to move on the multimedia data, the frames contain the predicted objects, and the length and the width are shortest, different predicted object feature maps are obtained by moving on the multimedia data, for example, on an image, each predicted object feature map contains a tensor value corresponding to the predicted object, that is, the frame size of the predicted object and a corresponding data type probability, the position corresponding to the predicted object can be determined according to the frame size of the predicted object, so that the predicted object is positioned, the data type of the predicted object can be determined according to the data type probability corresponding to the predicted object, the classification of the predicted object is realized, meanwhile, each predicted object corresponds to one data type probability value, and the size of the data type probability value can also be used for determining whether the predicted object is illegal type, that is, whether the predicted object contained in the multimedia data is illegal type data or not.
In the technical scheme of the embodiment, a tensor value corresponding to a predicted object is obtained, and the data type of the multimedia data is determined according to the frame size corresponding to the tensor value and the data type probability, so that the data type in the multimedia data is detected and judged, and the safe display of the multimedia data is ensured.
Referring to fig. 5, fig. 5 is a flowchart showing a fourth embodiment of the content detection method according to the present invention, wherein step S2231-step S2232 in the fourth embodiment are located after step S1232 in the third embodiment, and the fourth implementation includes the following steps:
step S2231, determining whether a probability of a data type corresponding to a prediction object included in the multimedia data is greater than a preset threshold;
step S2232, when the data type probability is greater than a preset threshold, determines that the data type of the multimedia data is an illegal type.
In this embodiment, in the training process, determining a data type probability corresponding to a predicted object through a large amount of training data, setting a corresponding preset threshold according to the data type probability, determining whether an illegal type exists in the multimedia data according to a comparison result of the data type probability and the preset threshold, specifically, in the actual detection process, comparing the data type probability corresponding to the predicted object in the multimedia data with the preset threshold, judging whether the data type probability corresponding to the predicted object contained in the multimedia data is greater than the preset threshold, and when the data type probability is greater than the preset threshold, indicating that the illegal type exists in the multimedia data corresponding to the predicted object, and when the illegal type exists in the multimedia data, indicating that the multimedia data at this time is tampered or has a problem of data defect, controlling the display device to stop displaying the multimedia data, and simultaneously, generating alarm information to remind a worker to process the multimedia data in time; when the data type probability is smaller than a preset threshold, the fact that illegal type data does not exist in the multimedia data corresponding to the predicted object is indicated, the fact that the positioned predicted object is not a target object is indicated, the fact that the positioned predicted object is an interference object is indicated, and the like is indicated, and at the moment, the multimedia data needs to be detected again or staff is reminded of intervention.
In the technical solution of this embodiment, by determining whether the probability of the data type corresponding to the prediction object included in the multimedia data is greater than a preset threshold; when the probability of the data type is larger than a preset threshold, judging the data type of the multimedia data to be an illegal type; and controlling the display device to stop displaying the multimedia data, thereby avoiding the problem of illegal type data display and realizing the effect of safe display of display content.
Referring to fig. 6, fig. 6 is a flowchart showing a fifth embodiment of the content detection method according to the present invention, and steps S111-S112 in the fifth embodiment are refinement steps of step S110 in the first embodiment, and the fifth implementation includes the following steps:
step S111, receiving a video stream sent by a cloud end, and acquiring a decryption key;
and step S112, analyzing the video stream by adopting the decryption key to obtain multimedia data.
In this embodiment, the video stream refers to a continuous video data stream transmitted and played on a data network in time sequence, where a transmission manner of the video stream includes Http stream, tcp stream, and the like, the video stream may be collected in real time by a cloud, and when a play instruction is received, the video stream collected in real time is sent to a display device in real time; the video stream may be pre-stored in the cloud end, and when a play instruction is received, the video stream pre-stored in the cloud end is sent to a display device in real time, in the application, a worker can send multimedia data in the video stream to the cloud end as a data packet, the multimedia data is compressed into a plurality of compressed packets, the compressed packets are continuously sent to the cloud end in real time through a terminal, and when the video stream sent by the cloud end is acquired, the data packet needs to be decompressed to obtain the multimedia data in the data packet; when a playing instruction is received, playing is performed without waiting until all data packets are loaded, the process of playing the multimedia data is essentially that the process of obtaining the data packets and pressurizing the data packets is continued while playing the multimedia data, until all the data packets are decompressed, and all the multimedia data are played, so that the playing efficiency is improved.
In this embodiment, the smart lever intelligent terminal and the cloud terminal adopt Https to transmit in the process of transmitting a video stream, and in the transmission process, the multimedia data may have a risk of being tampered, so that a transmission channel of the multimedia data or encrypting the multimedia data in the video stream is required, when the video stream sent by the cloud terminal is received, a decryption key is required to be obtained while the video stream data packet is decompressed, the encrypted multimedia data is decrypted by adopting the decryption key, after decryption is successful, the decrypted multimedia data is played, if the decryption key cannot decrypt the multimedia data, the multimedia data may have a problem of being tampered or lost, and alarm information is generated to remind a worker to detect the multimedia data in time.
In the technical scheme of the embodiment, the decryption key is obtained when the video stream sent by the cloud is received, the decryption key is adopted to decrypt the multimedia data, and the multimedia data is played after the decryption is successful, so that the problem that the multimedia data is maliciously tampered in the transmission process is avoided.
Based on the same inventive concept, the embodiments of the present application further provide a computer program product, where the computer program product includes a display content detection program, where the display content detection program when executed by a processor implements each step of the display content detection method as described above, and the same technical effects can be achieved, and for avoiding repetition, a description is omitted herein.
Since the computer program product provided in the embodiments of the present application is a computer program product used to implement the method in the embodiments of the present application, based on the method described in the embodiments of the present application, those skilled in the art can understand the specific structure and variations of the computer program product, and therefore, the details are not described herein. All computer program products employed in the methods of the embodiments of the present application are intended to be within the scope of the present application.
Based on the same inventive concept, the embodiments of the present application further provide a storage medium, where the storage medium stores a display content detection program, where each step of the display content detection method described above is implemented when the display content detection program is executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.
Because the storage medium provided in the embodiments of the present application is a storage medium used for implementing the method in the embodiments of the present application, based on the method described in the embodiments of the present application, a person skilled in the art can understand the specific structure and the modification of the storage medium, and therefore, the description thereof is omitted herein. All storage media used in the methods of the embodiments of the present application are within the scope of protection intended in the present application.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or deterministic computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of an deterministic machine program product embodied on one or more deterministic machine-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having deterministic machine-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These determiner program instructions can be provided to a processor of a general purpose determiner, special purpose determiner, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the determiner or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These determiner program instructions may also be stored in a determiner readable memory that can direct a determiner or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the determiner readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These determiner program instructions may also be loaded onto a determiner or other programmable data processing apparatus to cause a series of operational steps to be performed on the determiner or other programmable apparatus to produce a determiner-implemented process such that the instructions which execute on the determiner or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed machine. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, third, etc. do not denote any order. These words may be interpreted as labels.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (9)
1. A display content detection method, applied to an edge computing gateway, the method comprising:
acquiring multimedia data sent by a cloud;
the method comprises the steps of controlling a display device to display multimedia data and monitoring the data types of the multimedia data in real time, wherein the multimedia data is divided into two paths, one path is sent to a display device of an intelligent rod for display, the other path is used for monitoring the multimedia data in real time through an edge computing gateway mounted in the intelligent rod, the edge computing gateway is used for monitoring the multimedia data through a target detection model, and the target detection model is used for detecting whether illegal types exist in the data types of the multimedia data;
when the data type of the multimedia data is detected to be illegal, controlling the display device to stop displaying the multimedia data; and/or controlling the display device to display corresponding prompt information.
2. The display content detection method according to claim 1, wherein the step of controlling the display device to display the multimedia data and monitoring the data type of the multimedia data in real time includes:
preprocessing the multimedia data, wherein the preprocessing comprises gray scale processing and binarization processing;
determining a predicted object contained in the preprocessed multimedia data through a target detection model;
and determining the data type corresponding to the multimedia data according to the prediction object contained in the multimedia data.
3. The display content detection method according to claim 2, wherein the step of determining the data type corresponding to the multimedia data from the prediction object included in the multimedia data includes:
obtaining a tensor value corresponding to the predicted object, wherein the tensor value comprises the frame size of the predicted object and the corresponding data type probability;
and determining the data type of the multimedia data according to the frame size of the predicted object and the data type probability.
4. The display content detection method according to claim 3, wherein after the step of determining the data type of the multimedia data according to the frame size of the prediction object and the data type probability, further comprising:
judging whether the probability of the data type corresponding to the prediction object contained in the multimedia data is larger than a preset threshold value or not;
and when the data type probability is larger than a preset threshold value, judging that the data type of the multimedia data is illegal.
5. The display content detection method according to claim 2, wherein the object detection algorithm adopted by the object detection model includes at least one of YOLO, SSD, retina, and the multimedia data includes at least one of dynamic graphic data and static graphic data.
6. The method for detecting display contents according to claim 1, wherein the step of acquiring multimedia data transmitted from the cloud comprises:
receiving a video stream sent by a cloud end, and acquiring a decryption key;
and analyzing the video stream by adopting the decryption key to obtain multimedia data.
7. A computer program product comprising a display content detection program which, when executed by a processor, implements the steps of the display content detection method according to any of claims 1-6.
8. A smart device comprising a memory, a processor, and a display content detection program stored in the memory and executable on the processor, which when executed by the processor, implements the steps of the display content detection method of any of claims 1-6.
9. A storage medium storing a display content detection program which, when executed by a processor, implements the steps of the display content detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110462630.0A CN113076566B (en) | 2021-04-26 | 2021-04-26 | Display content detection method, apparatus, computer program product, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110462630.0A CN113076566B (en) | 2021-04-26 | 2021-04-26 | Display content detection method, apparatus, computer program product, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113076566A CN113076566A (en) | 2021-07-06 |
| CN113076566B true CN113076566B (en) | 2024-02-27 |
Family
ID=76618947
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110462630.0A Active CN113076566B (en) | 2021-04-26 | 2021-04-26 | Display content detection method, apparatus, computer program product, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113076566B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778260A (en) * | 2009-12-29 | 2010-07-14 | 公安部第三研究所 | Method and system for monitoring and managing videos on basis of structured description |
| CN109376603A (en) * | 2018-09-25 | 2019-02-22 | 北京周同科技有限公司 | A kind of video frequency identifying method, device, computer equipment and storage medium |
| CN109670055A (en) * | 2018-11-30 | 2019-04-23 | 广州市百果园信息技术有限公司 | A kind of multi-medium data checking method, device, equipment and storage medium |
| CN109862394A (en) * | 2019-03-27 | 2019-06-07 | 北京周同科技有限公司 | Checking method, device, equipment and the storage medium of video content |
| CN110619747A (en) * | 2019-09-27 | 2019-12-27 | 山东奥邦交通设施工程有限公司 | Intelligent monitoring method and system for highway road |
| CN111131780A (en) * | 2019-12-26 | 2020-05-08 | 深圳市华智有为科技有限公司 | Video playing system and management method thereof, and intelligent pole management system |
| CN111310665A (en) * | 2020-02-18 | 2020-06-19 | 深圳市商汤科技有限公司 | Violation event detection method and device, electronic equipment and storage medium |
| CN111901433A (en) * | 2020-07-31 | 2020-11-06 | 南京征途信息技术有限公司 | Intelligent power distribution gateway device for power distribution station room |
| CN112148896A (en) * | 2020-09-10 | 2020-12-29 | 京东数字科技控股股份有限公司 | Data processing method and device for terminal media monitoring and broadcasting |
| CN112346684A (en) * | 2020-10-20 | 2021-02-09 | 深圳Tcl新技术有限公司 | Data display method and device, television and computer readable storage medium |
| CN112583120A (en) * | 2020-11-26 | 2021-03-30 | 名匠智汇(中山)科技有限公司 | Smart pole of power grid management monitoring |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3006845B1 (en) * | 2013-06-10 | 2015-07-03 | Viaccess Sa | METHOD FOR IDENTIFYING TERMINAL IN A MULTIMEDIA CONTENT PROVIDING SYSTEM |
| EP3576045B1 (en) * | 2018-06-01 | 2020-05-20 | Axis AB | Correcting image overlay |
-
2021
- 2021-04-26 CN CN202110462630.0A patent/CN113076566B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101778260A (en) * | 2009-12-29 | 2010-07-14 | 公安部第三研究所 | Method and system for monitoring and managing videos on basis of structured description |
| CN109376603A (en) * | 2018-09-25 | 2019-02-22 | 北京周同科技有限公司 | A kind of video frequency identifying method, device, computer equipment and storage medium |
| CN109670055A (en) * | 2018-11-30 | 2019-04-23 | 广州市百果园信息技术有限公司 | A kind of multi-medium data checking method, device, equipment and storage medium |
| CN109862394A (en) * | 2019-03-27 | 2019-06-07 | 北京周同科技有限公司 | Checking method, device, equipment and the storage medium of video content |
| CN110619747A (en) * | 2019-09-27 | 2019-12-27 | 山东奥邦交通设施工程有限公司 | Intelligent monitoring method and system for highway road |
| CN111131780A (en) * | 2019-12-26 | 2020-05-08 | 深圳市华智有为科技有限公司 | Video playing system and management method thereof, and intelligent pole management system |
| CN111310665A (en) * | 2020-02-18 | 2020-06-19 | 深圳市商汤科技有限公司 | Violation event detection method and device, electronic equipment and storage medium |
| CN111901433A (en) * | 2020-07-31 | 2020-11-06 | 南京征途信息技术有限公司 | Intelligent power distribution gateway device for power distribution station room |
| CN112148896A (en) * | 2020-09-10 | 2020-12-29 | 京东数字科技控股股份有限公司 | Data processing method and device for terminal media monitoring and broadcasting |
| CN112346684A (en) * | 2020-10-20 | 2021-02-09 | 深圳Tcl新技术有限公司 | Data display method and device, television and computer readable storage medium |
| CN112583120A (en) * | 2020-11-26 | 2021-03-30 | 名匠智汇(中山)科技有限公司 | Smart pole of power grid management monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113076566A (en) | 2021-07-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111178523B (en) | Behavior detection method and device, electronic equipment and storage medium | |
| CN107911753B (en) | Method and device for adding digital watermark in video | |
| CN109255767B (en) | Image processing method and device | |
| CN110812845B (en) | Plug-in detection method, plug-in recognition model training method and related device | |
| CN112200081A (en) | Abnormal behavior identification method and device, electronic equipment and storage medium | |
| US11164004B2 (en) | Keyframe scheduling method and apparatus, electronic device, program and medium | |
| KR102002024B1 (en) | Method for processing labeling of object and object management server | |
| CN108235004B (en) | Video playing performance test method, device and system | |
| CN112200067B (en) | Intelligent video event detection method, system, electronic equipment and storage medium | |
| CN110177295B (en) | Subtitle out-of-range processing method and device and electronic equipment | |
| WO2020155297A1 (en) | Method for generating video mask information, bullet screen anti-occlusion method, server and client | |
| CN114881708A (en) | Method, device, system, equipment and storage medium for monitoring delivered content | |
| CN109271929B (en) | Detection method and device | |
| CN110191356A (en) | Video reviewing method, device and electronic equipment | |
| KR101984825B1 (en) | Method and Apparatus for Encoding a Cloud Display Screen by Using API Information | |
| CN113378834A (en) | Object detection method, device, apparatus, storage medium, and program product | |
| CN113076566B (en) | Display content detection method, apparatus, computer program product, and storage medium | |
| CN117745589A (en) | Watermark removing method, device and equipment | |
| CN113378025A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN111369591A (en) | Method, device and equipment for tracking moving object | |
| CN112380993A (en) | Intelligent illegal behavior detection system and method based on target real-time tracking information | |
| CN115061386B (en) | Intelligent driving automatic simulation test system and related equipment | |
| EP3709666A1 (en) | Method for fitting target object in video frame, system, and device | |
| CN115937578A (en) | Food safety early warning method and device, computer equipment and storage medium | |
| CN111259194B (en) | Method and apparatus for determining duplicate video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |