CN116233479A - Live broadcast information content auditing system and method based on data processing - Google Patents
Live broadcast information content auditing system and method based on data processing Download PDFInfo
- Publication number
- CN116233479A CN116233479A CN202310472738.7A CN202310472738A CN116233479A CN 116233479 A CN116233479 A CN 116233479A CN 202310472738 A CN202310472738 A CN 202310472738A CN 116233479 A CN116233479 A CN 116233479A
- Authority
- CN
- China
- Prior art keywords
- change
- sequence
- pixel block
- super
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 25
- 230000008859 change Effects 0.000 claims abstract description 289
- 238000007906 compression Methods 0.000 claims abstract description 62
- 230000006835 compression Effects 0.000 claims abstract description 61
- 230000003044 adaptive effect Effects 0.000 claims description 20
- 230000011218 segmentation Effects 0.000 claims description 9
- 230000002123 temporal effect Effects 0.000 claims description 3
- 230000033001 locomotion Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000037081 physical activity Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234318—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
- H04N21/2407—Monitoring of transmitted content, e.g. distribution time, number of downloads
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention relates to the technical field of data processing, in particular to a live information content auditing system and method based on data processing, comprising the following steps: acquiring live video with preset time length, and acquiring a plurality of frames of gray level images; constructing a plurality of gray value change sequences, obtaining the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence, and constructing a change time chart according to the change times; obtaining the importance degree of each pixel point in the change frequency chart, obtaining the self-adaptive compression threshold value of the gray value change sequence corresponding to each pixel point, compressing the gray value change sequence by using run-length coding according to the self-adaptive compression threshold value, obtaining the compressed live video within the preset time length, and checking the live information content by using the compressed live video within the preset time length. The invention aims to solve the problems of low compression efficiency and even data expansion when live video is compressed by using run-length coding.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a live broadcast information content auditing system and method based on data processing.
Background
Live broadcast content produced by a host in the live broadcast industry is good and bad; the existing live broadcast content auditing monitors live broadcast content on a live broadcast platform by matching with manual auditing through intelligent machine vision recognition.
Along with the rapid growth of live broadcast content, the traditional manual auditing has high labor intensity, is not in time, and is easy to misjudge the action of a user due to subjective errors of the manual auditing, so that the user is forced to be off line; the live video is required to be stored in a delayed mode, the auditing efficiency and accuracy are improved through machine auditing and then manual review, but huge pressure is caused to a live content auditing system of a live platform by compression transmission of massive live video, so that the live video is required to be stored in a compressed mode, a large number of continuous pixels with the same gray value are required to exist in the image when frame images in the live video are compressed by traditional stroke coding, and when the conditions are not met, the compression efficiency is low and even data expansion is caused; according to the method, inter-frame travel coding compression is carried out on the live video, the self-adaptive compression threshold value is set for each pixel point of the image according to inter-frame changes of the live video, lossy compression is carried out on the live video, and compression efficiency is improved while important information of the image is protected.
And then the compressed live video is transmitted to a live information content auditing system to complete the auditing of the live information content.
Disclosure of Invention
The invention provides a live information content auditing system and method based on data processing, which are used for solving the problems that a large number of continuous pixel points with the same gray value exist in an image when the frame image in a live video is compressed by using a travel code, and otherwise, the compression efficiency is lower.
The system and the method for auditing live information content based on data processing adopt the following technical scheme:
one embodiment of the invention provides a live information content auditing method based on data processing, which comprises the following steps:
acquiring live video with preset time length, and acquiring a plurality of frames of gray level images;
constructing a plurality of gray value change sequences by using pixel points at the same positions in the continuous frame gray map, acquiring the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence according to the data change condition of each gray value change sequence, and constructing a change times map according to the change times;
obtaining a plurality of super pixel blocks by utilizing a super pixel segmentation algorithm on the change frequency chart, taking a time point change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block as a time point change sequence of each pixel point, obtaining a standard change time sequence of each super pixel block, obtaining a time point change sequence of each pixel point in each super pixel block and a DTW distance of the standard change time sequence of each super pixel block by utilizing a DTW algorithm, obtaining an importance degree of each super pixel block according to a difference of the DTW distances in a DTW distance set of each super pixel block, and obtaining an importance degree of each pixel point in each super pixel block according to a difference of a sequence number value corresponding to the same numerical value and the importance degree of each super pixel block in a numerical change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block;
acquiring an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, and compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value to acquire a live video within a preset time length after compression;
and auditing the live information content by using the compressed live video within the preset time length.
Optionally, the step of obtaining the number of changes of each gray value change sequence, the numerical value change sequence and the time point change sequence according to the data change situation in each gray value change sequence includes the following specific steps:
recording any gray value change sequence as a target change sequence, acquiring two adjacent and unequal gray values in the target change sequence to form a gray value pair, recording the number of the gray value pairs in the target change sequence as the change times of the target change sequence, recording the two gray values in the gray value pair as a first comparison value and a second comparison value respectively, recording the sequence formed by all the first comparison values as the numerical change sequence of the target change sequence, acquiring the sequence number values of all the first comparison values, and forming the time point change sequence of the target change sequence.
Optionally, the step of obtaining the standard variation time sequence of each super pixel block includes the following specific steps:
and (3) marking any super pixel block as a target super pixel block, acquiring the maximum value of pixel values of all pixel points in the target super pixel block, marking the maximum variation times of the target super pixel block, sequentially acquiring the average value of the values corresponding to the same serial numbers in the time point variation sequences of all the pixel points in the target super pixel block according to a time sequence, and marking the average value as the standard variation time sequence of the target super pixel block.
Optionally, the obtaining the importance degree of each super pixel block according to the difference of the DTW distances in the DTW distance set of each super pixel block includes the following specific steps:
recording any super-pixel block as a target super-pixel block, recording a set formed by the time point change sequence of each pixel point in the target super-pixel block and the DTW distance of the standard change time sequence of the target super-pixel block as a DTW distance set of the target super-pixel block, acquiring the average value of the DTW distance set, and recording the average value as the change time similarity degree of the target super-pixel block; acquiring the average value of the pixel values of all pixel points in the target super-pixel block, and marking the average value as the change degree of the target super-pixel block;
wherein (1)>
Indicate->
Importance of each super pixel block, +.>
Indicate->
Degree of variation of the individual superpixel blocks, +.>
Indicate->
The degree of temporal similarity of the changes of the individual superpixel blocks,/->
Expressed as natural constant->
An exponential function of the base +.>
Is the first adjustment coefficient.
Optionally, the obtaining the importance degree of each pixel point in each super pixel block according to the difference of the intervals of the serial number values corresponding to the same data of the numerical values and the importance degree of each super pixel block in the numerical value change sequence of the gray value change sequence corresponding to each pixel point in each super pixel block includes the following specific steps:
recording any super pixel block as a target super pixel block, recording any pixel point in the target super pixel block as a target point, obtaining a numerical value change sequence of a gray value change sequence corresponding to the target point, recording the numerical value change sequence of the target point, recording the serial number value of the numerical value change sequence as a change time node, classifying the same values in the numerical value change sequence into one type to obtain a plurality of gray value types, and recording the number of the gray value types as the number of the gray value types of the target point;
recording any gray value type as a target gray value type, and sequentially acquiring the change time nodes corresponding to the data with the same value as the target gray value type in the value change sequenceThe difference values of two adjacent change time nodes form a change time node difference value set, the variance of data in the change time node difference value set is obtained, and the variance is recorded as a reference value of a target gray value type; acquiring the number of data which is the same as the number of the target gray value type in the number change sequence, recording the number as the number of the target gray value type, and setting a numerical threshold value
The number of the values is less than or equal to a preset number threshold +.>
When the reference value of the target gray value type is adjusted to 0, the number of the values is larger than the preset number threshold value +.>
When the gray scale value type is not adjusted, the reference value of the target gray scale value type is not adjusted;
Optionally, the obtaining the adaptive compression threshold value of the gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart includes the following specific steps:
recording the importance degree of each pixel point on the change frequency chart as the importance degree of the gray value change sequence;
wherein (1)>
Indicate->
Adaptive compression threshold for a sequence of grey value changes, < >>
Indicate->
Importance of the sequence of grey value changes, +.>
Is the maximum compression threshold.
Another embodiment of the present invention provides a live information content auditing system based on data processing, the system comprising the following modules:
and a data acquisition module: acquiring live video with preset time length, and acquiring a plurality of frames of gray level images;
and a data processing module: constructing a plurality of gray value change sequences by using pixel points at the same positions in the continuous frame gray map, acquiring the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence according to the data change condition of each gray value change sequence, and constructing a change times map according to the change times;
obtaining a plurality of super pixel blocks by utilizing a super pixel segmentation algorithm on the change frequency chart, taking a time point change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block as a time point change sequence of each pixel point, obtaining a standard change time sequence of each super pixel block, obtaining a time point change sequence of each pixel point in each super pixel block and a DTW distance of the standard change time sequence of each super pixel block by utilizing a DTW algorithm, obtaining an importance degree of each super pixel block according to a difference of the DTW distances in a DTW distance set of each super pixel block, and obtaining an importance degree of each pixel point in each super pixel block according to a difference of a sequence number value corresponding to the same numerical value and the importance degree of each super pixel block in a numerical change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block;
acquiring an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, and compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value to acquire a live video within a preset time length after compression;
the information content auditing module: and auditing the live information content by using the compressed live video within the preset time length.
The technical scheme of the invention has the beneficial effects that: when the traditional run-length coding is used for compressing the frame images in the live video, a large number of pixel points with the same gray value continuously exist in the video frame images, and when the conditions are not met, the compression efficiency is low, and even data expansion can be caused.
According to the method, according to the characteristics of illegal contents in live broadcast, a change frequency chart is obtained according to the change frequency of each pixel point among live video frames, super-pixel segmentation is carried out on the change frequency chart, then according to the similarity degree of gray value change time nodes under the same gray value change frequency, the importance degree of each super-pixel block is obtained, further, according to the repeated periodic characteristics of time intervals corresponding to the same gray value pixel points on the time nodes in the change process of the gray value of the pixel point at the same coordinate position, the importance degree of each pixel point on the change frequency chart is obtained, a self-adaptive compression threshold is set according to the importance degree of each pixel point, then video inter-frame stroke coding compression is carried out, a larger compression threshold is set for a non-important area, namely a background area, the compression effect is improved, and a smaller compression threshold is set for an important area, namely a main broadcast gesture area, important information in live video is protected, storage space is reduced while important information is saved, and the auditing efficiency and accuracy of live broadcast information content are improved.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart illustrating steps of a method for auditing live information content based on data processing according to an embodiment of the present invention;
fig. 2 is a block diagram of a live information content auditing system based on data processing according to another embodiment of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following detailed description is given below of the specific implementation, structure, characteristics and effects of the system and method for auditing live information content based on data processing according to the invention in combination with the accompanying drawings and the preferred embodiment.
In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same.
Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the system and method for auditing live broadcast information content based on data processing provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of steps of a method for auditing live information content based on data processing according to an embodiment of the present invention is shown, where the method includes the following steps:
and S001, acquiring live video with preset time length, and acquiring a plurality of frame gray level images.
It should be noted that, due to the rapid growth of live broadcast content, the traditional manual auditing has large labor intensity, is not in time, and is easy to cause misjudgment due to subjective errors of manual auditing, thus forcing users to get off line; therefore, delay storage is required for live video, after machine auditing is performed, manual review is performed, auditing efficiency and accuracy are improved, but compression transmission of massive live video causes huge pressure on a live content auditing system of a live platform, compression processing is required for live video, and therefore, live video needs to be acquired at first.
Setting a preset time length
In this embodiment, a predetermined time length is set +.>
For 1 minute, the implementation practitioner can set the preset time length according to the actual situation>
The live video within the preset time length is collected frame by frame according to the time sequence, and is marked as a video frame image, and each video frame image is subjected to gray processing and is marked as a frame gray image.
So far, a plurality of frame gray level diagrams are obtained.
Step S002, a plurality of gray value change sequences are obtained according to a plurality of frame gray images, the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence are obtained, and the change times images are obtained according to the change times.
By detecting the behavior movement of the anchor, the method identifies the important region which possibly violates the relevant regulation, sets a larger compression threshold for the unimportant region, improves the compression effect, sets a smaller compression threshold for the important region, and protects important information in the live video.
Starting from a first pixel point at the upper left corner of each frame of gray level diagram, collecting gray level values of each pixel point from left to right and from top to bottom in an S-shaped manner to form a gray level value sequence of each frame of gray level diagram, obtaining a serial number value of each data in the gray level value sequence, wherein the serial number value represents position information of the pixel point in the image, and each position in the frame of gray level diagram corresponds to one serial number value; and marking any sequence number as a comparison sequence number, sequentially acquiring gray values corresponding to the comparison sequence numbers in the gray value sequences of each frame gray image according to the frame number sequence to form gray value change sequences of the comparison sequence numbers, representing gray values of pixel points at the same positions in different frame gray images, wherein each sequence number corresponds to one gray value change sequence, acquiring the sequence number of each gray value in the gray value change sequence, and the sequence number in the gray value change sequence represents the frame number of the frame gray image corresponding to the gray value.
So far, a plurality of gray value change sequences are obtained.
Recording any gray value change sequence as a target change sequence, acquiring two adjacent and unequal gray values in the target change sequence to form a gray value pair, recording the number of the gray value pairs in the target change sequence as the change times of the target change sequence, recording the two gray values in the gray value pair as a first comparison value and a second comparison value respectively, recording the sequence formed by all the first comparison values as the numerical change sequence of the target change sequence, acquiring the sequence number values of all the first comparison values, and forming the time point change sequence of the target change sequence.
So far, the change times, the numerical value change sequence and the time point change sequence of each gray value change sequence are obtained.
Because each serial number value corresponds to a gray value change sequence, each gray value change sequence corresponds to a change frequency, and each serial number value corresponds to a change frequency; and replacing the gray value of the pixel point in the gray image of the first frame with the change times according to the sequence number value to form a change times image, wherein the pixel value of each pixel point in the change times image is the change times of the gray value change sequence of the pixel point.
So far, the change frequency chart is obtained according to the change frequency.
Step S003, a plurality of super pixel blocks are obtained from the change frequency chart by utilizing a super pixel segmentation algorithm, a time point change sequence of each pixel point in each super pixel block and a standard change time sequence of each super pixel block are obtained, further the importance degree of each super pixel block is obtained, and the importance degree of each pixel point of each super pixel block on the change frequency chart is obtained by combining the importance degree of each super pixel block.
It should be noted that, in the conventional method for detecting bad video by analyzing motion vectors in video stream, feature vector extraction is mainly required to be performed on the size and direction of the motion vectors, so as to analyze the motion rule of people in the video.
However, when the motion vector is a relative offset, and the motion vector and the actual situation have larger errors and influence the detection result when the motion of the human body is faster and the behavior is complex, the embodiment performs super-pixel segmentation according to the gray value change frequency of each pixel point to obtain a super-pixel block containing the pixel points with similar gray value change frequency, and further calculates the similarity of gray value change time nodes on the same-frequency pixel points, namely whether the motion of the human body is faster or slower or the behavior is simple and complex, whether the motion of the object in the same-frequency super-pixel block has a rule or not can be accurately analyzed, and the accuracy of motion rule detection is improved.
Dividing the change frequency graph into a plurality of super pixel blocks by using a super pixel dividing algorithm, wherein the pixel values of the pixel points in each super pixel block are similar, calculating the average value of the pixel values of all the pixel points in each super pixel block, and obtaining a set
Wherein->
Representing the number of segmented superpixel blocks, for example>
Indicate->
The average value of the pixel values of all pixel points in the super pixel blocks; it should be noted that, the super-pixel segmentation algorithm is in the prior art, and this embodiment is not repeated. />
Recording any super-pixel block as a target super-pixel block, acquiring the maximum value of the pixel values of all pixel points in the target super-pixel block, and recording the maximum value as the maximum change times of the target super-pixel block; and (3) marking any pixel point in the target super-pixel block as a target point, acquiring a time point change sequence of a gray value change sequence corresponding to the target point, marking the time point change sequence of the target point, sequentially acquiring the average value of the numerical values corresponding to the same serial numbers in the time point change sequences of all the pixel points in the target super-pixel block according to a time sequence, and marking the average value as a standard change time sequence of the target super-pixel block.
So far, the time point change sequence of each pixel point in each super pixel block and the standard change time sequence of each super pixel block are obtained.
Recording any super-pixel block as a target super-pixel block, acquiring a DTW distance between a time point change sequence of each pixel point in the target super-pixel block and a standard change time sequence of the target super-pixel block by using a DTW algorithm, recording as a DTW distance set of the target super-pixel block, acquiring an average value of the DTW distance set, and recording as a change time similarity degree of the target super-pixel block; acquiring the average value of the pixel values of all pixel points in the target super-pixel block, and marking the average value as the change degree of the target super-pixel block; it should be noted that, the DTW algorithm is the prior art, which can be used to measure the similarity of two sequences, and the DTW distance of two sequences obtained by using the DTW algorithm is the prior art, which is not described in detail in this embodiment.
When the anchor breaks down, the anchor moves back and forth with a certain periodicity in the live video, and in the area where the anchor is located, the gray values of the pixels at the same position in the frame gray images of the continuous frames are different, so that the number of times of changing the gray values of the pixels at the same position in the frame gray images of the continuous frames is more, and meanwhile, the number of times of changing the pixels with similar distances in the images is similar; and for the background area, in the frame gray level map of the continuous frames, the number of times of changing the gray level value of the pixel point at the same position is less, and the importance degree of each super pixel block is obtained according to the change time similarity degree and the change degree of each super pixel block.
Specifically, by the first
Taking a super pixel block as an example, the importance degree of the super pixel block is acquired>
The calculation method of (1) is as follows:
wherein (1)>
Indicate->
Degree of variation of the individual superpixel blocks, +.>
Indicate->
The degree of temporal similarity of the changes of the individual superpixel blocks,/->
Expressed as natural constant->
An exponential function of the base +.>
For the first adjustment factor, a smaller is prevented
Let->
Early approach to 0, the present embodiment sets +.>
The implementation process implementation person can set the first adjustment coefficient according to the actual situation>
Is of a size of (a) and (b).
If it is
The larger the expression->
The region where the super pixel blocks are located has larger variation in live video with preset time length, and the region further represents the higher the degree that the host may have illegal behaviors, the +.>
Of super-pixel blocksThe greater the degree of importance; since the smaller the DTW distance, the more similar the two sequences are, and thus if +.>
The smaller, the description of->
The change trend of the pixel points in the region where the super pixel blocks are located is similar, and as the body activity region appears in the live video when the rule violation occurs in the anchor, the change trend of the pixel values of the pixel points at the same position in the activity region in the live video of continuous frames is similar, and further the region represents the higher the degree that the rule violation possibly occurs in the anchor, the # -th # -of the region is characterized>
The greater the importance of each super pixel block; therefore, inversely proportional normalized value +.>
As a degree of change +.>
Is set in the above-described table.
So far, the importance degree of each super pixel block is obtained.
Recording any super pixel block as a target super pixel block, recording any pixel point in the target super pixel block as a target point, obtaining a numerical value change sequence of a gray value change sequence corresponding to the target point, recording the numerical value change sequence of the target point, recording the serial number value of the numerical value change sequence as a change time node, classifying the same values in the numerical value change sequence into one type to obtain a plurality of gray value types, and recording the number of the gray value types as the number of the gray value types of the target point.
Recording any gray value type as a target gray value type, sequentially acquiring differences of two adjacent change time nodes in change time nodes corresponding to data with the same value as the target gray value type in a value change sequence to form a change time node difference set, acquiring variances of the data in the change time node difference set, and recording the variances as the target gray value typeA reference value; acquiring the number of data which is the same as the number of the target gray value type in the number change sequence, recording the number as the number of the target gray value type, and setting a numerical threshold value
The present embodiment sets +.>
The implementation practitioner can set a preset numerical threshold value according to the actual situation>
The magnitude of the number of the values is less than or equal to a preset value threshold +.>
When the reference value of the target gray value type is adjusted to 0, the number of the values is larger than the preset number threshold value +.>
And when the reference value of the target gray value type is not adjusted.
Specifically, by the first
The +.>
For example, a pixel is obtained, and the importance degree of the pixel is obtained>
The calculation method of (1) is as follows: />
Wherein (1)>
Indicate->
Importance of each super pixel block, +.>
Indicate->
The +.>
Gray value type number of each pixel point, < >>
Indicate->
The +.>
The +.>
Reference value of the individual gray value types, +.>
For the second adjustment factor, a smaller +.>
Let->
Early approach to 0, the present embodiment sets +.>
The implementation process implementation person can set the second adjustment coefficient according to the actual situation>
Size of->
Expressed as natural constant->
An exponential function of the base.
When the host is illegal, the human body has a periodThe linear reciprocating motion, in the gray value change sequence of the pixel points with the same coordinate position on the frame gray map of the adjacent frames in the region of the human body, the time node interval of the gray value change corresponding to each gray value is similar, and if so
The smaller the variance, the smaller the instruction +.>
The gray value change time node intervals of the gray value types are similar, and the pixel is more important the higher the possibility that the pixel is the region where the rule violation occurs by the anchor, therefore the pixel is more important>
As->
Importance of each super pixel block +.>
Is represented by the product of the two +.>
The +.>
Importance degree of each pixel point.
And carrying out linear normalization processing on the importance degrees of all the pixel points of all the super pixel blocks.
So far, the importance degree of each pixel point of each super pixel block on the change frequency chart is obtained.
Step S004, obtaining an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value, and obtaining the compressed live video within a preset time length.
It should be noted that, when the conventional run-length encoding compresses an image, a large number of pixels with the same gray value continuously exist in the image, and when the pixels do not meet the condition, the compression efficiency is low; according to the method, the device and the system, the travel coding compression between video frames is carried out according to the characteristics of illegal behaviors in live broadcasting, the self-adaptive compression threshold value of each pixel point is set according to the importance degree of each pixel point, the larger compression threshold value is set for unimportant areas, the compression effect is improved, the smaller compression threshold value is set for important areas, and important information in live broadcasting video is protected.
Because the embodiment compresses the change value sequence, the compression of the live video is further completed; each pixel point on the change frequency chart corresponds to a gray value change sequence, and the importance degree of each pixel point on the change frequency chart is recorded as the importance degree of the gray value change sequence.
Specifically, by the first
Taking a gray value change sequence as an example, obtaining an adaptive compression threshold value of the gray value change sequence
The calculation method of (1) is as follows: />
Wherein (1)>
Indicate->
Importance of the sequence of grey value changes, +.>
For maximum compression threshold, the present embodiment sets +.>
The practitioner can set the maximum compression threshold according to the actual situation
Is of a size of (a) and (b).
If it is
The larger, the description of->
The gray value change sequence needs smaller compression threshold value to protect important image area and further +.>
The smaller.
So far, the adaptive compression threshold of each gray value change sequence is obtained.
Compressing each gray value change sequence by using run-length coding, wherein the compression process is as follows: when the absolute value of the difference value between the first data and the second data in the gray value change sequence is smaller than or equal to the adaptive compression threshold value of the gray value change sequence, the second data is adjusted to be the first data value, and when the absolute value of the difference value between the adjusted second data and the third data is smaller than or equal to the adaptive compression threshold value of the gray value change sequence, the third data is adjusted to be the adjusted second data value; when the absolute value of the difference value between the first data and the second data in the gray value change sequence is larger than the self-adaptive compression threshold value of the gray value change sequence, the second data is not adjusted, the magnitude relation between the absolute value of the difference value between the second data and the third data and the self-adaptive compression threshold value of the gray value change sequence is analyzed, and the like, the adjusted gray value change sequence is obtained, and the adjusted gray value change sequence is compressed by using run-length coding.
So far, the compression processing of all gray value change sequences is completed, and the compression of the live video within the preset time length is completed.
And S005, auditing live information content by using the compressed live video within a preset time length.
Inputting the compressed live video within a preset time length into a live information content auditing system, extracting key frames in the live video through a sampling module in the system, carrying out image analysis on the key frames through an analysis module, identifying bad information in the images, and finally stopping broadcasting operation on the live video corresponding to the key frames with the bad content.
Through the steps, the auditing of the live information content is completed.
Referring to fig. 2, a block diagram of a live information content auditing system based on data processing according to another embodiment of the present invention is shown, where the system includes:
and the data acquisition module S101 acquires live video with preset time length and acquires a plurality of frame gray level images.
The data processing module S102:
(1) Acquiring a plurality of gray value change sequences according to a plurality of frame gray images, acquiring the change times, the numerical value change sequences and time point change sequences of each gray value change sequence, and acquiring a change times image according to the change times;
(2) Obtaining a plurality of super pixel blocks from the change frequency map by using a super pixel segmentation algorithm, obtaining a time point change sequence of each pixel point in each super pixel block and a standard change time sequence of each super pixel block, further obtaining the importance degree of each super pixel block, and obtaining the importance degree of each pixel point of each super pixel block on the change frequency map by combining the importance degree of each super pixel block;
(3) And obtaining an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, and compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value to obtain the compressed live video within a preset time length.
And the information content auditing module S103 is used for auditing the live information content by utilizing the compressed live video within the preset time length.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (7)
1. The method for auditing the live information content based on the data processing is characterized by comprising the following steps:
acquiring live video with preset time length, and acquiring a plurality of frames of gray level images;
constructing a plurality of gray value change sequences by using pixel points at the same positions in the continuous frame gray map, acquiring the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence according to the data change condition of each gray value change sequence, and constructing a change times map according to the change times;
obtaining a plurality of super pixel blocks by utilizing a super pixel segmentation algorithm on the change frequency chart, taking a time point change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block as a time point change sequence of each pixel point, obtaining a standard change time sequence of each super pixel block, obtaining a time point change sequence of each pixel point in each super pixel block and a DTW distance of the standard change time sequence of each super pixel block by utilizing a DTW algorithm, obtaining an importance degree of each super pixel block according to a difference of the DTW distances in a DTW distance set of each super pixel block, and obtaining an importance degree of each pixel point in each super pixel block according to a difference of a sequence number value corresponding to the same numerical value and the importance degree of each super pixel block in a numerical change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block;
acquiring an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, and compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value to acquire a live video within a preset time length after compression;
and auditing the live information content by using the compressed live video within the preset time length.
2. The method for auditing live information content based on data processing according to claim 1, wherein the steps of obtaining the number of changes, the number of values and the time point change sequence of each gray value change sequence according to the condition of data change in each gray value change sequence are as follows:
recording any gray value change sequence as a target change sequence, acquiring two adjacent and unequal gray values in the target change sequence to form a gray value pair, recording the number of the gray value pairs in the target change sequence as the change times of the target change sequence, recording the two gray values in the gray value pair as a first comparison value and a second comparison value respectively, recording the sequence formed by all the first comparison values as the numerical change sequence of the target change sequence, acquiring the sequence number values of all the first comparison values, and forming the time point change sequence of the target change sequence.
3. The method for auditing live information content based on data processing according to claim 1, wherein the step of obtaining the standard change time sequence of each super pixel block comprises the following specific steps:
and (3) marking any super pixel block as a target super pixel block, acquiring the maximum value of pixel values of all pixel points in the target super pixel block, marking the maximum variation times of the target super pixel block, sequentially acquiring the average value of the values corresponding to the same serial numbers in the time point variation sequences of all the pixel points in the target super pixel block according to a time sequence, and marking the average value as the standard variation time sequence of the target super pixel block.
4. The method for auditing live information content based on data processing according to claim 1, wherein the step of obtaining the importance degree of each super-pixel block according to the difference of the DTW distances in the DTW distance set of each super-pixel block comprises the following specific steps:
recording any super-pixel block as a target super-pixel block, recording a set formed by the time point change sequence of each pixel point in the target super-pixel block and the DTW distance of the standard change time sequence of the target super-pixel block as a DTW distance set of the target super-pixel block, acquiring the average value of the DTW distance set, and recording the average value as the change time similarity degree of the target super-pixel block; acquiring the average value of the pixel values of all pixel points in the target super-pixel block, and marking the average value as the change degree of the target super-pixel block;
5. The method for auditing live information content based on data processing according to claim 1, wherein the step of obtaining the importance level of each pixel point in each super pixel block according to the difference of the intervals of sequence number values corresponding to the same data and the importance level of each super pixel block in the value change sequence of the gray value change sequence corresponding to each pixel point in each super pixel block comprises the following specific steps:
recording any super pixel block as a target super pixel block, recording any pixel point in the target super pixel block as a target point, obtaining a numerical value change sequence of a gray value change sequence corresponding to the target point, recording the numerical value change sequence of the target point, recording the serial number value of the numerical value change sequence as a change time node, classifying the same values in the numerical value change sequence into one type to obtain a plurality of gray value types, and recording the number of the gray value types as the number of the gray value types of the target point;
recording any gray value type as a target gray value type, sequentially acquiring differences of two adjacent change time nodes in change time nodes corresponding to data with the same value as the target gray value type in a value change sequence to form a change time node difference set, acquiring variances of the data in the change time node difference set, and recording the variances as reference values of the target gray value type; acquiring the number of data which is the same as the number of the target gray value type in the number change sequence, recording the number as the number of the target gray value type, and setting a numerical threshold value
The number of the values is less than or equal to a preset number threshold +.>
When the reference value of the target gray value type is adjusted to 0, the number of the values is larger than the preset number threshold value +.>
When the gray scale value type is not adjusted, the reference value of the target gray scale value type is not adjusted;
6. The method for auditing live information content based on data processing according to claim 1, wherein the step of obtaining the adaptive compression threshold value of the gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart comprises the following specific steps:
recording the importance degree of each pixel point on the change frequency chart as the importance degree of the gray value change sequence;
7. The live information content auditing system based on data processing is characterized by comprising the following modules:
and a data acquisition module: acquiring live video with preset time length, and acquiring a plurality of frames of gray level images;
and a data processing module: constructing a plurality of gray value change sequences by using pixel points at the same positions in the continuous frame gray map, acquiring the change times, the numerical value change sequences and the time point change sequences of each gray value change sequence according to the data change condition of each gray value change sequence, and constructing a change times map according to the change times;
obtaining a plurality of super pixel blocks by utilizing a super pixel segmentation algorithm on the change frequency chart, taking a time point change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block as a time point change sequence of each pixel point, obtaining a standard change time sequence of each super pixel block, obtaining a time point change sequence of each pixel point in each super pixel block and a DTW distance of the standard change time sequence of each super pixel block by utilizing a DTW algorithm, obtaining an importance degree of each super pixel block according to a difference of the DTW distances in a DTW distance set of each super pixel block, and obtaining an importance degree of each pixel point in each super pixel block according to a difference of a sequence number value corresponding to the same numerical value and the importance degree of each super pixel block in a numerical change sequence of a gray value change sequence corresponding to each pixel point in each super pixel block;
acquiring an adaptive compression threshold value of a gray value change sequence corresponding to each pixel point according to the importance degree of each pixel point on the change frequency chart, and compressing the gray value change sequence by using run-length coding according to the adaptive compression threshold value to acquire a live video within a preset time length after compression;
the information content auditing module: and auditing the live information content by using the compressed live video within the preset time length.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310472738.7A CN116233479B (en) | 2023-04-28 | 2023-04-28 | Live broadcast information content auditing system and method based on data processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310472738.7A CN116233479B (en) | 2023-04-28 | 2023-04-28 | Live broadcast information content auditing system and method based on data processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116233479A true CN116233479A (en) | 2023-06-06 |
| CN116233479B CN116233479B (en) | 2023-07-21 |
Family
ID=86573461
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310472738.7A Active CN116233479B (en) | 2023-04-28 | 2023-04-28 | Live broadcast information content auditing system and method based on data processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116233479B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117676136A (en) * | 2023-11-16 | 2024-03-08 | 广州群接龙网络科技有限公司 | Method and system for processing group-connected data |
| CN118509561A (en) * | 2024-07-17 | 2024-08-16 | 国网江西省电力有限公司信息通信分公司 | All-link monitoring method and system based on video |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6252989B1 (en) * | 1997-01-07 | 2001-06-26 | Board Of The Regents, The University Of Texas System | Foveated image coding system and method for image bandwidth reduction |
| CN111709483A (en) * | 2020-06-18 | 2020-09-25 | 山东财经大学 | Multi-feature-based super-pixel clustering method and equipment |
| CN112637645A (en) * | 2019-09-24 | 2021-04-09 | Lg电子株式会社 | Signal processing device and image display apparatus including the same |
| CN115205289A (en) * | 2022-09-15 | 2022-10-18 | 山东雅满家生物质科技有限公司 | Vision-based cork wood floor raw material grading method |
| CN115297289A (en) * | 2022-10-08 | 2022-11-04 | 南通第二世界网络科技有限公司 | Efficient storage method for monitoring video |
| CN115861135A (en) * | 2023-03-01 | 2023-03-28 | 铜牛能源科技(山东)有限公司 | Image enhancement and identification method applied to box panoramic detection |
| CN115914634A (en) * | 2022-12-16 | 2023-04-04 | 苏州迈创信息技术有限公司 | Environmental security engineering monitoring data management method and system |
-
2023
- 2023-04-28 CN CN202310472738.7A patent/CN116233479B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6252989B1 (en) * | 1997-01-07 | 2001-06-26 | Board Of The Regents, The University Of Texas System | Foveated image coding system and method for image bandwidth reduction |
| CN112637645A (en) * | 2019-09-24 | 2021-04-09 | Lg电子株式会社 | Signal processing device and image display apparatus including the same |
| US20210150972A1 (en) * | 2019-09-24 | 2021-05-20 | Lg Electronics Inc. | Signal processing device and image display apparatus including same |
| CN111709483A (en) * | 2020-06-18 | 2020-09-25 | 山东财经大学 | Multi-feature-based super-pixel clustering method and equipment |
| CN115205289A (en) * | 2022-09-15 | 2022-10-18 | 山东雅满家生物质科技有限公司 | Vision-based cork wood floor raw material grading method |
| CN115297289A (en) * | 2022-10-08 | 2022-11-04 | 南通第二世界网络科技有限公司 | Efficient storage method for monitoring video |
| CN115914634A (en) * | 2022-12-16 | 2023-04-04 | 苏州迈创信息技术有限公司 | Environmental security engineering monitoring data management method and system |
| CN115861135A (en) * | 2023-03-01 | 2023-03-28 | 铜牛能源科技(山东)有限公司 | Image enhancement and identification method applied to box panoramic detection |
Non-Patent Citations (3)
| Title |
|---|
| KARETH M.LEON-LOPEZ等: "Online Tensor Sparsifying Transform Based on Temporal Superpixels From Compressive Spectral Video Measurements", IEEE TRANSACTIONS ON IMAGE PROCESSING * |
| 韩硕;童晓冲;卢学良;安子阁;孙阔原;: "手持线激光扫描视频三维重建中的运动线提取算法", 计算机应用与软件, no. 07 * |
| 鲍冠伯;张金芳;吕品;袁麟;: "星空背景图像序列压缩编码方法研究", 计算机工程与应用, no. 27 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117676136A (en) * | 2023-11-16 | 2024-03-08 | 广州群接龙网络科技有限公司 | Method and system for processing group-connected data |
| CN118509561A (en) * | 2024-07-17 | 2024-08-16 | 国网江西省电力有限公司信息通信分公司 | All-link monitoring method and system based on video |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116233479B (en) | 2023-07-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116233479B (en) | Live broadcast information content auditing system and method based on data processing | |
| WO2023134791A2 (en) | Environmental security engineering monitoring data management method and system | |
| CN101534444B (en) | Image processing method, system and device | |
| CN111062314B (en) | Image selection method and device, computer readable storage medium and electronic equipment | |
| US8873869B2 (en) | Image processing apparatus and control method for the same | |
| JP6016332B2 (en) | Image processing apparatus and image processing method | |
| JP2003523587A (en) | Visual attention system | |
| CN115914649A (en) | Data transmission method and system for medical video | |
| CN103313047B (en) | A kind of method for video coding and device | |
| JP2006506726A (en) | Image segmentation using template prediction | |
| CN107135401B (en) | Key frame selection method and system | |
| US20130155228A1 (en) | Moving object detection method and apparatus based on compressed domain | |
| CN117132510A (en) | Monitoring image enhancement method and system based on image processing | |
| dos Santos et al. | CV-C3D: action recognition on compressed videos with convolutional 3d networks | |
| CN103020138A (en) | Method and device for video retrieval | |
| CN101237581B (en) | H.264 compression domain real time video object division method based on motion feature | |
| CN113810654A (en) | Image video uploading method and device, storage medium and electronic equipment | |
| CN111079567B (en) | Sampling method, model generation method, video behavior identification method and device | |
| Ouyang et al. | The comparison and analysis of extracting video key frame | |
| CN102509076A (en) | Principal-component-analysis-based video image background detection method | |
| JP4036321B2 (en) | Video search device and search program | |
| CN111145193B (en) | Self-adaptive global threshold binarization method | |
| CN104410863B (en) | Image processor and image processing method | |
| CN109862207B (en) | KVM video content change detection method based on compressed domain | |
| CN113810692A (en) | Method for framing changes and movements, image processing apparatus and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |