CN115359393A - Image screen-splash abnormity identification method based on weak supervision learning - Google Patents

Image screen-splash abnormity identification method based on weak supervision learning Download PDF

Info

Publication number
CN115359393A
CN115359393A CN202210979113.5A CN202210979113A CN115359393A CN 115359393 A CN115359393 A CN 115359393A CN 202210979113 A CN202210979113 A CN 202210979113A CN 115359393 A CN115359393 A CN 115359393A
Authority
CN
China
Prior art keywords
screen
image
image set
splash
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210979113.5A
Other languages
Chinese (zh)
Inventor
聂晖
陈黎
杨小波
李军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Eastwit Technology Co ltd
Original Assignee
Wuhan Eastwit Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Eastwit Technology Co ltd filed Critical Wuhan Eastwit Technology Co ltd
Priority to CN202210979113.5A priority Critical patent/CN115359393A/en
Publication of CN115359393A publication Critical patent/CN115359393A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to the field of computer vision and machine learning, and discloses an image screen-splash abnormity identification method based on weak supervised learning, which comprises the following steps: initializing parameters in the screen splash abnormity identification network and screen splash characteristic generation processing; sampling from video data to obtain a first image set, and performing screen-blooming feature generation processing on the first image set to obtain a second image set and tag data; and carrying out weak supervision training based on the first image set and the second image set to obtain a screen-blooming identification model. The method adopts a data simulation mode, reduces the acquisition difficulty of the video-monitoring flower-screen training sample, and provides great convenience for the application of a machine learning method in flower-screen recognition; by using a weak supervision mode, the requirement of the model on the training data volume is reduced, the accuracy of model identification is ensured, and the practical requirement of abnormal screen splash identification of the monitoring video image is met by an optimal cost ratio scheme.

Description

Image screen-splash abnormity identification method based on weak supervised learning
Technical Field
The invention relates to the field of computer vision and machine learning, in particular to an image screen-splash abnormity identification method based on weak supervision learning.
Background
The application and construction of the public safety video monitoring networking is an important means for maintaining national safety and social stability and preventing and fighting terrorism crimes in new situations, and has important significance for improving the urban and rural management level and innovating social management systems. In recent years, video monitoring system construction is greatly promoted in various places, and the video monitoring system plays an active role in crime fighting, public security and precaution, social management, civil life service and the like. However, due to factors such as network transmission performance limitation, function defects of encoding and decoding equipment or improper manual operation, some frame damages may be generated in the transmission and storage processes of the video, which are expressed as image anomalies (screen splash) after decoding and displaying. The abnormal images can affect the use efficiency of a public safety video monitoring system, harm the construction of a social security prevention and control system and even seriously affect the detection work of social public security and major criminal cases.
Weakly supervised learning is a branch of the field of machine learning that uses limited, noisy or inaccurately labeled data for training of model parameters, as compared to traditional supervised learning. In the practice of screen image recognition, if fully supervised machine learning is desired, a large amount of tagged screen image data is required. Such data acquisition often requires high labor costs, which makes fully supervised florid recognition training often difficult to implement. The user hopes to use a small amount of sample data, the cost is low, a practical and accurate model is trained, and the weak supervised learning is the current preferred scheme.
Disclosure of Invention
The invention provides an image screen-splash abnormity identification method based on weak supervised learning, aiming at the problems of difficult data annotation, multiple data types, complicated data annotation and the like in the video screen-splash detection problem.
In order to solve the technical problem, the invention provides an image screen-splash abnormity identification method based on weak supervised learning, which comprises the following steps:
s1, initializing parameters in a screen splash abnormity identification network and screen splash characteristic generation processing;
s2, sampling from the video data to obtain a first image set, and performing screen-blooming feature generation processing on the first image set to obtain a second image set and tag data;
s3, performing weak supervision training based on the first image set and the second image set to obtain a screen pattern recognition model;
further, step S2 specifically includes:
s21, collecting video data, and randomly selecting a certain number of video files as video data to be processed;
s22, each video file in the video data to be processed is at a time interval t v Carrying out image sampling to obtain a corresponding video image set, and constructing according to the obtained video image set to obtain a first image set;
s23, processing the first image set according to a screen-patterned airspace generation method or a screen-patterned time domain generation method to obtain an intermediate screen-patterned image;
and S24, processing the screen-blooming region of the middle screen-blooming image based on a random plaque algorithm to obtain a second image set.
And S25, automatically labeling the first image set and the second image set based on data characteristics to obtain the label data.
Further, the method for generating the screen-blooming airspace specifically includes:
let the image to be processed be I, whose height is H. Performing screen-knurling treatment on I:
Figure BDA0003799632570000021
where I (x, y) represents the pixel value at the original non-screenful image coordinates (x, y), I * (x, y) represents the pixel value at the coordinates (x, y) of the processed intermediate screenful image, f 1 Representing a method of vertical transfer of patches, f 2 Representative hue region compression method, k h The value is between 0 and 1.
Further, the color block vertical transmission method specifically comprises the following steps:
f 1 (I(x,y),h)＝I(h,y)
further, the hue region compression method specifically includes:
Figure BDA0003799632570000022
wherein r is xy 、g xy 、b xy The RGB three channel values for I (x, y) are represented, and p represents the compression factor.
Further, the generation of the screen-blooming time domain specifically includes:
acquiring non-screen-splash image I 1 、I 2 Wherein, the image I 1 And I 2 Adjacent images in the video image set;
for image I 1 、I 2 Graying to obtain corresponding grayscale image G 1 、G 2 ；
Calculating a difference matrix delta between the gray level images;
Δ＝|G 2 -G 1 |
obtaining a screen area mask M according to the difference matrix and a set screen area threshold T;
Figure BDA0003799632570000031
according to the flower screen mask M and the image I 2 Obtaining a target screen pattern image I * ：
Figure BDA0003799632570000032
The screen area is { (x, y) | M (x, y) =1}.
Further, the random plaque algorithm specifically comprises the following steps:
setting the size P of the plaque and the number of the plaque
Figure BDA0003799632570000033
H and W are the height and the width of the middle screen image, and m is a sparse coefficient;
generating a plaque matrix sequence:
{A i |A i ＝ONE P×P×3 ×k i ,i∈{0,1,…N p }}
wherein ONE P×P×3 A matrix of P × P × 3 elements all 1, k i Is [ -10,10]Random integers within the range;
generating a random coordinate sequence with the number of coordinates N p ：
Initialization plaque mask M p Wherein M is p Matrix with all 0 elements initially H × W × 3
Adding the plaque sequence into the plaque mask by taking the coordinate in the coordinate sequence as the coordinate of the upper left corner of the plaque in the plaque mask, and updating M p 。
Processing the middle screen-splash picture according to a random plaque algorithm, wherein the calculation formula is as follows:
I f ＝I * +M p
wherein, I f For the final screenful image, I * The intermediate screen-patterned image is obtained.
Further, step S3 specifically includes:
s31, constructing a training data set based on the first image set, the second image set and corresponding label data;
and S32, sending the training data set into a screen-blooming recognition network for training. Verifying the obtained screen identification network to obtain a verification index;
s33, adjusting adjustable parameters of the screen splash generation processing in the step S2 according to the verification indexes;
s34, obtaining a new first image set and a new second image set according to the step S2;
and S35, repeating the steps S31-S34 until the model reaches the required index.
Further, the adjusting the adjustable parameter specifically includes:
adjusting the value range of the parameter h in the method for generating the screen-blooming airspace, or adjusting the time interval t of image acquisition in the method for generating the screen-blooming time domain v 。
The beneficial technical effects of the invention are as follows:
the method adopts a data simulation mode, reduces the acquisition difficulty of the video-monitoring flower-screen training sample, and provides great convenience for the application of a machine learning method in flower-screen recognition; by using a weak supervision mode, the requirement of the model on training data volume is reduced, the accuracy of model identification is ensured, and the practical requirement of abnormal screen-splash identification of the monitoring video image is met by an optimal cost ratio scheme.
Drawings
The technical solution of the present invention will be further specifically described with reference to the accompanying drawings and the detailed description.
FIG. 1 is a schematic flow chart of an embodiment of the method of the present invention.
Fig. 2 is a schematic flow chart of data generation in the embodiment of the method of the present invention.
FIG. 3 is a schematic diagram of a weak supervised model training process in an embodiment of the method of the present invention.
Detailed Description
For a further understanding of the present invention, reference will now be made to the following preferred embodiments of the invention in conjunction with the examples, but it is to be understood that the description is intended to further illustrate the features and advantages of the invention and is not intended to limit the scope of the claims which follow.
Referring to fig. 1, an embodiment of the present invention is a method for identifying an image splash screen abnormality based on weak supervised learning, specifically including S1-S3:
referring to fig. 2, the step of generating the screen-splash image is:
the first step is to collect video data and randomly select a certain number of video files as video data to be processed. In the embodiment, 10 video files are collected as initial video data, and 4 video data are randomly selected as data to be processed each time;
second, each video file in the video data to be processed is timedInterval t v Image sampling is performed. A first set of images is obtained. In the present embodiment, t v The initial value is 0.1s, and 100 images are collected in each video;
thirdly, processing the first image set according to a screen-patterned airspace generation method or a screen-patterned time domain generation method to obtain an intermediate screen-patterned image;
and fourthly, processing the screen-blooming area of the middle screen-blooming image based on a random patch algorithm to obtain a second image set. And automatically labeling the first image set and the second image set based on data characteristics to obtain the label data while the screen is being displayed.
The method for generating the screen space domain specifically comprises the following steps:
let the image to be processed be I, whose height is H. Performing screen-knurling treatment on I:
Figure BDA0003799632570000051
where I (x, y) represents the pixel value at the original non-checkered image coordinates (x, y), I * (x, y) represents the pixel value at the coordinates (x, y) of the processed intermediate screenful image, f 1 Representing a vertical transfer method of patches, f 2 Representative hue region compression method, k h The value is between 0 and 1, k in this embodiment h The value is 0.8 and alpha is 1.
The color block vertical transmission method specifically comprises the following steps:
f 1 (I(x,y),h)＝I(h,y)
the hue region compression method specifically comprises the following steps:
Figure BDA0003799632570000052
wherein r is xy 、g xy 、b xy The RGB three channel values for I (x, y) are shown, and p represents the compression factor, which is set to 5 in this embodiment.
The method for generating the screen-blooming time domain comprises the following specific steps of:
firstly, acquiring a non-screen-splash image I 1 、I 2 Wherein, the image I 1 And I 2 Adjacent images in the video image set;
second, image I 1 、I 2 Graying to obtain corresponding grayscale image G 1 、G 2 ；
Thirdly, calculating a difference matrix delta between the gray level images;
Δ＝|G 2 -G 1 |
fourthly, according to the difference matrix and the set screen area threshold value T, which is set to 15 in the embodiment, a screen area mask M is obtained;
Figure BDA0003799632570000061
fifthly, according to the screen mask M and the image I 2 Obtaining a target screen pattern image I * ：
Figure BDA0003799632570000062
The screen area is { (x, y) | M (x, y) =1}.
The random plaque algorithm comprises the following specific steps:
firstly, setting the size P of the patch, wherein P is generally a multiple of 2 and is set to 8 in the embodiment; number of patches
Figure BDA0003799632570000063
Wherein H, W are the height and width of the middle flower screen image, m is a sparse coefficient, which is set to 100 in this embodiment;
secondly, generating a plaque matrix sequence:
{A i |A i ＝ONE P×P×3 ×k i ,i∈{0,1,…N p }}
wherein ONE P×P×3 A matrix of P × P × 3 elements all 1, k i Is [ -10,10]In-range falseA random number;
thirdly, generating a random coordinate sequence with the number of coordinates N p ：
Fourthly, initializing a patch mask M p Wherein M is p A matrix with all 0 elements initially H × W × 3;
fifthly, taking the coordinate in the coordinate sequence as the coordinate of the upper left corner of the plaque in the plaque mask, adding the plaque sequence into the plaque mask, and updating M p 。
Sixthly, processing the middle screenplay picture according to a random plaque algorithm, wherein the calculation formula is as follows:
I f ＝I * +M p
wherein, I f For the final screenful image, I * Is the intermediate screenprint image.
Referring to fig. 3, the weakly supervised model training process includes:
a first step of constructing a training data set based on the first image set and the second image set and corresponding labels;
and secondly, sending the training data set into a flower screen recognition network for training. Verifying the obtained screen identification model to obtain a verification index;
and thirdly, adjusting the adjustable parameters of the screen splash generation processing according to the verification indexes, comprising the following steps: parameter k in the method for generating the screen space domain h Time interval t of image acquisition in the screen-blooming time domain generation method v ；
Fourthly, according to the method for generating and processing the screen-blooming data, a new first image set and a new second image set are obtained;
and fifthly, repeating the steps, and performing iterative training on the recognition model until the model reaches the required index.
The experimental reference results data for this example are as follows:
in this embodiment, for the screen identification network, the number of samples in the training data set is 1200 (including 400 normal images and 400 screen images processed in the time domain and the spatial domain) each time, 5 times of iterative training are performed, and the obtained screen identification model is subjected to 26407 test set images (positive example 7614, negative example 18793), and the obtained identification accuracy indexes are as follows:
the precision ratio P =97.86%, and the F1 fraction was 94.46%.
Finally, it should be noted that the above-mentioned embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, which should be covered by the claims of the present invention.

Claims (9)

1. An image screen splash abnormity identification method based on weak supervision learning is characterized by comprising the following steps:
s1, initializing a screen splash abnormity identification network and parameters in screen splash characteristic generation processing;
s2, sampling from the video data to obtain a first image set, and performing screen-blooming feature generation processing on the first image set to obtain a second image set and tag data;
s3, performing weak supervision training based on the first image set and the second image set to obtain a screen pattern recognition model;
wherein, step S2 specifically includes:
s21, collecting video data, and randomly selecting a certain number of video files as video data to be processed;
s22, each video file in the video data to be processed is divided into time intervals t v Carrying out image sampling to obtain a corresponding video image set, and constructing according to the obtained video image set to obtain a first image set;
s23, processing the first image set according to a screen-patterned airspace generation method or a screen-patterned time domain generation method to obtain an intermediate screen-patterned image;
s24, processing the flower screen area of the middle flower screen image based on a random plaque algorithm to obtain a second image set;
and S25, automatically labeling the first image set and the second image set based on data characteristics to obtain the label data.
2. The image screenplay abnormality recognition method based on weak supervised learning as recited in claim 1, wherein the screenplay spatial domain generation method specifically comprises:
setting the image to be processed as I and the height of the image to be processed as H, and performing screen-knurling processing on the I:
Figure FDA0003799632560000011
where I (x, y) represents the pixel value at the original non-screenful image coordinates (x, y), I * (x, y) represents the pixel value at the coordinates (x, y) of the processed intermediate screenful image, f 1 Representing a vertical transfer method of patches, f 2 Representative hue region compression method, k h The value is between 0 and 1.
3. The image screen splash abnormality identification method based on weak supervised learning as recited in claim 2, wherein the color block vertical transfer method specifically comprises:
f 1 (I(x,y),h)＝I(h,y)。
4. the image screen-blooming abnormality recognition method based on weak supervised learning as claimed in claim 2, wherein the hue region compression method specifically comprises:
Figure FDA0003799632560000021
wherein r is xy 、g xy 、b xy Denote the RGB three channel values of I (x, y), with p representing the compression factor.
5. The image screen-blooming abnormality recognition method based on weak supervision learning according to claim 2, wherein the screen-blooming time domain generation specifically includes:
acquiring non-screen-splash image I 1 、I 2 Wherein, the image I 1 And I 2 Adjacent images in the video image set;
for image I 1 、I 2 Graying to obtain corresponding grayscale image G 1 、G 2 ；
Calculating a difference matrix delta between the gray level images;
Δ＝|G 2 -G 1 |
obtaining a screen area mask M according to the difference matrix and a set screen area threshold T;
Figure FDA0003799632560000022
according to the pattern mask M and the image I 2 Obtaining a target screen pattern image I * ：
Figure FDA0003799632560000023
The screen area is { (x, y) | M (x, y) =1}.
6. The image screen-splash abnormality recognition method based on weak supervision learning according to claim 1, characterized in that the random patch algorithm specifically comprises the steps of:
setting the size P and the number of plaques
Figure FDA0003799632560000024
H, W is the height and width of the middle screen pattern image, and m is a sparse coefficient;
generating a plaque matrix sequence:
{A i |A i ＝ONE P×P×3 ×k i ,i∈{0,1,…N p }}
wherein ONE P×P×3 A matrix of P × P × 3 elements all 1, k i Is [ -10,10 [)]Random integers within the range;
generating a random coordinate sequence with the number of coordinates N p ：
Initialization plaque mask M p Wherein M is p Matrix with all 0 elements initially H × W × 3
Adding the plaque sequence into the plaque mask by taking the coordinate in the coordinate sequence as the coordinate of the upper left corner of the plaque in the plaque mask, and updating M p 。
7. The image screenplay abnormality recognition method based on weak supervised learning according to claim 6, wherein the middle screenplay picture is processed according to a random plaque algorithm, and the calculation formula is as follows:
I f ＝I * +M p
wherein, I f For the final screenful image, I * Is the intermediate screenprint image.
8. The image screen splash abnormality recognition method based on weak supervision learning according to claim 1, wherein the step S3 specifically includes:
s31, constructing a training data set based on the first image set, the second image set and corresponding label data;
s32, sending the training data set into a screen-patterned recognition network for training; verifying the obtained screen identification model to obtain a verification index;
s33, adjusting adjustable parameters of the screen splash generation processing in the step S2 according to the verification indexes;
s34, obtaining a new first image set and a new second image set according to the step S2;
and S35, repeating the steps S31 to S34 until the model reaches the required index.
9. The image screen splash abnormality recognition method based on weak supervision learning according to claim 8, wherein the adjusting of the adjustable parameters specifically includes:
adjusting the value range of the parameter h in the method for generating the screen space domain, or adjusting the screen space domainTime interval t of image acquisition in screen time domain generation method v 。
CN202210979113.5A 2022-08-16 2022-08-16 Image screen-splash abnormity identification method based on weak supervision learning Pending CN115359393A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210979113.5A CN115359393A (en) 2022-08-16 2022-08-16 Image screen-splash abnormity identification method based on weak supervision learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210979113.5A CN115359393A (en) 2022-08-16 2022-08-16 Image screen-splash abnormity identification method based on weak supervision learning

Publications (1)

Publication Number Publication Date
CN115359393A true CN115359393A (en) 2022-11-18

Family

ID=84033885

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210979113.5A Pending CN115359393A (en) 2022-08-16 2022-08-16 Image screen-splash abnormity identification method based on weak supervision learning

Country Status (1)

Country Link
CN (1) CN115359393A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116048912A (en) * 2022-12-20 2023-05-02 中科南京信息高铁研究院 Cloud server configuration anomaly identification method based on weak supervision learning

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116048912A (en) * 2022-12-20 2023-05-02 中科南京信息高铁研究院 Cloud server configuration anomaly identification method based on weak supervision learning

Similar Documents

Publication Publication Date Title
CN110059694B (en) Intelligent identification method for character data in complex scene of power industry
CN109272455B (en) Image defogging method based on weak supervision generation countermeasure network
CN111339902B (en) Liquid crystal display indication recognition method and device for digital display instrument
CN112598054B (en) Power transmission and transformation project quality common disease prevention and detection method based on deep learning
CN110096945B (en) Indoor monitoring video key frame real-time extraction method based on machine learning
CN113947570B (en) Crack identification method based on machine learning algorithm and computer vision
CN114741553B (en) Image feature-based picture searching method
CN112258490A (en) Low-emissivity coating intelligent damage detection method based on optical and infrared image fusion
CN115359393A (en) Image screen-splash abnormity identification method based on weak supervision learning
CN112785578A (en) Road crack detection method and system based on U-shaped codec neural network
CN114663392A (en) Knowledge distillation-based industrial image defect detection method
CN116433634A (en) Industrial image anomaly detection method based on domain self-adaption
CN108921872B (en) Robust visual target tracking method suitable for long-range tracking
Yang et al. EHNQ: Subjective and objective quality evaluation of enhanced night-time images
CN110135274B (en) Face recognition-based people flow statistics method
CN114140731B (en) Traction substation abnormality detection method
CN111626102B (en) Bimodal iterative denoising anomaly detection method and terminal based on video weak marker
CN114494887A (en) Remote sensing image classification method and device, computer equipment and readable storage medium
CN112949666B (en) Image recognition method for fault cathode plate of zinc stripping machine
CN109886276B (en) Half-word judging method for dial rolling digital character
CN110855467B (en) Network comprehensive situation prediction method based on computer vision technology
CN111783794B (en) Multi-scale target detection method based on depth separable convolution residual block and improved NMS (network management system)
CN116823678B (en) Intelligent repairing system for image defect points
CN117789078B (en) Experiment operation evaluation method and system based on AI visual recognition
CN115937601B (en) Transformer substation anomaly detection method based on normal sample modeling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination