CN113762084A - Building night scene light abnormity detection method based on RetinaXNet - Google Patents

Building night scene light abnormity detection method based on RetinaXNet Download PDF

Info

Publication number
CN113762084A
CN113762084A CN202110909371.1A CN202110909371A CN113762084A CN 113762084 A CN113762084 A CN 113762084A CN 202110909371 A CN202110909371 A CN 202110909371A CN 113762084 A CN113762084 A CN 113762084A
Authority
CN
China
Prior art keywords
image
formula
retinaxnet
follows
night scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110909371.1A
Other languages
Chinese (zh)
Other versions
CN113762084B (en
Inventor
宋雪桦
王赟
王昌达
金华
杜聪
刘思雨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangxi Wangkai Construction Co ltd
Original Assignee
Jiangsu University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu University filed Critical Jiangsu University
Priority to CN202110909371.1A priority Critical patent/CN113762084B/en
Publication of CN113762084A publication Critical patent/CN113762084A/en
Application granted granted Critical
Publication of CN113762084B publication Critical patent/CN113762084B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明涉及一种基于RetinaXNet的建筑夜景灯光异常检测方法，数据集采用均衡化处理，保留图像的纹理信息，降低图像复杂度。RetinaXNet网络的输入模块将视频帧缩减为224*224的图像，主干模块采用改进的残差结构提取图像的轮廓信息，检测头模块采用XNet网络加强信息的整合，进行分类与回归，输出模块按照缩减比例将图像重新恢复成原大小。本发明提出的RetinaXNet网络能够用于检测图像中的故障灯的位置以及故障分类，实现自动化检测异常，提高检测的正确率，降低误检的情况，为建筑夜景异常灯光的检测提供一种可靠的方法。

Figure 202110909371

The invention relates to a RetinaXNet-based building night scene lighting abnormality detection method. The data set adopts equalization processing, retains the texture information of the image, and reduces the complexity of the image. The input module of the RetinaXNet network reduces the video frame to a 224*224 image, the backbone module uses an improved residual structure to extract the contour information of the image, the detection head module uses the XNet network to strengthen the integration of information, perform classification and regression, and the output module follows the reduction Scale restores the image to its original size. The RetinaXNet network proposed by the present invention can be used to detect the position of the fault light in the image and the fault classification, realize automatic abnormal detection, improve the correct rate of detection, reduce the situation of false detection, and provide a reliable method for the detection of abnormal lights in building night scenes. method.

Figure 202110909371

Description

Building night scene light abnormity detection method based on RetinaXNet
Technical Field
The invention relates to the field of image processing and abnormity detection, in particular to a building night scene light abnormity detection method based on RetinaXNet.
Background
Along with the application of modern city science and technology and the high-speed development of economic strength, city lighting engineering plays a remarkable role in improving city environment, building livable cities, improving the overall functions of the cities, pulling internal requirements, promoting the development of city economy, improving the images of corresponding enterprises and the like. However, the arrangement of the night scene light of the building is exposed outdoors all the year round, and the night scene light of the building has frequent faults due to the problems of lamp aging, installation environment, heat dissipation and the like. The existing detection means mainly take manual inspection visual inspection as a main part, and the manual inspection has the defects of high cost, low real-time property, strong subjectivity and the like. With the development of artificial intelligence technology, the detection method based on deep learning can replace the traditional artificial-based method in some image-related fields, and the early-training network is adopted to detect the abnormal light of the night scene of the building, so that the detection accuracy is improved, the artificial subjectivity is reduced, and the detection automation is realized.
Disclosure of Invention
Aiming at the defects of high cost, low real-time performance, strong subjectivity and the like of the existing manual inspection, the building night scene light abnormity detection method based on RetinaXNet is provided, and the automation of night scene light abnormity detection is realized through a camera and a network model, and the detection accuracy is improved.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows: a building night scene light abnormity detection method based on RetinaXNet comprises the following steps:
1) constructing an initial night scene lamp image set C and sending the initial night scene lamp image set C to a GPU (graphic processing Unit) computing server for storage;
2) processing the image set C to obtain a data set, and dividing the data set into a training set E and a test set T;
3) constructing a RetinaXNet network model; the RetinaXNet network model comprises an input module, a trunk module and a detection head module;
4) optimizing the weight of the RetinaXNet network model by using a UFL function;
5) training a RetinaXNet network model;
6) and (3) night scene lamp abnormity detection, namely acquiring a frame to be detected through a camera, sending the frame to be detected into a RetinaXNet network, mapping an output result of the network back to an original image, and judging whether the night scene lamp is abnormal or not.
Further, the step 1) comprises the following steps:
1.1) acquiring video data V of night scene light by using a camera, wherein the camera is fixedly arranged at a place where night scene light detection is needed in advance;
1.2) extracting one frame of image from the video data V at intervals of delta t, and constructing an initial night scene light image set C which is recorded as C ═ I1，I2...In}，IiThe number of the ith frame image is n, and the number of the night scene light images is n;
1.3) sending the initial night scene light image set C to a GPU computing server for storage.
Further, the step 2) includes the following steps:
2.1) for each frame image in the image set C, calculating the pixel occurrence probability p with the pixel value less than ji(j) The calculation formula is as follows:
Figure BDA0003202921150000021
pi(j) representing the probability of occurrence of gray scale greater than 0 and less than j in the ith frame image, ntNumber of pixels having a gray level smaller than j, nIThe total number of pixels of each frame image;
2.2) calculating a histogram result G (i) of each frame image in the set C, wherein the calculation formula is as follows:
Figure BDA0003202921150000022
g (i) is the gray level histogram processing result of the ith frame, wherein i is more than or equal to 0 and less than 256, pi(j) Representing the occurrence probability that the pixel is more than 0 and less than j in the ith frame image;
2.3) calculating a result h (v) of pixel equalization, equalizing the set of images C, where C ═ I1,I2...InAnd recording the processed image set as C', C ═ I1',I2'...In' }, the calculation formula is as follows:
Figure BDA0003202921150000023
wherein v is the pixel value of a single image I in the image set C, H (v) is the calculation result of v equalization, G (v) is the histogram processing result of the current v, G (v)minAs the minimum value of histogram processing, GmaxThe maximum value of histogram processing is shown, L is the gray level number, round represents the rounding of the pixel value result, and all pixels are calculated to obtain a single image I ', and the set is marked as C';
2.4) calculate the average pixel value a for the images in the set of images C', where C ═ { I ═ I1',I2'...In' }, the calculation formula is as follows:
Figure BDA0003202921150000031
where M is the length pixel value of the image, N is the width pixel value of the image, It'(r, C) are the coordinates of the image pixels in the image set C', and t is the number of the selected image;
2.5) carrying out missing and filling processing on the images in the image set C', wherein the missing and filling values are g (i, j), and obtaining a data set C ", and the calculation formula is as follows:
Figure BDA0003202921150000032
wherein g (I, j) is a missing or filled value, I ' (I, j) is a pixel value of the image I ' in the image set C ' with the coordinate of (I, j), and Th is a set threshold;
2.6) divide the data set C' into a training set E and a test set T in a ratio of m: n.
Further, the step 3) of constructing the RetinaXNet network model includes the following steps:
3.1) uniformly reducing the images in the training set E into images of r: l size by utilizing an input module, wherein r is the reduced length pixel valueL is the reduced width pixel value, and the transformed set is denoted as X ═ X1,x2...x3}；
3.2) extracting the contour characteristics of each frame in the set X by using a trunk module through a residual error structure, wherein the residual error structure formula is as follows:
F(x)＝f(x)+f(f(x))+f(f(f(x)))
wherein f (x) δ (W x) + c
In the formula, each frame image x is used as the input of a convolution layer, W is a parameter required to be learned by convolution, delta is an activation function, and F (x) is the output result of a residual error structure;
3.3) Structure F for setting input parameters for Classification fusion in the detection head ModuleCLSThe formula is as follows:
FCLS＝δ[WCLS*F(x)]+c
wherein F (x) is the output result of the residual structure, WCLSAs training parameters, δ is the activation function, and c is a constant term;
3.4) Structure for setting nonlinear regression connection parameters in the detection head Module
Figure BDA0003202921150000033
The formula is as follows:
Figure BDA0003202921150000034
in the formula, FCLSAs input part for parameter fusion, W1，W2For training parameters, δ is an activation function, c is a constant term, and LCR connection is nonlinear connection and has the function of strengthening the connection between classification and regression;
3.5) calculating the enhancement parameters of the regression
Figure BDA0003202921150000041
The calculation formula is as follows:
Figure BDA0003202921150000042
in the formula, FregAre the original regression parameters of the network,
Figure BDA0003202921150000043
are the enhanced regression parameters;
3.6) Structure for setting nonlinear classification connection parameters in the detection head Module
Figure BDA0003202921150000044
The formula is as follows:
Figure BDA0003202921150000045
in the formula, W3In order to train the parameters of the device,
Figure BDA0003202921150000046
in order to have the regression parameters strengthened on,
Figure BDA0003202921150000047
classifying the structure of the connection parameter for non-linearity;
3.7) calculating enhanced parameters of the classification
Figure BDA0003202921150000048
The calculation formula is as follows:
Figure BDA0003202921150000049
in the formula, FclsIs the original classification parameter of the network,
Figure BDA00032029211500000410
are enhanced classification parameters.
Further, the weight of the RetinaXNet network model is optimized by using the UFL function in step 4), and the optimization formula is as follows:
Figure BDA00032029211500000411
wherein y is a real label and takes the value of 0 or 1,
Figure BDA00032029211500000412
is the dynamic adjustment factor parameter of the UFL, gamma is the rate of adjusting the sample weights, and alpha is the weight parameter.
Further, the step 5) of training the RetinaXNet network model includes the following steps:
5.1) calculating the accuracy rate accurve, wherein the calculation formula is as follows:
Figure BDA00032029211500000413
in the formula, accuracycacy is accuracy, TP represents that network output in the test set T is a positive sample, and the reference standard is also the number of positive samples; TN represents the network output as negative sample, and the reference standard is also the number of negative samples; FP represents that the network output is a positive sample, but the reference standard is the number of negative samples; FN represents the number of positive samples of which the network output is negative samples and the reference standard is positive samples;
5.2) calculating the recall rate recall, wherein the calculation formula is as follows:
Figure BDA0003202921150000051
in the formula, recall is recall rate. The TP represents the number of positive samples judged to be positive samples, namely the positive samples; FN represents the number of samples that are judged to be negative, but in fact positive;
5.3) calculating F1The value, the calculation formula is as follows:
Figure BDA0003202921150000052
in the formula, F1The method is a calculation result of the balance of accuracy and recall, the comprehensive accuracy and the recall rate;
5.4) judgment of F1If the value is less than t, turning to the step 5.1) for retraining, otherwise, turning to the step 6).
Further, the step 6) of determining whether the night scene light is abnormal includes the following steps:
6.1) mapping the network output result back to the original image, wherein the coordinate, length and height formula mapped to the original image is as follows:
Figure BDA0003202921150000053
Figure BDA0003202921150000054
Figure BDA0003202921150000055
Figure BDA0003202921150000056
in the formula (I), the compound is shown in the specification,
Figure BDA0003202921150000057
x, y, W and H are the coordinates of the upper left corner of the detection frame and the length and height of the detection frame respectively, and xN、yN、WN、HNThe coordinates and the length and the height of the original image are mapped;
6.2) anomaly detection, wherein the detection formula is as follows:
Figure BDA0003202921150000058
in the formula, when warner is 0, no abnormality is found, and when warner is 1, an abnormality is found, an alarm is triggered.
Compared with the traditional method for detecting the abnormal light of the night scene of the building, the method has the advantages that the cost can be saved, the subjectivity of manual judgment is reduced, the detection efficiency is improved, and the good detection effect can be achieved under the conditions of illumination change, angle change, impaired definition and the like.
Drawings
Fig. 1 is a flowchart of a building night scene light abnormality detection method according to the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and specific embodiments, wherein the technical solutions and design principles of the present invention are described in detail below, but the present invention is not limited thereto. Any obvious improvement, replacement or modification by a person skilled in the art can be made without departing from the spirit of the invention.
The invention relates to a building night scene lamplight abnormity detection method based on RetinaXNet, and used equipment comprises an Internet of things camera, a GPU (graphic processing unit) calculation server and an alarm.
The building night scene light abnormity detection method based on RetinaXNet is shown in figure 1, and comprises the following steps:
1) an initial night scene light image set C is constructed and sent to a GPU computing server for storage, and the method comprises the following steps:
1.1) acquiring video data V of night scene light by using a camera, wherein the camera is fixedly arranged in advance at a place where night scene light detection is needed;
1.2) extracting one frame of image from the video data V at intervals of delta t, and constructing an initial night scene light image set C which is recorded as C ═ I1，I2...In}，IiThe number of the ith frame image is n, and the number of the night scene light images is n;
1.3) sending the initial night scene light image set C to a GPU computing server for storage;
2) processing the image set C to obtain an image set C ', performing deletion and filling processing on images in the image set C' to obtain a data set C ', and dividing the C' into a training set E and a test set T; as a preferred embodiment of the invention, the method comprises the following steps:
2.1) for each frame image in the image set C, calculating the pixel occurrence probability p with the pixel value less than ji(j) The calculation formula is as follows:
Figure BDA0003202921150000061
pi(j) representing the probability of occurrence of gray scale greater than 0 and less than j in the ith frame image, ntNumber of pixels having a gray level smaller than j, nIThe total number of pixels of each frame image;
2.2) calculating a histogram result G (i) of each frame image in the set C, wherein the calculation formula is as follows:
Figure BDA0003202921150000071
g (i) is the gray level histogram processing result of the ith frame, wherein i is more than or equal to 0 and less than 256, pi(j) Indicating the occurrence probability of pixels larger than 0 and smaller than j in the ith frame image.
2.3) calculating a result h (v) of pixel equalization, equalizing the set of images C, where C ═ I1,I2...InAnd recording the processed image set as C', C ═ I1',I2'...In' }, the calculation formula is as follows:
Figure BDA0003202921150000072
wherein v is the pixel value of a single image I in the image set C, H (v) is the calculation result of v equalization, G (v) is the histogram processing result of the current v, G (v)minAs the minimum value of histogram processing, GmaxAnd (3) taking the maximum value of histogram processing, wherein L is the gray level number, round represents the rounding of the pixel value result, and a single image I 'is obtained after all pixels are calculated and is recorded as C' in a set.
2.4) calculate the average pixel value a for the images in the set of images C', where C ═ { I ═ I1',I2'...In' }, the formula is as follows:
Figure BDA0003202921150000073
where M is the length pixel value of the image, N is the width pixel value of the image, It'(r, C) are the coordinates of the image pixels in the image set C', and t is the number of the selected image;
2.5) carrying out missing and filling processing on the images in the image set C', wherein the missing and filling values are g (i, j), and obtaining a data set C ", and the formula is as follows:
Figure BDA0003202921150000074
wherein g (I, j) is a missing or filled value, I ' (I, j) is a pixel value of the image I ' in the image set C ' whose coordinate is (I, j), Th is a set threshold, and in the embodiment of the present invention, Th is 180;
2.6) the data set C "is divided into a training set E and a test set T in a ratio m: n, where m: n is 9: 1;
3) constructing a RetinaXNet network model; the RetinaXNet network model comprises an input module, a trunk module and a detection head module; as a preferred embodiment of the invention, the method comprises the following steps:
and 3.1) uniformly reducing the images in the training set E into images of r: l size by using an input module, wherein r is a reduced length pixel value, l is a reduced width pixel value, and a converted set is recorded as X ═ X1,x2...x3}; in a particular embodiment of the invention, r 224, l 224;
3.2) extracting the contour characteristics of each frame in the set X by using a trunk module through a residual error structure, wherein the residual error structure formula is as follows:
F(x)＝f(x)+f(f(x))+f(f(f(x)))
wherein f (x) δ (W x) + c
In the formula, each frame image x is used as the input of the convolution layer, W is the parameter to be learned for convolution, δ is the activation function, and f (x) is the output result of the residual error structure. The method uses n repeated calculations, and in a specific embodiment of the invention, n is set to 50.
3.3) Structure F for setting input parameters for Classification fusion in the detection head ModuleCLSThe formula is as follows:
FCLS＝δ[WCLS*F(x)]+c
wherein F (x) is the output result of the residual structure, WCLSTo train the parameters, δ is the activation function and c is a constant term.
3.4) Structure for setting nonlinear regression connection parameters in the detection head Module
Figure BDA0003202921150000081
The formula is as follows:
Figure BDA0003202921150000082
in the formula, FCLSAs input part for parameter fusion, W1，W2To train the parameters, δ is the activation function, c is a constant term, and the LCR linkage is a nonlinear linkage that acts to strengthen the link between classification and regression.
3.5) calculating the enhancement parameters of the regression
Figure BDA0003202921150000083
The formula is as follows:
Figure BDA0003202921150000084
in the formula, FregAre the original regression parameters of the network,
Figure BDA0003202921150000085
are the regression parameters that are strengthened.
3.6) Structure for setting nonlinear classification connection parameters in the detection head Module
Figure BDA0003202921150000086
The formula is as follows:
Figure BDA0003202921150000087
in the formula, W3In order to train the parameters of the device,
Figure BDA0003202921150000088
in order to have the regression parameters strengthened on,
Figure BDA0003202921150000089
the structure of the connection parameters is classified for non-linearity.
3.7) calculating enhanced parameters of the classification
Figure BDA0003202921150000091
The formula is as follows:
Figure BDA0003202921150000092
in the formula, FclsIs the original classification parameter of the network,
Figure BDA0003202921150000093
are enhanced classification parameters.
4) Optimizing the weight of the RetinaXNet network model by using a UFL function; the formula is as follows:
Figure BDA0003202921150000094
wherein y is a real label and takes the value of 0 or 1,
Figure BDA0003202921150000095
the dynamic adjustment factor parameter of UFL, γ is the rate of adjusting the sample weight, α is the weight parameter, in the embodiment of the present invention, γ is 2, and γ is 0.25;
5) the training network model, as a preferred embodiment of the present invention, comprises the following steps:
5.1) calculating the accuracy rate accurve, wherein the calculation formula is as follows:
Figure BDA0003202921150000096
in the formula, accuracycacy is accuracy, TP represents that network output in the test set T is a positive sample, and the reference standard is also the number of positive samples; TN represents the network output as negative sample, and the reference standard is also the number of negative samples; FP represents that the network output is a positive sample, but the reference standard is the number of negative samples; FN represents the number of positive samples for which the network output is negative, but the reference criterion is positive.
5.2) calculating the recall rate recall, wherein the calculation formula is as follows:
Figure BDA0003202921150000097
in the formula, recall is recall rate. The TP represents the number of positive samples judged to be positive samples, namely the positive samples; FN represents the number of samples that are judged to be negative, but in fact positive;
5.3) calculating F1The value, the calculation formula is as follows:
Figure BDA0003202921150000098
in the formula, F1The method is a calculation result of the balance of accuracycacy and recall, and the comprehensive accuracy and recall rate of the accuracycacy and recall.
5.4) judgment of F1If the value is less than t, turning to step 5.1) for retraining, otherwise turning to step 6), in a specific embodiment of the present invention, t is 0.6;
6) the method comprises the following steps of acquiring a frame to be detected through a camera, sending the frame to be detected to a network for detection, mapping an output result of the network back to an original image, and judging whether a fault lamp occurs, wherein the method comprises the following steps:
6.1) mapping the network output result back to the original image, wherein the coordinate, length and height formula mapped to the original image is as follows:
Figure BDA0003202921150000101
Figure BDA0003202921150000102
Figure BDA0003202921150000103
Figure BDA0003202921150000104
in the formula (I), the compound is shown in the specification,
Figure BDA0003202921150000105
x, y, W and H are the coordinates of the upper left corner of the detection frame and the length and height of the detection frame respectively, and xN、yN、WN、HNTo the coordinates and length and height of the original.
6.2) anomaly detection, wherein the detection formula is as follows:
Figure BDA0003202921150000106
in the formula, when warner is 0, no abnormality is found, and when warner is 1, an abnormality is found, an alarm is triggered.

Claims (7)

1.一种基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，包括如下步骤：1. a kind of abnormal detection method of building night scene lights based on RetinaXNet, is characterized in that, comprises the steps: 1)构建初始夜景灯图像集合C并发送到GPU计算服务器进行存储；1) Build the initial night scene light image set C and send it to the GPU computing server for storage; 2)对图像集合C进行处理得到数据集，并将数据集划分为训练集E和测试集T；3)构建RetinaXNet网络模型；所述RetinaXNet网络模型包括输入模块、主干模块、检测头模块；2) process the image set C to obtain a data set, and divide the data set into a training set E and a test set T; 3) build a RetinaXNet network model; the RetinaXNet network model includes an input module, a backbone module, and a detection head module; 4)利用UFL函数优化RetinaXNet网络模型的权值；4) Use the UFL function to optimize the weights of the RetinaXNet network model; 5)训练RetinaXNet网络模型；5) Train the RetinaXNet network model; 6)夜景灯异常检测，即通过摄像头获取待检测帧并送入RetinaXNet网络，将网络的输出结果映射回原图，判断夜景灯是否异常。6) Anomaly detection of night scene lights, that is, the frames to be detected are obtained through the camera and sent to the RetinaXNet network, and the output results of the network are mapped back to the original image to determine whether the night scene lights are abnormal. 2.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤1)包括如下步骤：2. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, described step 1) comprises the steps: 1.1)利用摄像头采集夜景灯光的视频数据V，所述摄像头事先固定安装于可以拍摄需要进行夜景灯光检测的地点；1.1) Utilize a camera to collect video data V of night scene lights, and the camera is fixed and installed in advance at a location where night scene lighting detection can be performed; 1.2)从视频数据V中每隔时间Δt提取一帧图像，构建初始夜景灯图像集合C，记为C＝{I1，I2...In}，Ii为第i帧图像，n为夜景灯图像的个数；1.2) Extract a frame of image every time Δt from the video data V, and construct the initial night scene light image set C, denoted as C={I 1 , I 2 ... I n }, I i is the ith frame image, n is the number of night scene light images; 1.3)将初始夜景灯图像集合C发送到GPU计算服务器进行存储。1.3) Send the initial night scene light image set C to the GPU computing server for storage. 3.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤2)包括如下步骤：3. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, described step 2) comprises the steps: 2.1)对图像集合C中的每一帧图像，计算像素值小于j的像素出现概率pi(j)，计算公式如下：2.1) For each frame of image in the image set C, calculate the occurrence probability p i (j) of the pixel whose pixel value is less than j, and the calculation formula is as follows:
Figure FDA0003202921140000011
Figure FDA0003202921140000011
 pi(j)表示第i帧图像中灰度大于0小于j的出现概率，nt为灰度级小于j的像素个数，nI为每帧图像的像素总个数；p i (j) represents the probability of occurrence of gray level greater than 0 and less than j in the image of the ith frame, n t is the number of pixels with gray level less than j, and n I is the total number of pixels in each frame of image; 2.2)计算集合C中每一帧图像的直方图结果G(i)，计算公式如下：2.2) Calculate the histogram result G(i) of each frame of image in the set C, and the calculation formula is as follows:
Figure FDA0003202921140000012
Figure FDA0003202921140000012
 G(i)为第i帧的灰度直方图处理结果，其中0≤i＜256，pi(j)表示第i帧图像中像素大于0小于j的出现概率；G(i) is the grayscale histogram processing result of the ith frame, where 0≤i<256, and p i (j) represents the probability of the occurrence of pixels in the image of the ith frame greater than 0 and less than j; 2.3)计算像素均衡化的结果H(v)，均衡化图像集合C，其中C＝{I1,I2...In}，处理后的图像集合记为C′，C'＝{I1',I2'...In'}，计算公式如下：2.3) Calculate the pixel equalization result H(v), equalize the image set C, where C={I 1 , I 2 . . . I n }, the processed image set is denoted as C′, C′={I 1 ',I 2 '...In '}, the calculation formula is as follows:
Figure FDA0003202921140000021
Figure FDA0003202921140000021
 式中，v为图像集合C中单张图像I上的像素值，H(v)为对v均衡化的计算结果，G(v)为当前v的直方图处理结果，Gmin为直方图处理的最小值，Gmax为直方图处理的最大值，L为灰度级数，round代表对像素值结果的四舍五入，所有像素计算完成得到单张图像I'，集合记为C′；In the formula, v is the pixel value of a single image I in the image set C, H(v) is the calculation result of the equalization of v, G(v) is the histogram processing result of the current v, and G min is the histogram processing The minimum value of , G max is the maximum value of histogram processing, L is the number of gray levels, round represents the rounding of the pixel value result, all pixel calculations are completed to obtain a single image I', and the set is denoted as C'; 2.4)对图像集合C′中的图像计算平均像素值a，其中C'＝{I1',I2'...In'}，计算公式如下：2.4) Calculate the average pixel value a for the images in the image set C', where C'= { I 1 ', I 2 '...In '}, and the calculation formula is as follows:
Figure FDA0003202921140000022
Figure FDA0003202921140000022
 式中，M为图像的长度像素值，N为图像的宽度像素值，It'(r,c)为图像集合C′中的图像像素的坐标，t为选取图像的编号；In the formula, M is the length pixel value of the image, N is the width pixel value of the image, I t '(r, c) is the coordinate of the image pixel in the image set C', and t is the number of the selected image; 2.5)对图像集合C′中的图像进行缺失和填补处理，缺失和填补的值为g(i,j)，得到数据集C”，计算公式如下：2.5) Perform missing and filling processing on the images in the image set C', the missing and filling values are g(i, j), and the data set C" is obtained. The calculation formula is as follows:
Figure FDA0003202921140000023
Figure FDA0003202921140000023
 式中，g(i,j)为缺失和填补的值，I'(i,j)为图像集合C′中图像I'的坐标为(i,j)的像素值，Th为设定的阈值；In the formula, g(i,j) is the missing and filled value, I'(i,j) is the pixel value of the image I' in the image set C' whose coordinates are (i,j), and Th is the set threshold ; 2.6)按比例m:n将数据集C”分为训练集E和测试集T。2.6) Divide the dataset C" into training set E and test set T according to the ratio m:n. 4.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤3)构建RetinaXNet网络模型包括如下步骤：4. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, described step 3) builds RetinaXNet network model and comprises the steps: 3.1)利用输入模块将训练集E中的图像统一缩减为r:l大小的图像，其中r为缩减后的长度像素值，l为缩减后的宽度像素值，变换后的集合记为X＝{x1,x2...x3}；3.1) Use the input module to uniformly reduce the images in the training set E to images of size r:l, where r is the reduced length pixel value, l is the reduced width pixel value, and the transformed set is denoted as X={ x 1 , x 2 ... x 3 }; 3.2)利用主干模块通过残差结构提取集合X中每一帧的轮廓特征，残差结构公式如下：3.2) Use the backbone module to extract the contour features of each frame in the set X through the residual structure. The residual structure formula is as follows: F(x)＝f(x)+f(f(x))+f(f(f(x)))F(x)=f(x)+f(f(x))+f(f(f(x))) 其中f(x)＝δ(W*x)+cwhere f(x)=δ(W*x)+c 式中，每一帧图像x作为卷积层的输入，W为卷积需要学习的参数，δ为激活函数，F(x)为残差结构的输出结果；In the formula, each frame of image x is used as the input of the convolution layer, W is the parameter to be learned by the convolution, δ is the activation function, and F(x) is the output result of the residual structure; 3.3)在检测头模块中设置分类融合的输入参数的结构FCLS，公式如下：3.3) Set the structure F CLS of the input parameters of classification fusion in the detection head module, the formula is as follows: FCLS＝δ[WCLS*F(x)]+cF CLS = δ[W CLS *F(x)]+c 式中，F(x)为残差结构的输出结果，WCLS为训练参数，δ为激活函数，c为常数项；In the formula, F(x) is the output result of the residual structure, W CLS is the training parameter, δ is the activation function, and c is the constant term; 3.4)在检测头模块中设置非线性回归连接参数的结构
Figure FDA0003202921140000031
公式如下：3.4) Setting the structure of nonlinear regression connection parameters in the detection head module
Figure FDA0003202921140000031
The formula is as follows:
Figure FDA0003202921140000032
Figure FDA0003202921140000032
 式中，FCLS为参数融合的输入部分，W1，W2为训练参数，δ为激活函数，c为常数项，LCR连接为非线性连接，其作用为加强分类与回归之间的联系；In the formula, F CLS is the input part of parameter fusion, W 1 , W 2 are training parameters, δ is the activation function, c is a constant term, and the LCR connection is a nonlinear connection, which is used to strengthen the connection between classification and regression; 3.5)计算回归的加强参数
Figure FDA0003202921140000033
计算公式如下：3.5) Calculate the reinforcement parameters for regression
Figure FDA0003202921140000033
Calculated as follows:
Figure FDA0003202921140000034
Figure FDA0003202921140000034
 式中，Freg为网络原先的回归参数，
Figure FDA0003202921140000035
为经过加强的回归参数；In the formula, F reg is the original regression parameter of the network,
Figure FDA0003202921140000035
are the enhanced regression parameters; 3.6)在检测头模块中设置非线性分类连接参数的结构
Figure FDA0003202921140000036
公式如下：3.6) The structure of setting the nonlinear classification connection parameters in the detection head module
Figure FDA0003202921140000036
The formula is as follows:
Figure FDA0003202921140000037
Figure FDA0003202921140000037
 式中，W3为训练参数，
Figure FDA0003202921140000038
为经过加强的回归参数，
Figure FDA0003202921140000039
为非线性分类连接参数的结构；In the formula , W3 is the training parameter,
Figure FDA0003202921140000038
are the enhanced regression parameters,
Figure FDA0003202921140000039
is the structure of the connection parameters for nonlinear classification; 3.7)计算分类的加强参数
Figure FDA00032029211400000310
计算公式如下：3.7) Calculate the reinforcement parameters for classification
Figure FDA00032029211400000310
Calculated as follows:
Figure FDA00032029211400000311
Figure FDA00032029211400000311
 式中，Fcls为网络原先的分类参数，
Figure FDA00032029211400000312
为经过加强的分类参数。In the formula, F cls is the original classification parameter of the network,
Figure FDA00032029211400000312
are the enhanced classification parameters. 5.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤4)中利用UFL函数优化RetinaXNet网络模型的权值，优化公式如下：5. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, utilize UFL function to optimize the weight of RetinaXNet network model in described step 4), optimization formula is as follows:
Figure FDA0003202921140000041
Figure FDA0003202921140000041
 式中，y为真实标签，其取值为0或1，
Figure FDA0003202921140000042
为UFL的动态调整因子参数，γ为调整样本权重的速率，α为权重参数。In the formula, y is the real label, and its value is 0 or 1,
Figure FDA0003202921140000042
is the dynamic adjustment factor parameter of UFL, γ is the rate at which the weight of the sample is adjusted, and α is the weight parameter. 6.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤5)训练RetinaXNet网络模型包括如下步骤：6. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, described step 5) training RetinaXNet network model comprises the steps: 5.1)计算准确率accuracy，计算公式如下：5.1) Calculate the accuracy rate, and the calculation formula is as follows:
Figure FDA0003202921140000043
Figure FDA0003202921140000043
 式中，accuracy为准确率，TP代表测试集T中网络输出为正样本，且参考标准也为正样本的个数；TN代表网络输出为负样本，且参考标准也为负样本的个数；FP代表网络输出为正样本，但参考标准为负样本的个数；FN代表网络输出为负样本，但参考标准为正样本的个数；In the formula, accuracy is the accuracy rate, TP represents that the network output in the test set T is a positive sample, and the reference standard is also the number of positive samples; TN represents that the network output is a negative sample, and the reference standard is also the number of negative samples; FP represents that the network output is a positive sample, but the reference standard is the number of negative samples; FN represents that the network output is a negative sample, but the reference standard is the number of positive samples; 5.2)计算召回率recall，计算公式如下：5.2) Calculate the recall rate recall, the calculation formula is as follows:
Figure FDA0003202921140000044
Figure FDA0003202921140000044
 式中，recall为召回率TP，代表判被判定为正样本，即正样本的个数；FN代表被判定为负样本，但事实上是正样本的个数；In the formula, recall is the recall rate TP, which represents the number of positive samples that are judged to be positive samples; FN represents the number of positive samples that are judged to be negative samples; 5.3)计算F1值，计算公式如下：5.3) Calculate the F 1 value, the calculation formula is as follows:
Figure FDA0003202921140000045
Figure FDA0003202921140000045
 式中，F1是对accuracy和recall的平衡，其综合准确率和召回率的计算结果；In the formula, F 1 is the balance of accuracy and recall, and the calculation result of its comprehensive accuracy and recall; 5.4)判断F1是否小于t，如小于则转步骤5.1)重新训练，否则转步骤6)。5.4) Determine whether F1 is less than t, if it is less than, go to step 5.1) to retrain, otherwise go to step 6). 7.如权利要求1所述的基于RetinaXNet的建筑夜景灯光异常检测方法，其特征在于，所述步骤6)判断夜景灯是否异常包括如下步骤：7. the abnormal detection method of building night scene lights based on RetinaXNet as claimed in claim 1, is characterized in that, described step 6) judges whether night scene lights are abnormal and comprises the steps: 6.1)将网络输出结果映射回原图，映射到原图的坐标、长和高公式如下：6.1) Map the network output results back to the original image, and the coordinates, length and height formulas mapped to the original image are as follows:
Figure FDA0003202921140000051
Figure FDA0003202921140000051
Figure FDA0003202921140000052
Figure FDA0003202921140000052
Figure FDA0003202921140000053
Figure FDA0003202921140000053
Figure FDA0003202921140000054
Figure FDA0003202921140000054
 式中，
Figure FDA0003202921140000055
为原图像的长和高，x、y、W、H分别为检测框的左上角坐标与检测框的长和高，xN、yN、WN、HN为映射到原图的坐标与长和高；In the formula,
Figure FDA0003202921140000055
are the length and height of the original image, x, y, W, H are the coordinates of the upper left corner of the detection frame and the length and height of the detection frame, respectively, x N , y N , W N , H N are the coordinates mapped to the original image and length and height; 6.2)异常检测，检测公式如下：6.2) Anomaly detection, the detection formula is as follows:
Figure FDA0003202921140000056
Figure FDA0003202921140000056
 式中，Warn＝0代表无异常，Warn＝1代表出现异常，触发报警器。In the formula, Warn=0 represents no abnormality, and Warn=1 represents abnormality and triggers the alarm.
CN202110909371.1A 2021-08-09 2021-08-09 RetinaXNet-based building night scene lamplight anomaly detection method Active CN113762084B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110909371.1A CN113762084B (en) 2021-08-09 2021-08-09 RetinaXNet-based building night scene lamplight anomaly detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110909371.1A CN113762084B (en) 2021-08-09 2021-08-09 RetinaXNet-based building night scene lamplight anomaly detection method

Publications (2)

Publication Number Publication Date
CN113762084A true CN113762084A (en) 2021-12-07
CN113762084B CN113762084B (en) 2024-11-29

Family

ID=78788759

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110909371.1A Active CN113762084B (en) 2021-08-09 2021-08-09 RetinaXNet-based building night scene lamplight anomaly detection method

Country Status (1)

Country Link
CN (1) CN113762084B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115294456A (en) * 2022-08-23 2022-11-04 山东巍然智能科技有限公司 Building lightening project detection method, equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709511A (en) * 2016-12-08 2017-05-24 华中师范大学 Urban rail transit panoramic monitoring video fault detection method based on depth learning
WO2019169895A1 (en) * 2018-03-09 2019-09-12 华南理工大学 Fast side-face interference resistant face detection method
CN112200019A (en) * 2020-09-22 2021-01-08 江苏大学 A fast building night scene lighting lighting fault detection method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709511A (en) * 2016-12-08 2017-05-24 华中师范大学 Urban rail transit panoramic monitoring video fault detection method based on depth learning
WO2019169895A1 (en) * 2018-03-09 2019-09-12 华南理工大学 Fast side-face interference resistant face detection method
CN112200019A (en) * 2020-09-22 2021-01-08 江苏大学 A fast building night scene lighting lighting fault detection method

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115294456A (en) * 2022-08-23 2022-11-04 山东巍然智能科技有限公司 Building lightening project detection method, equipment and storage medium

Also Published As

Publication number Publication date
CN113762084B (en) 2024-11-29

Similar Documents

Publication Publication Date Title
CN111339882B (en) Power transmission line hidden danger detection method based on example segmentation
CN108615226B (en) An Image Dehazing Method Based on Generative Adversarial Networks
CN111784633A (en) An automatic detection algorithm for insulator defects for power inspection video
CN109712127B (en) A transmission line fault detection method for machine patrol video stream
CN116485709A (en) A Bridge Concrete Crack Detection Method Based on YOLOv5 Improved Algorithm
CN114926407A (en) Steel surface defect detection system based on deep learning
CN108596883A (en) It is a kind of that method for diagnosing faults is slid based on the Aerial Images stockbridge damper of deep learning and distance restraint
CN113327255A (en) Power transmission line inspection image processing method based on YOLOv3 detection, positioning and cutting and fine-tune
CN110009601A (en) Large-Scale Equipment irregular contour detection method of surface flaw based on HOG
CN114757925A (en) Method and system for defect detection of non-contact high-voltage circuit breaker
CN117451716A (en) A method for detecting surface defects of industrial products
CN115631407B (en) Underwater transparent biological detection based on fusion of event camera and color frame image
CN113762084B (en) RetinaXNet-based building night scene lamplight anomaly detection method
CN113781388A (en) Image recognition method and device for hidden dangers of transmission line channel based on image enhancement
CN112069894A (en) A Method for Identifying Strands of Wires Based on Faster-RCNN Model
CN114743099A (en) Method and device for identifying faults of train key components
CN107992799B (en) Preprocessing methods for smoke detection applications
CN111310899B (en) Power defect recognition method based on symbiotic relationship and small sample learning
CN112132088A (en) Inspection point location missing inspection identification method
CN117292183A (en) Power transmission inspection system and method based on image processing
CN117671452A (en) Construction method and system of broken gate detection model of lightweight up-sampling YOLOX
CN114998315B (en) Defect detection model training method and device based on deep learning
CN112801072B (en) A computer vision-based elevator non-leveling door opening fault identification device and method
CN116994161A (en) An insulator defect detection method based on improved YOLOv5
CN117789017A (en) Attention mechanism-based yolov7 bow net electric spark detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20240418

Address after: 1003, Building A, Zhiyun Industrial Park, No. 13 Huaxing Road, Henglang Community, Dalang Street, Longhua District, Shenzhen City, Guangdong Province, 518000

Applicant after: Shenzhen Wanzhida Technology Transfer Center Co.,Ltd.

Country or region after: China

Address before: 212013 No. 301, Xuefu Road, Zhenjiang, Jiangsu

Applicant before: JIANGSU University

Country or region before: China

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20241031

Address after: 334000, No. 2 Luzhou Avenue, Fengxi Street, Guangfeng District, Shangrao City, Jiangxi Province

Applicant after: Jiangxi Wangkai Construction Co.,Ltd.

Country or region after: China

Address before: 1003, Building A, Zhiyun Industrial Park, No. 13 Huaxing Road, Henglang Community, Dalang Street, Longhua District, Shenzhen City, Guangdong Province, 518000

Applicant before: Shenzhen Wanzhida Technology Transfer Center Co.,Ltd.

Country or region before: China

GR01 Patent grant
GR01 Patent grant