CN112584135A

CN112584135A - Monitoring equipment fault identification method, device, equipment and storage medium

Info

Publication number: CN112584135A
Application number: CN202011476101.8A
Authority: CN
Inventors: 黄惠群
Original assignee: Ping An International Smart City Technology Co Ltd
Current assignee: Ping An International Smart City Technology Co Ltd
Priority date: 2020-12-15
Filing date: 2020-12-15
Publication date: 2021-03-30

Abstract

The invention relates to the technical field of pedestal operation and maintenance, and discloses a monitoring equipment fault identification method, a monitoring equipment fault identification device, monitoring equipment and a storage medium. The method comprises the following steps: acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment; initiating heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment; judging whether the monitoring equipment normally operates according to the heartbeat detection feedback result; if so, screening the target frame images from the target video data through a preset screening rule, and determining the similarity between every two target frame images according to a preset image feature extraction model so as to determine whether the camera of the monitoring equipment breaks down. The scheme can monitor and identify the monitoring equipment of the monitoring station, provides real-time and accurate video monitoring data for monitoring personnel, and solves the technical problem of low operation efficiency of the monitoring equipment.

Description

Monitoring equipment fault identification method, device, equipment and storage medium

Technical Field

The invention relates to the technical field of pedestal operation and maintenance, in particular to a monitoring equipment fault identification method, a monitoring equipment fault identification device, monitoring equipment and a storage medium.

Background

In the modern society of industrial and agricultural rapid development, human labor and water resources for production and living are not very dense. Meanwhile, improper and irregular sewage discharge gradually causes water body pollution and water quality deterioration, and especially high-pollution industrial wastewater causes serious threat to the safety of the ecological environment due to discharge under the condition of not managing and controlling strict sewage discharge.

In order to realize the control of water pollution and water resource protection, a water quality monitoring system is in the process of transportation. The water quality monitoring is a process of monitoring and measuring the types of pollutants in the water body, the concentrations and the variation trends of various pollutants and evaluating the water quality condition. At present, most water quality monitoring stations in China are provided with video monitoring equipment and the like, but due to the fact that the environment around rivers is complex and the humidity is high, the electronic monitoring equipment has high faults, and the situations of long-time blocking, disconnection, even other artificial damages and the like easily occur. Therefore, an effective method is needed to judge the operation status of the video monitoring device to ensure the complete operation of the whole automatic water quality monitoring system.

Disclosure of Invention

The invention mainly aims to solve the technical problems of low operation efficiency and poor timeliness of monitoring equipment and improve the working efficiency of water quality monitoring.

The invention provides a monitoring equipment fault identification method in a first aspect, which comprises the following steps:

acquiring an IP address and a preset detection port of the monitoring equipment, and reading target video data acquired by the monitoring equipment;

performing heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

judging whether the monitoring equipment normally operates or not based on the heartbeat detection feedback result;

if the monitoring equipment normally operates, screening a target frame image from the target video data through a preset screening rule;

and determining the similarity between every two target frame images through a preset image feature extraction model, and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

Optionally, in a first implementation manner of the first aspect of the present invention, before the obtaining an IP address and a preset detection port of the monitoring device, and shooting a target area through the monitoring device to obtain target video data acquired by the monitoring device, the method further includes:

receiving a shooting instruction for the target area;

and shooting the target area based on the shooting instruction to obtain target video data of the target area.

Optionally, in a second implementation manner of the first aspect of the present invention, the screening, by using a preset screening rule, a target frame image from the target video data includes:

acquiring a scene change parameter of the target area according to historical video data of the target area, wherein the scene change parameter is used for indicating the stability of the target area;

and selecting an image from the target video according to the scene change parameter of the target area and a preset time interval, and taking the selected image as the target frame image one frame image at a time.

Optionally, in a third implementation manner of the first aspect of the present invention, the determining, by using a preset image feature extraction model, a similarity between each two of the target frame images, and based on the similarity between each two of the target frame images, determining whether a camera of the monitoring device fails includes:

inputting the target frame image into a preset image feature extraction model to obtain a feature vector of the target frame image on each layer with an output neuron as a feature;

respectively calculating the similarity between every two feature vectors of the target frame image for each layer;

calculating the similarity between every two target frame images through a regression algorithm according to the similarity between every two feature vectors of the target frame images, wherein the similarity between every two feature vectors calculated on each layer correspondingly takes different weights;

and determining whether the camera of the monitoring equipment fails or not based on the similarity between every two target frame images.

Optionally, in a fourth implementation manner of the first aspect of the present invention, before the determining, by using a preset image feature extraction model, a similarity between every two target frame images, and determining, based on the similarity between every two target frame images, whether a camera of the monitoring device fails, the method further includes:

selecting a deep convolutional neural network model, wherein the deep convolutional neural network model comprises M convolutional layers and N fully-connected layers;

selecting a specified K layer from the M + N layer for feature extraction, and taking neuron output of the K layer as a feature vector;

acquiring historical video data, and reading historical image data through the historical video data;

and inputting the historical image data serving as a training set into the deep convolutional neural network model, and training the deep convolutional neural network model based on a back propagation algorithm to obtain an image feature extraction model with K feature vectors.

Optionally, in a fifth implementation manner of the first aspect of the present invention, after performing heartbeat detection on the monitoring device through the IP address of the monitoring device and a preset detection port to obtain a heartbeat detection feedback result of the monitoring device, the method further includes:

if the heartbeat detection feedback result returned by the monitoring equipment is not received after the preset time threshold value is exceeded, identifying the target frame image and determining whether the monitoring equipment is disconnected;

if the monitoring equipment is disconnected, identifying the monitoring equipment as target fault equipment based on the IP address and restarting the monitoring equipment;

if the target failure equipment fails to restart, acquiring the GPS position information of the target failure equipment;

and sending alarm information to a monitoring center through the GPS position information of the target fault equipment, and intervening the target fault monitoring equipment.

A second aspect of the present invention provides a monitoring device fault identifying apparatus, including:

the first acquisition module is used for acquiring the IP address and the preset detection port of the monitoring equipment and reading target video data acquired by the monitoring equipment;

the heartbeat detection module is used for carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

the judging module is used for judging whether the monitoring equipment normally operates or not based on the heartbeat detection feedback result;

the screening module is used for screening a target frame image from the target video data through a preset screening rule if the monitoring equipment normally operates;

and the first feature extraction module is used for determining the similarity between every two target frame images through a preset image feature extraction model and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

Optionally, in a first implementation manner of the second aspect of the present invention, the monitoring device fault identifying apparatus further includes:

the receiving module is used for receiving a shooting instruction aiming at the target area;

and the shooting module is used for shooting the target area based on the shooting instruction to obtain the target video data of the target area.

Optionally, in a second implementation manner of the second aspect of the present invention, the screening module is specifically configured to:

Optionally, in a third implementation manner of the second aspect of the present invention, the first feature extraction module includes:

the input unit is used for inputting the target frame image into a preset image feature extraction model to obtain a feature vector of the target frame image on each layer with an output neuron as a feature;

the calculating unit is used for calculating the similarity between every two feature vectors of the target frame image for each layer; calculating the similarity between every two target frame images through a regression algorithm according to the similarity between every two feature vectors of the target frame images, wherein the similarity between every two feature vectors calculated on each layer correspondingly takes different weights; and determining whether the camera of the monitoring equipment fails or not based on the similarity between every two target frame images.

Optionally, in a fourth implementation manner of the second aspect of the present invention, the monitoring device fault identifying apparatus further includes:

optionally, in a fifth implementation manner of the second aspect of the present invention, the monitoring device fault identifying apparatus further includes:

the identification module is used for identifying the target frame image and determining whether the monitoring equipment is disconnected or not when a heartbeat detection feedback result returned by the monitoring equipment is not received after a preset time threshold value is exceeded;

the identification module is used for identifying the monitoring equipment as target fault equipment and restarting the monitoring equipment based on the IP address when the monitoring equipment is disconnected;

the second acquisition module is used for acquiring the GPS position information of the target failure equipment when the target failure equipment fails to restart;

and the sending module is used for sending alarm information to a monitoring center through the GPS position information of the target fault equipment and intervening the target fault monitoring equipment.

A third aspect of the present invention provides a monitoring device fault identifying device, including: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invokes the instructions in the memory to cause the monitoring device fault identification device to perform the monitoring device fault identification method described above.

A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to execute the monitoring device fault identification method described above.

In the technical scheme provided by the invention, the IP address and the preset detection port of the monitoring equipment are obtained, and the target video data acquired by the monitoring equipment is read; initiating heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment; judging whether the monitoring equipment normally operates according to the heartbeat detection feedback result; if so, screening the target frame images from the target video data through a preset screening rule, and determining the similarity between every two target frame images according to a preset image feature extraction model so as to determine whether the camera of the monitoring equipment breaks down. The scheme can monitor and identify the monitoring equipment of the monitoring station, provides real-time and accurate video monitoring data for monitoring personnel, and solves the technical problem of low operation efficiency of the monitoring equipment.

Drawings

FIG. 1 is a schematic diagram of a first embodiment of a monitoring device fault identification method of the present invention;

FIG. 2 is a schematic diagram of a monitoring device fault identification method according to a second embodiment of the present invention;

FIG. 3 is a schematic diagram of a monitoring device fault identification method according to a third embodiment of the present invention;

FIG. 4 is a schematic diagram of a fourth embodiment of the monitoring device fault identification method of the present invention;

FIG. 5 is a schematic diagram of a fifth embodiment of the monitoring device fault identification method of the present invention;

FIG. 6 is a schematic diagram of a first embodiment of the fault identification device of the monitoring equipment of the present invention;

FIG. 7 is a schematic diagram of a second embodiment of the fault recognition apparatus for a monitoring device according to the present invention;

fig. 8 is a schematic diagram of an embodiment of the monitoring device fault identification device of the present invention.

Detailed Description

The embodiment of the invention provides a monitoring equipment fault identification method, a monitoring equipment fault identification device, monitoring equipment and a storage medium, wherein in the technical scheme of the invention, an IP address and a preset detection port of the monitoring equipment are obtained firstly, and target video data acquired by the monitoring equipment are read; initiating heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment; judging whether the monitoring equipment normally operates according to the heartbeat detection feedback result; if so, screening the target frame images from the target video data through a preset screening rule, and determining the similarity between every two target frame images according to a preset image feature extraction model so as to determine whether the camera of the monitoring equipment breaks down. The scheme can monitor and identify the monitoring equipment of the monitoring station, provides real-time and accurate video monitoring data for monitoring personnel, and solves the technical problem of low operation efficiency of the monitoring equipment.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

For convenience of understanding, a specific flow of the embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of a monitoring device fault identification method in the embodiment of the present invention includes:

101. acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment;

in this embodiment, the system obtains real-time video stream data of the video monitoring device and obtains GPS location information of the video monitoring device through the configured IP and port address of the video monitoring device, which is convenient for maintenance.

In this embodiment, in a target area into which a target object such as a person or a wild animal is prohibited from entering, the target area may be photographed by a video monitoring apparatus (photographing device) for the personal safety and the property safety of a user, so as to obtain target video data of the target area. The shooting device can be a panoramic shooting device or a hemispherical shooting device, and the target area can be an area of a river or a lake of the monitored water quality.

When a sensor in the monitoring equipment detects that human or animals and the like enter a monitoring area, a shooting device of the monitoring equipment is triggered to shoot the monitoring area to obtain video data of the monitoring area, for example, the sensor transmits infrared spectrum and receives reflected infrared spectrum to calculate the time interval between the transmitted infrared spectrum and the reflected infrared spectrum, when the time interval is lower than a preset time threshold, the human or the animal entering the target area is determined, the shooting device of the monitoring equipment is triggered to shoot the monitoring area to obtain the video data of the monitoring area.

Meanwhile, in order to reduce the pressure of processing image data by the monitoring equipment, the monitoring equipment can monitor a target area in a certain time period, specifically, a shooting time period is set for the shooting device, and when the time is within the shooting time period of the shooting device, the shooting device of the monitoring equipment is triggered to shoot the monitoring area, so that the video data of the monitoring area is obtained.

102. Carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

in this embodiment, the intermediate server receives a heartbeat subscription request sent by a consuming terminal to a specified terminal (video monitoring device), where the heartbeat subscription request carries heartbeat detection time interval information. In network information transmission, a heartbeat mechanism is a mechanism that regularly sends a self-defined structure (heartbeat packet, heartbeat data) to let the other side know that the other side is still alive so as to ensure the validity of connection, wherein the heartbeat state is classified as static (i.e. abnormal) or active (i.e. normal). The heartbeat detection of the invention is to detect whether the appointed terminal is abnormal or not by using the heartbeat detection feedback result.

The heartbeat detection request corresponding to the initiated heartbeat detection in this embodiment may be any heartbeat detection request, for example, the heartbeat detection request includes information such as a time stamp, a heartbeat interval, and a service IP. The designated terminal refers to a terminal or a server terminal device which interacts with the presence information of the consumption terminal, such as a social software server, a video monitoring device and the like. The heartbeat subscription request is a request for initiating heartbeat detection to a specified terminal according to the heartbeat detection time interval information. The consumption terminal and the designated terminal are relatively speaking, that is, when a certain terminal (or server) requires heartbeat detection of another terminal (or server), the certain terminal (or server) is the consumption terminal, and correspondingly, the other terminal (or server) is the designated terminal; otherwise, the certain terminal (or server) is the designated terminal, and correspondingly, the other terminal (or server) is the consuming terminal.

In this embodiment, since the number of the video monitoring device systems recorded by the third-party heartbeat detection center may be extremely large, and the computational resource and the computational capability of the execution main body are limited, the addresses (IP addresses) and the call parameters (detection interfaces) of the video monitoring device systems are obtained in sequence, and the earlier obtained video monitoring device system enters the heartbeat detection link, which is more beneficial to the implementation of the execution main body and the purpose of heartbeat detection.

After the address and the calling parameter of the video monitoring equipment system are obtained, the heartbeat detection can be performed on the video monitoring equipment system with the obtained address and the calling parameter, so that a heartbeat detection feedback result about the video monitoring equipment system is obtained, and the heartbeat detection feedback result comprises a fault code and running state information of the video monitoring equipment system, a calling state and the like.

It will be appreciated that after heartbeat detection is performed, the normally returned heartbeat detection feedback results include the calling status of the callee system, such as whether it can be called, call latency, and so on. The calling time delay can also be divided into high time delay and low time delay, wherein the high time delay can greatly reduce the real-time property of calling of the calling system, so that the calling system can be informed together when the corresponding calling system is informed in the subsequent steps. After the heartbeat detection feedback result is obtained, fault identification can be carried out on the video monitoring equipment according to the heartbeat detection feedback result.

103. Judging whether the monitoring equipment normally operates or not based on a heartbeat detection feedback result;

in this embodiment, after the execution main body obtains the heartbeat detection feedback result, whether the monitoring device normally operates is determined. If the monitoring equipment has a fault, the heartbeat detection feedback result of the abnormal video monitoring equipment system can be sent to a calling side system corresponding to the abnormal (video monitoring equipment) system, wherein the abnormal system refers to a called side system with the calling state being abnormal in the heartbeat detection feedback result, so that the corresponding calling side system is informed in real time, and the calling side system can take corresponding emergency and remedial measures to process conveniently. In addition, the execution main body can also send the heartbeat detection feedback result of the abnormal system to the maintenance personnel corresponding to the abnormal system, so that the maintenance personnel can know the abnormal information of the abnormal system in time and carry out system fault repair as soon as possible.

104. If so, screening a target frame image from the target video data through a preset screening rule;

in this embodiment, in order to improve the efficiency of image recognition, the monitoring device may screen the target frame image from the target video data according to a preset screening rule.

105. And determining the similarity between every two target frame images through a preset image feature extraction model, and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

In this embodiment, a regression algorithm may be used to calculate the similarity between two pictures, the similarity between every two feature vectors calculated on each layer may be weighted differently, the calculation result of the similarity between the pictures may be more accurate by adjusting the weight value, and the calculation accuracy may be improved.

In this embodiment, the similarity of the pictures is represented by floating point numbers between [0,1], and a larger numerical value indicates more similarity, and a smaller numerical value indicates less similarity. Optionally, the calculating the similarity between every two feature vectors of the target frame image for each layer may include:

and respectively calculating the similarity between the feature vector of the first picture and the feature vector of the second picture for each layer by using a cosine algorithm, a Jaccard algorithm or a Pearson algorithm.

In this embodiment, optionally, the calculating the similarity between two target frame images according to the similarity between two feature vectors of the target frame images on each layer may include: and calculating the similarity of the first picture and the second picture according to the similarity between the feature vector of the first picture and the feature vector of the second picture on each layer by using a linear regression algorithm or a logistic regression algorithm.

In the embodiment of the invention, the IP address and the preset detection port of the monitoring equipment are obtained, and the target video data acquired by the monitoring equipment is read; initiating heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment; judging whether the monitoring equipment normally operates according to the heartbeat detection feedback result; if so, screening the target frame images from the target video data through a preset screening rule, and determining the similarity between every two target frame images according to a preset image feature extraction model so as to determine whether the camera of the monitoring equipment breaks down. The scheme can monitor and identify the monitoring equipment of the monitoring station, provides real-time and accurate video monitoring data for monitoring personnel, and solves the technical problem of low operation efficiency of the monitoring equipment.

Referring to fig. 2, a second embodiment of the method for identifying faults of monitoring equipment according to the embodiment of the present invention includes:

201. receiving a shooting instruction for a target area;

in this embodiment, a shooting instruction sent by a user is received, and a shooting device of the monitoring device is triggered to shoot the monitored area, so as to obtain video data of the monitored area, and the user can send the shooting instruction to the shooting device in a touch (such as pressing a key, sliding, or clicking) or voice manner.

202. Shooting the target area based on the shooting instruction to obtain target video data of the target area;

in this embodiment, temperature information of a target area is acquired through a sensor, and when the temperature information of the target area indicates that a temperature value of the target area is greater than a preset temperature value, a step of shooting the target area through a shooting device to obtain target video data of the target area is executed; or receiving a shooting instruction aiming at the target area, and executing the step of shooting the target area by the shooting device to obtain target video data of the target area.

In order to reduce the pressure of processing image data by the monitoring equipment, the monitoring equipment can trigger a shooting device to shoot a video according to parameters in a target area, specifically, temperature information of the target area is obtained through a sensor, when the temperature information of the target area indicates that the temperature value of the target area is greater than a preset temperature value, an object with temperature in the target area breaks into the target area, the object can be human or animal, in order to avoid that the broken-into object is human or animal, the shooting device of the monitoring equipment is triggered to shoot the monitoring area, and video data of the monitoring area are obtained. The method comprises the steps of receiving a shooting instruction sent by a user, triggering a shooting device of the monitoring equipment to shoot a monitored area to obtain video data of the monitored area, and sending the shooting instruction to the shooting device by the user through touch (such as pressing a key, sliding or clicking) or voice and the like.

203. Acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment;

204. carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

205. when the heartbeat detection feedback result returned by the monitoring equipment is not received after the preset time threshold value is exceeded, identifying a target frame image and judging whether the monitoring equipment is disconnected or not;

in this embodiment, the heartbeat detection request may be sent at a preset time interval, and the preset time may be set by the video monitoring device, or may be set by the video monitoring device according to the preset time set by the intermediate server, or may be set by the video monitoring device according to the heartbeat detection time interval information carried by the heartbeat subscription request. Accordingly, a heartbeat detection feedback result is received.

In this embodiment, a heartbeat detection request is sent to the consuming terminal that sends the heartbeat subscription request according to a time interval specified by the heartbeat detection time interval information. Because the heartbeat subscription request sent by the consumption terminal to the video monitoring device carries heartbeat detection time interval information, that is, the consumption terminal requires a heartbeat detection time interval, the heartbeat packet is sent according to the heartbeat detection time interval, and the sending of the heartbeat detection request can be completed. And the consumption terminal receiving the heartbeat detection feedback result can complete the heartbeat detection according to the heartbeat detection feedback result so as to determine the state (abnormal or normal) of the video monitoring equipment. And if the heartbeat detection feedback result returned by the monitoring equipment is not received after the preset time threshold value is exceeded, identifying the collected target frame image, and judging whether the equipment is disconnected or not according to the image identification result and the heartbeat detection feedback result.

206. If yes, identifying the monitoring equipment as target fault equipment based on the IP address and restarting;

in this embodiment, the collected target frame image is identified, and if it is determined that the device is disconnected according to the image identification result and the heartbeat detection feedback result, the monitoring device is marked as a faulty device according to the IP address corresponding to the monitoring device. The original work flow that only can rely on the manual work to carry out troubleshooting is changed, and the problems of high manual troubleshooting cost, high difficulty, much consumed time and the like are solved.

207. If not, acquiring the GPS position information of the target fault equipment;

in this embodiment, when a monitoring device fails, the failed monitoring device is restarted; if the restart fails, the specific position information of the monitoring equipment is determined, and the original work flow which only can be manually checked for faults is changed. The system is convenient for workers to replace new equipment, provides more real-time and accurate video monitoring data for monitoring personnel, ensures the normal operation of the whole automatic river water quality monitoring system, ensures that the river is not polluted, and completely inhibits the phenomena of sewage stealing, draining, stealing and the like

208. And sending alarm information to a monitoring center through the GPS position information of the target fault equipment, and intervening the target fault monitoring equipment.

In this embodiment, the restart failure or the phenomenon after the restart is still notified to the monitoring personnel by a short message for processing.

And in the operation of the equipment, after the faulty equipment is detected, the fault is stored, the alarm is reported to the system in real time according to the requirement, and monitoring personnel are informed to process the alarm.

209. Judging whether the monitoring equipment normally operates or not based on a heartbeat detection feedback result;

210. if so, screening a target frame image from the target video data through a preset screening rule;

211. and determining the similarity between every two target frame images through a preset image feature extraction model, and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

In this embodiment, the blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

Step 209-211 in this embodiment is similar to step 103-105 in the first embodiment, and will not be described herein again.

Referring to fig. 3, a third embodiment of the method for identifying a monitoring device fault according to the embodiment of the present invention includes:

301. acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment;

302. carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

303. judging whether the monitoring equipment normally operates or not based on a heartbeat detection feedback result;

304. acquiring a scene change parameter of a target area according to historical video data of the target area, wherein the scene change parameter is used for indicating the stability of the target area;

in this embodiment, because the scene change parameters of the target area have greater similarity in the same time period, the monitoring device may obtain the current time, obtain historical video data of the target area corresponding to the current time, and determine the scene change parameters of the target area according to the historical video data of the target area corresponding to the current time. For example, when the current time is 6:00 nighttime, the monitoring device may acquire historical video data of the target area in a time period of 6:00 to 12:00 nighttime, and acquire the scene change parameter of the target area according to the historical video data of the target area in the time period of 6:00 to 12:00 nighttime.

305. Selecting an image from a target video according to a scene change parameter of a target area and a preset time interval, and taking the selected image as a target frame image one frame image at a time;

in this embodiment, the monitoring device may obtain the reference image and the target frame image according to the scene change parameter of the target area, and specifically, the monitoring device may obtain historical video data of the target area within a preset time period, obtain the scene change parameter of the target area according to the historical video data, obtain the reference image according to the scene change parameter of the target area, and screen the target frame image from the target video data according to the scene change parameter.

In the same time period, the scene change parameters of the target area have greater similarity, so that the monitoring device can acquire the current time, acquire historical video data of the target area corresponding to the current time, and determine the scene change parameters of the target area according to the historical video data of the target area corresponding to the current time. For example, when the current time is 6:00 nighttime, the monitoring device may acquire historical video data of the target area in a time period of 6:00 to 12:00 nighttime, and acquire the scene change parameter of the target area according to the historical video data of the target area in the time period of 6:00 to 12:00 nighttime.

And when the scene change parameter indicates that the stability of the target area is greater than or equal to a preset stability value, selecting images from the target video according to a first preset time interval, and taking the selected images as target frame images one frame at a time.

306. And determining the similarity between every two target frame images through a preset image feature extraction model, and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

The steps 301-303-306 in this embodiment are similar to the steps 101-103-105 in the first embodiment, and are not described herein again.

Referring to fig. 4, a fourth embodiment of the method for identifying a monitoring device fault according to the embodiment of the present invention includes:

401. acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment;

402. carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

403. judging whether the monitoring equipment normally operates or not based on a heartbeat detection feedback result;

404. if so, screening a target frame image from the target video data through a preset screening rule;

405. inputting the target frame image into a preset image feature extraction model to obtain a feature vector of the target frame image on each layer with an output neuron as a feature;

in this embodiment, there are a plurality of target frame images, any two target frame images are selected, and the similarity between each two target frame images is determined by the method provided in this embodiment.

406. Respectively calculating the similarity between every two feature vectors of the target frame image for each layer;

in this embodiment, each layer of the output neuron as a feature usually includes multiple layers, and each layer of the output neuron can obtain a feature vector corresponding to the target frame image.

For example, if there are 3 layers, i.e., a layer a, a layer B, and a layer C, in the feature extraction model, which output neurons as features, the feature vector Va of the first target frame image output on the layer a can be obtained by inputting the first target frame image into the model₁Characteristic vector Vb output at layer B₁And the feature vector Vc output at the C layer₁(ii) a After the second target frame image is input into the model, the feature vector Va of the second target frame image output on the A layer can be obtained₂Characteristic vector Vb output at layer B₂And the feature vector Vc output at the C layer₂. When calculating the similarity of the feature vectors, Va may be calculated separately₁And Va₂Degree of similarity, Vb₁And Vb₂Degree of similarity ofAnd Vc₁And Vc₂The similarity of (c).

407. And calculating the similarity between every two target frame images through a regression algorithm according to the similarity between every two feature vectors of the target frame images, wherein the similarity between every two feature vectors calculated on each layer correspondingly takes different weights.

In this embodiment, a regression algorithm may be used to calculate the similarity between each two images of the target frame, the similarity between each two feature vectors calculated on each layer may be weighted differently, the calculation result of the image similarity may be more accurate by adjusting the weight value, and the calculation accuracy may be improved.

In this embodiment, the similarity of the images is characterized by floating point numbers between [0,1], and a larger numerical value indicates more similar, and a smaller numerical value indicates less similar.

In this embodiment, optionally, the calculating the similarity between each two of the feature vectors of the first target frame image and the feature vector of the second target frame image for each layer may include:

and respectively calculating the similarity between the feature vector of the first target frame image and the feature vector of the second target frame image for each layer by using a cosine algorithm, a Jaccard algorithm or a Pearson algorithm.

The cosine algorithm (cosine similarity) measures the similarity between two vectors by measuring the cosine value of the space angle of the inner product of the two vectors. The Jaccard algorithm and Pearson algorithm are also algorithms for calculating similarity, and will not be described herein. The cosine algorithm is described as an example, and can be represented by the following formula:

simlayern＝cosine(fvni，fvnj)；

wherein i and j represent any two different pictures, n represents the number of layers in the deep convolutional neural network, fvni represents a feature vector extracted by the picture i on the nth layer, fvnj represents a feature vector extracted by the picture j on the nth layer, and simlayern represents the similarity between every two feature vectors of the two pictures on the nth layer.

The linear regression algorithm and the logistic regression algorithm are both algorithms for regression calculation, and are not described herein again, and of course, in other implementation manners, other regression algorithms may also be adopted, which is not specifically limited in this embodiment. The following is a specific description of a linear regression algorithm, which can be expressed by the following formula:

wherein m represents the number of layers designated in the deep convolutional neural network for feature extraction, k is any one of m layers, k is 1.. the m, simlayerk is the similarity between every two feature vectors of two pictures calculated on the k layers, Wk is the weight corresponding to the simlayerk, similarity represents the similarity between the two pictures, and belongs to the [0,1] interval, the larger the value is, the more similar the pictures are, and the similarity of the same picture is 1. The weight may be set according to a test result of the test data, which is not specifically limited in this embodiment.

408. And determining whether the camera of the monitoring equipment fails or not based on the similarity between every two target frame images.

In this embodiment, optionally, the calculating the similarity between the first target frame image and the second target frame image according to the similarity between the feature vector of the first target frame image and the feature vector of the second target frame image on each layer may include: and calculating the similarity between every two target frame images according to the similarity between every two feature vectors of the first target frame image and the second target frame image on each layer by using a linear regression algorithm or a logistic regression algorithm.

The steps 401-404 in this embodiment are similar to the steps 101-104 in the first embodiment, and are not described herein again.

Referring to fig. 5, a fifth embodiment of the method for identifying faults of monitoring equipment according to the embodiment of the present invention includes:

501. acquiring an IP address and a preset detection port of monitoring equipment, and reading target video data acquired by the monitoring equipment;

502. carrying out heartbeat detection on the monitoring equipment through the IP address of the monitoring equipment and a preset detection port to obtain a heartbeat detection feedback result of the monitoring equipment, wherein the heartbeat detection feedback result comprises running state information and a fault code of the monitoring equipment;

503. judging whether the monitoring equipment normally operates or not based on a heartbeat detection feedback result;

504. if so, screening a target frame image from the target video data through a preset screening rule;

505. selecting a deep convolutional neural network model, wherein the deep convolutional neural network model comprises M convolutional layers and N full-connection layers;

in this embodiment, a Neural Network (NN) is a complex network system formed by a large number of simple processing units (called neurons) widely connected to each other, reflects many basic features of human brain functions, and is a highly complex nonlinear dynamical learning system. The neural network model has large-scale parallel, distributed storage and processing, self-organization, self-adaptation and self-learning capabilities, and is particularly suitable for processing inaccurate and fuzzy information processing problems which need to consider many factors and conditions simultaneously.

In this embodiment, the training data may be an image, and the type of the image is not limited, for example, the training data may be an image of an e-commerce industry or an image in an ImageNet database.

The deep convolutional neural network (deep convolutional neural network) is an artificial neural network, and is also one of deep supervised machine learning models and deep learning representing methods. Deep learning (deep learning) is a branch of machine learning that attempts to learn a multi-layered representation of features or concepts, forming more abstract high-level features by combining low-level features to discover a distributed feature representation of the data.

506. Selecting a specified K layer from the M + N layer for feature extraction, and taking neuron output of the K layer as a feature vector;

in the embodiment, a deep convolutional neural network model is selected, wherein the deep convolutional neural network model comprises M convolutional layers and N full-connection layers; selecting a specified K layer from the M + N layers for feature extraction, and taking neuron output of the K layer as a feature vector; training the deep convolutional neural network model by using training data based on a back propagation algorithm to obtain a frame image feature extraction model with K feature vectors.

Preferably, the K layers are all full-connection layers, or comprise a convolution layer and a full-connection layer. In this embodiment, the convolutional layers include, but are not limited to: convolution, Rectified Linear Units (neural network excitation functions), max-boosting (downsampling method), normalization, and the like, which are not particularly limited in this embodiment. In order to improve the calculation accuracy of the similarity, the K layers may be selected to include a convolution layer and a full link layer.

507. Acquiring historical video data, and reading historical image data through the historical video data;

in this embodiment, in a target area (a monitored river) into which a target object such as a person or an animal is prohibited from entering, a video monitoring device (a shooting device) shoots a working area to obtain video data of the working area of the video monitoring device, and uploads the video data to a preset database, where the video data is historical video data. The shooting device can be video monitoring equipment for monitoring river water quality and the like, and the working area can be an area such as a monitored river, a monitored lake and the like.

In one embodiment, when a sensor in the monitoring device detects that a person or an animal or the like intrudes into the monitoring area, a shooting device of the monitoring device is triggered to shoot the monitoring area to obtain video data of the monitoring area, for example, the sensor transmits an infrared spectrum and receives a reflected infrared spectrum, a time interval between the transmitted infrared spectrum and the reflected infrared spectrum is calculated, when the time interval is lower than a preset time threshold, it is determined that the person or the animal intrudes into the target area, and the shooting device of the monitoring device is triggered to shoot the monitoring area to obtain the video data of the monitoring area.

508. Inputting historical image data serving as a training set into a deep convolution neural network model, training the deep convolution neural network model based on a back propagation algorithm, and obtaining an image feature extraction model with K feature vectors;

in this embodiment, a Back Propagation (BP) algorithm is a supervised learning algorithm, and is often used to train a multi-layer perceptron and a forward neural network. In this embodiment, a back propagation algorithm is used to train the deep convolutional neural network model. The back propagation algorithm mainly comprises two links: and (4) carrying out loop iteration repeatedly through the two links until the response of the model to the input reaches a preset target range.

The advantage of using the deep convolutional neural network is that it can directly use the frame image as the input of the network, avoiding a series of complex preprocessing processes when extracting features manually, converting the explicit feature extraction mode into implicit feature extraction, greatly saving time and improving the efficiency of feature extraction. In addition, such network structures are highly invariant to translation, scaling, tilting, or other forms of deformation.

In this embodiment, a deep convolutional neural network (deep convolutional neural network) is an artificial neural network, and is also one of deep supervised machine learning models and deep learning representing methods. Deep learning (deep learning) is a branch of machine learning that attempts to learn a multi-layered representation of features or concepts, forming more abstract high-level features by combining low-level features to discover a distributed feature representation of the data.

509. And determining the similarity between every two target frame images through a preset image feature extraction model, and determining whether the camera of the monitoring equipment breaks down or not based on the similarity between every two target frame images.

The steps 501-504, 509 in the present embodiment are similar to the steps 101-104, 105 in the first embodiment, and are not described herein again.

With reference to fig. 6, the method for identifying a fault of a monitoring device in the embodiment of the present invention is described above, and a monitoring device fault identifying apparatus in the embodiment of the present invention is described below, where a first embodiment of the monitoring device fault identifying apparatus in the embodiment of the present invention includes:

a first obtaining module 601, configured to obtain an IP address and a preset detection port of the monitoring device, and read target video data acquired by the monitoring device;

a heartbeat detection module 602, configured to perform heartbeat detection on the monitoring device through an IP address of the monitoring device and a preset detection port to obtain a heartbeat detection feedback result of the monitoring device, where the heartbeat detection feedback result includes running state information and a fault code of the monitoring device;

a judging module 603, configured to judge whether the monitoring device operates normally based on the heartbeat detection feedback result;

a screening module 604, configured to screen a target frame image from the target video data according to a preset screening rule if the monitoring device operates normally;

the first feature extraction module 605 is configured to determine a similarity between each two of the target frame images through a preset image feature extraction model, and determine whether a camera of the monitoring device fails based on the similarity between each two of the target frame images.

Referring to fig. 7, a monitoring apparatus fault identifying apparatus according to a second embodiment of the present invention specifically includes:

In this embodiment, the monitoring device fault recognition apparatus further includes:

a receiving module 606, configured to receive a shooting instruction for the target area;

a shooting module 607, configured to shoot the target area based on the shooting instruction to obtain target video data of the target area

In this embodiment, the screening module 604 is specifically configured to:

In this embodiment, the first feature extraction module 605 includes:

an input unit 6051, configured to input the target frame image into a preset image feature extraction model, and obtain a feature vector of the target frame image on each layer where an output neuron is used as a feature;

a calculating unit 6052, configured to calculate similarity between every two feature vectors of the target frame image for each layer; calculating the similarity between every two target frame images through a regression algorithm according to the similarity between every two feature vectors of the target frame images, wherein the similarity between every two feature vectors calculated on each layer correspondingly takes different weights; and determining whether the camera of the monitoring equipment fails or not based on the similarity between every two target frame images.

a selecting module 608, configured to select a deep convolutional neural network model, where the deep convolutional neural network model includes M convolutional layers and N fully-connected layers;

a second feature extraction module 609, configured to select a specified K layer from the M + N layers for feature extraction, and use neuron outputs of the K layer as feature vectors;

the reading module 610 is configured to acquire historical video data and read historical image data through the historical video data;

the training module 611 is configured to input the historical image data as a training set to the deep convolutional neural network model, train the deep convolutional neural network model based on a back propagation algorithm, and obtain an image feature extraction model with K feature vectors.

Fig. 6 and fig. 7 describe the monitoring device fault identification apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the monitoring device fault identification apparatus in the embodiment of the present invention is described in detail from the perspective of hardware processing.

Fig. 8 is a schematic structural diagram of a monitoring device fault recognition device according to an embodiment of the present invention, where the monitoring device fault recognition device 800 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 810 (e.g., one or more processors) and a memory 820, and one or more storage media 830 (e.g., one or more mass storage devices) storing an application 833 or data 832. Memory 820 and storage medium 830 may be, among other things, transient or persistent storage. The program stored in storage medium 830 may include one or more modules (not shown), each of which may include a series of instruction operations for monitoring device fault identification device 800. Further, the processor 810 may be configured to communicate with the storage medium 830, and execute a series of instruction operations in the storage medium 830 on the monitoring device fault identification device 800 to implement the steps of the monitoring device fault identification method provided by the above-mentioned method embodiments.

The monitoring device fault identification device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input-output interfaces 860, and/or one or more operating systems 831, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that the monitoring device fault identification device configuration shown in fig. 8 does not constitute a limitation of the monitoring device fault identification device provided herein, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and may also be a volatile computer-readable storage medium, where instructions are stored, and when the instructions are executed on a computer, the instructions cause the computer to execute the steps of the monitoring device fault identification method.

The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A monitoring equipment fault identification method is characterized by comprising the following steps:

2. The method for identifying the fault of the monitoring equipment according to claim 1, wherein before the step of obtaining the IP address and the preset detection port of the monitoring equipment, and shooting the target area through the monitoring equipment to obtain the target video data acquired by the monitoring equipment, the method further comprises the following steps:

receiving a shooting instruction for the target area;

3. The monitoring equipment fault identification method according to claim 1, wherein the screening out target frame images from the target video data through a preset screening rule comprises:

4. The method for identifying the fault of the monitoring equipment as claimed in claim 1, wherein the determining the similarity between two images of the target frame through a preset image feature extraction model, and the determining whether the camera of the monitoring equipment has the fault or not based on the similarity between two images of the target frame comprises:

5. The method for identifying the fault of the monitoring equipment according to claim 1, wherein before the determining the similarity between two images of the target frame through a preset image feature extraction model and determining whether the camera of the monitoring equipment has the fault or not based on the similarity between two images of the target frame, the method further comprises:

6. The monitoring device fault identification method according to claims 1 to 5, wherein after the heartbeat detection of the monitoring device is performed through the IP address of the monitoring device and a preset detection port to obtain a heartbeat detection feedback result of the monitoring device, the method further comprises:

7. A monitoring device failure recognition apparatus, characterized in that the monitoring device failure recognition apparatus includes:

8. The monitoring device fault recognition apparatus according to claim 7, further comprising:

9. A monitoring device failure recognition device, characterized in that the monitoring device failure recognition device comprises: a memory having instructions stored therein and at least one processor, the memory and the at least one processor interconnected by a line;

the at least one processor invokes the instructions in the memory to cause the monitoring device fault identification device to perform the monitoring device fault identification method of any of claims 1-6.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a monitoring device fault identification method according to any one of claims 1 to 6.