CN116310760B - Intelligent water conservancy monitoring system based on machine vision - Google Patents
Intelligent water conservancy monitoring system based on machine vision Download PDFInfo
- Publication number
- CN116310760B CN116310760B CN202310370298.4A CN202310370298A CN116310760B CN 116310760 B CN116310760 B CN 116310760B CN 202310370298 A CN202310370298 A CN 202310370298A CN 116310760 B CN116310760 B CN 116310760B
- Authority
- CN
- China
- Prior art keywords
- water conservancy
- facility
- fault
- image
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 228
- 238000012544 monitoring process Methods 0.000 title claims abstract description 42
- 238000000605 extraction Methods 0.000 claims abstract description 20
- 238000007405 data analysis Methods 0.000 claims abstract description 12
- 239000013598 vector Substances 0.000 claims description 81
- 238000000034 method Methods 0.000 claims description 52
- 239000011159 matrix material Substances 0.000 claims description 37
- 230000006378 damage Effects 0.000 claims description 10
- 230000032683 aging Effects 0.000 claims description 9
- 238000009434 installation Methods 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 6
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- SLXKOJJOQWFEFD-UHFFFAOYSA-N 6-aminohexanoic acid Chemical compound NCCCCCC(O)=O SLXKOJJOQWFEFD-UHFFFAOYSA-N 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract description 16
- 238000001514 detection method Methods 0.000 abstract description 13
- 238000005516 engineering process Methods 0.000 description 16
- 238000007689 inspection Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 7
- 238000003745 diagnosis Methods 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000011161 development Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 4
- 239000004973 liquid crystal related substance Substances 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 201000005569 Gout Diseases 0.000 description 2
- 229910009447 Y1-Yn Inorganic materials 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 102000010637 Aquaporins Human genes 0.000 description 1
- 238000006424 Flood reaction Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000005086 pumping Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- General Health & Medical Sciences (AREA)
- Tourism & Hospitality (AREA)
- Evolutionary Computation (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Entrepreneurship & Innovation (AREA)
- Computing Systems (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- Primary Health Care (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The utility model relates to an intelligent water conservancy monitoring system based on machine vision acquires the multi-angle image of each water conservancy facility through multi-view image acquisition device, uses target water conservancy facility locking device to carry out data analysis to the operation data of target water conservancy system to judge the water conservancy facility that probably breaks down as target water conservancy facility. And then, using an image extraction device to screen out a part containing the target water conservancy facilities from the multi-angle image as a judging image, using a fault judging device to obtain a judging result by taking the operation parameters related to the judging image and the target water conservancy facilities as influence factors based on a preset fault judging model, and representing the water conservancy facilities and fault types of the target water conservancy system with faults. The system can efficiently monitor water conservancy facilities, improves fault detection accuracy, reduces operation and maintenance cost, realizes intelligent management, and has important practical significance and application value.
Description
Technical Field
The present disclosure relates to, but is not limited to, the field of water conservancy technology, and in particular to an intelligent water conservancy monitoring system based on machine vision.
Background
In the field of water conservancy, it is very important to detect and maintain the reliability and safety of water conservancy facilities. In the past, manual inspection and maintenance are generally required for detecting and maintaining water conservancy facilities, and the method is time-consuming, labor-consuming, low in efficiency and has a certain potential safety hazard. With the development of artificial intelligence and machine vision technologies, intelligent water conservancy detection systems based on machine vision are gradually widely used.
Currently, several technologies related to the fields of machine vision and water conservancy have been applied. While the prior art has provided effective solutions for the detection and maintenance of water conservancy facilities, there are still some problems. First, the situation that the prior art needs manual intervention still exists, for example, when detecting and maintaining water conservancy facilities, some complicated operations still need to be performed manually. Second, the detection accuracy and efficiency of the prior art still have room for improvement. While current machine vision and artificial intelligence techniques have enabled automatic inspection and fault diagnosis of water conservancy facilities, there are still some uncertainties and errors. Thirdly, the prior art has certain potential safety hazards when detecting and maintaining water conservancy facilities. For example, when inspecting a high water pressure pipeline, the equipment is easily damaged due to high pressure, and even safety accidents can occur.
Therefore, there is a need for a more efficient, accurate and safe intelligent water conservancy detection system based on machine vision to meet the overall demands of the water conservancy field. This new technology needs to overcome the drawbacks and problems of the prior art, and also needs to address new needs and challenges of future water conservancy facilities.
In recent years, with the development of deep learning and artificial intelligence technology, the detection accuracy and efficiency of the machine vision technology are further improved. By utilizing the novel technologies, the automatic inspection and fault diagnosis of the water conservancy facilities can be realized, the accuracy and the efficiency of detection are improved, and the requirement of manual intervention is reduced. For example, the deep learning algorithm may identify and classify different features in the image and automatically determine damage and aging of the hydraulic facility. And the predictive maintenance algorithm can predict the service life and maintenance time of the hydraulic facility according to data analysis and processing, and discover and process potential problems in advance.
In addition, in order to cope with the scenes such as high water pressure pipelines and the like which need high safety guarantee, automatic inspection and maintenance can be performed by utilizing autonomous navigation equipment such as robots and the like. The autonomous navigation equipment can automatically complete the inspection and maintenance of the water conservancy facilities according to the preset path and task, and the safety risk of personnel is greatly reduced.
In summary, with the continuous development of machine vision and artificial intelligence technology, the intelligent water conservancy detection system based on machine vision is expected to realize automatic inspection and fault diagnosis, improve the accuracy and efficiency of detection, and reduce the requirement of manual intervention. In the future, the technology can also be combined with technologies such as autonomous navigation equipment and the like to realize the detection and maintenance of the water conservancy facilities in all directions, high efficiency and high safety.
Disclosure of Invention
The intelligent water conservancy monitoring system based on machine vision has the capabilities of automatic inspection and fault diagnosis, can detect and diagnose problems possibly occurring in water conservancy facilities, and improves the reliability and safety of the water conservancy facilities. The system utilizes a machine vision technology and an artificial intelligence algorithm to realize efficient and automatic detection and analysis of water conservancy facilities, and provides an efficient and reliable detection solution for the water conservancy industry.
The technical scheme of the invention is realized as follows:
intelligent water conservancy monitoring system based on machine vision includes: the multi-view image acquisition device is configured to acquire multi-angle images of all water conservancy facilities aiming at all the water conservancy facilities of the target water conservancy system; the target hydraulic facility locking device is configured to acquire the operation data of the target hydraulic system, perform data analysis on the operation data, and judge the hydraulic facility which possibly fails in the target hydraulic system as the target hydraulic facility; an image extraction device configured to screen out a portion including the target water conservancy facility from the multi-angle image as a determination image based on the obtained target water conservancy facility; the fault judging device is configured to obtain a judging result by using a preset fault judging model according to the judging image, taking the judging image as a model input and taking the operation parameters related to the target water conservancy facilities as influence factors, wherein the judging result represents the water conservancy facilities with faults of the target water conservancy system and the types of faults of the target water conservancy system.
Further, the types of faults include: aging, damage and foreign body effects.
Further, the hydraulic facility in the target hydraulic system includes: water pipes, water pumps, water towers, sluice gates and reservoir dams.
Further, the multi-view image acquisition device includes a plurality of cameras installed at different positions, and the positions where the cameras are installed are calculated according to the positions of all hydraulic facilities in the target hydraulic system through a coordinate calculation model, and specifically includes: the method comprises the steps of taking the center of a target water conservancy system as an origin, establishing a three-dimensional space coordinate system, obtaining position coordinates of various water conservancy facilities in the target water conservancy system, and representing the position coordinates through a matrix: p= [ P1, P2, ], pn ]; wherein P is a coordinate matrix, pn is the coordinates of the water conservancy facilities, and n is the number of the water conservancy facilities; based on basic parameters of cameras, calculating an optimal installation position of each camera by using a preset camera optimal position determining algorithm and combining a coordinate matrix; wherein the basic parameters of different cameras are different from each other; the basic parameters include: camera field of view range θ, camera resolution M, and camera focal length K.
Further, the executing process of the camera optimal position determining algorithm includes: assuming that the coordinate of the ith camera is ci= (Xi, yi, zi), the angle of view is θi, the resolution is Mi, and the focal length is Ki, the range that can be observed by the camera is represented as a view cone, and the shape and the size of the view cone are related to the basic parameters of the camera; assuming that the coordinates of the jth hydraulic facility are sj= (Xj, yj, zj), the condition of the hydraulic facility in the field of view of the ith camera is expressed as:
(Xj-Xi)^2+(Yj-Yi)^2＜＝(Zj-Zi)*tan(θi/2)^2；
wherein tan (θi/2) is half of the field of view; according to the above conditions, the following constraints are constructed:
(Xj-Xi)^2+(Yj-Yi)^2＜＝(Zj-Zi)*tan(θi/2)^2，j＝1，2，...，n；
wherein n is the number of water conservancy facilities; converting the constraint conditions into a matrix form to obtain the following components: a C < = b; wherein A is an n×4 matrix, C is a 4×1 matrix, b is an n×1 matrix, and the specific form is:
A＝[X1-X2，Y1-Y2，Z1-Z2-K2*tan(θ2/2)；...；X1-Xn，Y1-Yn，Z1-Zn-Kn*tan(θn/2)]；
C＝[X1；Y1；Z1；1]；
b＝[-X1*X2-Y1*Y2+Z1*Z2+K2*tan(θ2/2)；...；-X1*Xn-Y1*Yn+Z1*Zn+Kn*tan(θn/2)]；
wherein the symbol "x" represents a matrix multiplication; and solving the constraint conditions by using a linear programming method, so as to obtain the optimal installation position of each camera.
Further, the target water conservancy facility locking device comprises: the operation data acquisition unit is configured to acquire various operation data in the target water conservancy system through the sensor; the operation data at least comprises: water flow speed, water level, water pressure, water quality, water pump current, water pump voltage, water pump temperature, water pump vibration, sluice current, sluice voltage, sluice temperature and sluice vibration; and the locking unit is configured to perform data analysis on the operation data, and judge the hydraulic facilities possibly having faults in the target hydraulic system as the target hydraulic facilities.
Further, the locking unit performs data analysis on the operation data, and determines a hydraulic facility with a possibility of failure in the target hydraulic system, and the method for using the hydraulic facility as the target hydraulic facility includes: assuming that the operational data of the target water conservancy facility is x= [ X1, X2, ], xm where m is the data dimension; firstly, processing data by using a feature selection technology or a feature extraction technology to obtain feature vectors Z= [ Z1, Z2, ], wherein l is the feature quantity; then, constructing a group of feature sets S containing different feature subsets according to the feature vector Z; specifically, the feature set S is recursively constructed by: step 1: initializing a feature set S as an empty set; step 2: for each feature zi in the feature vector Z, adding it to the feature set S; step 3: for each subset Si in the feature set S, adding features which are not in Si in the feature vector Z into Si to obtain a new subset Si+1; step 4: repeating step 3 until si+1 contains all features; after obtaining the feature set S, constructing a fault judging model fi (Zi) for each feature subset Si epsilon S, wherein Zi is a feature vector corresponding to the feature subset Si; for each water conservancy facility in the target water conservancy system, calculating a corresponding feature vector Zi, and taking the feature vector Zi as the input of a model fi (Zi) to obtain a fault judgment result yi, wherein yi=1 indicates that the facility has a fault, and yi=0 indicates that the facility operates normally; and finally, combining all the fault judgment results to obtain a fault judgment result Y= [ Y1, Y2, ], wherein yi represents the fault judgment result of the ith water conservancy facility.
Further, the image extraction device screens out a part containing the target hydraulic facility from the multi-angle image based on the obtained target hydraulic facility, and the method for judging the image comprises the following steps: for the images of each angle, carrying out feature extraction by using a SIFT algorithm to obtain a series of feature points and corresponding feature descriptors of each image; matching feature points in images of different angles to determine their relative positions; estimating the position and the direction of the target water conservancy facilities in each image according to the matching result of the characteristic points; cutting each image according to the position and direction information of the target water conservancy facilities, and extracting a local image of the target water conservancy facilities; and (5) enhancing the brightness, contrast and color of the cut local image of the target water conservancy facility.
Further, the fault determination device uses a preset fault determination model according to the determination image, takes the determination image as a model input, takes the operation parameters related to the target water conservancy facilities as influence factors, and the method for obtaining the determination result comprises the following steps: assuming n hydraulic facilities, the fault state of each facility is represented as a k-dimensional vector yi= (yi, 1, y) i，2 ，...，y i，k ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein y is i，j Indicating whether the ith facility has failed; if y i，j =1 indicates that the ith facility has failed j, otherwise y i，j =0; m operating parameters are arranged, and the value range of each operating parameter is [0,1 ]]They are represented as an m X n matrix x= [ X ] 1 ，x2，...，xn]Where xi= (x 1, i, x2, i,) xm, i represents m of the ith facilityThe value of the operation parameter; set I i Multi-angle image representing the ith facility, I i Is a three-dimensional tensor, expressed as an image of size h×w×c, where h represents the image height, w represents the image width, and c represents the number of channels; the input of the failure determination model includes a multi-angle image I i And m operating parameters x 1 ，x 2 ，...，x m The output of the failure determination model is two n×k matrices y= [ Y 1 ，y 2 ，...，y n ]And t= [ T ] 1 ，t 2 ，...，tn]Wherein Y represents a fault status of each facility, and T represents a type of fault of each facility; let F= [ F 1 ，f 2 ，...，f n ]Representing a feature vector for each facility, where f i A feature vector representing an ith facility; the failure determination model may be expressed as: f=cnn (I i ；θ c ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein CNN represents convolutional neural network, θ c Parameters representing the network; the output of the fault determination model is an n x d feature matrix F, where d is the dimension of the feature vector; the fault determination model uses a fully connected neural network to combine the operating parameters X and the feature matrix F to generate a fault state vector y for each facility i And fault type vector t i The method comprises the steps of carrying out a first treatment on the surface of the Let z= [ Z ] 1 ，z 2 ，...，z n ]Representing an intermediate vector for each facility, where z i Is a d+m dimension vector, which represents the combination of the feature vector and the operation parameter vector of the ith facility, and can be obtained: z= [ concat (f) 1 ，x 1 )，concat(f 2 ，x 2 )，...，concat(f n ，x n )]The method comprises the steps of carrying out a first treatment on the surface of the Where concat represents the join operation of the vector, z i ＝concat(f i ，x i ) The characteristic vector and the operation parameter vector are sequentially connected to obtain a d+m-dimensional vector; the fully connected layer of the model can be expressed as: y is i ，t i ＝FCN(Z；θ f ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein θ is f Representing parameters of the fully connected layer, y i Is a 1 xk vector representing the fault state vector of the ith facility, t i Is a 1 x 3 vector representing the fault type vector of the ith facility, whereThe 1 st element represents the probability of an aged failure, the 2 nd element represents the probability of a damaged failure, and the 3 rd element represents the probability of a foreign matter affecting the failure.
Further, the system also comprises an early warning unit, which is configured to send out an alarm to the far end according to the judging result.
The intelligent water conservancy monitoring system based on machine vision has the following beneficial effects: first, the system is capable of efficiently monitoring water conservancy facilities. Traditional water conservancy facility monitoring mode generally needs artifical inspection, and complex operation is inefficiency, and monitoring result receives the human factor influence easily moreover, has certain misjudgement rate. In contrast, the system adopts the multi-view image acquisition device and the target hydraulic facility locking device, can automatically acquire multi-angle images and operation data of a plurality of hydraulic facilities, and screens out the part containing the target hydraulic facilities from the multi-angle images as a judgment image through the image extraction device, so that the monitoring efficiency and accuracy are greatly improved. Secondly, the system can improve fault detection accuracy. By means of the fault judging device, the system utilizes a preset fault judging model, operation parameters related to the judging image and the target water conservancy facilities are used as influence factors, judging results are obtained, and the water conservancy facilities and fault types of the target water conservancy systems are represented. Compared with the traditional water conservancy facility monitoring mode, the system not only can find the faults of the water conservancy facility, but also can accurately judge the types of the faults, provides more accurate fault analysis results, is beneficial to rapidly positioning and solving the faults, and reduces the loss and influence caused by the faults. In addition, the system can also reduce operation and maintenance cost. Traditional water conservancy facility monitoring mode needs a large amount of manpower and material resources to throw into, including patrolling and examining personnel, equipment maintenance personnel, monitoring facilities etc.. The system adopts a machine vision technology to automatically acquire multi-angle images and operation data of a plurality of hydraulic facilities, and utilizes a preset fault judgment model to perform fault analysis, so that additional manpower and material resources are not required, and the operation and maintenance cost is saved. Finally, the system can realize intelligent management. Through the combination of a plurality of devices such as multi-view image acquisition device, target water conservancy facility locking device, image extraction device and trouble judgement device, this system can realize the automatic monitoring and the trouble analysis to water conservancy facility, has improved the intelligent degree of monitoring and maintenance, helps promoting management efficiency and level. To sum up, the intelligent water conservancy monitoring system based on machine vision that this patent provided has a plurality of beneficial effects such as high-efficient, accurate, cost-saving and intelligent management, has important practical meaning and using value to the monitoring and the maintenance of water conservancy facilities. The system can be widely applied to monitoring and maintenance of various water conservancy facilities, such as dams, reservoirs, sluice gates, channels, water pumping stations and the like. Through monitoring and maintaining the hydraulic facilities, the safe operation of the hydraulic engineering can be better ensured, and reliable water resource guarantee is provided for economic development and people living.
Drawings
Fig. 1 is a schematic system structure diagram of an intelligent water conservancy monitoring system based on machine vision according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present disclosure more clear and obvious, the present disclosure is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure.
Example 1
Referring to fig. 1, an intelligent water conservancy monitoring system based on machine vision comprises four devices: the system comprises a multi-view image acquisition device, a target water conservancy facility locking device, an image extraction device and a fault determination device. The multi-view image acquisition device is configured to acquire a multi-angle image of each hydraulic facility for each hydraulic facility of the target hydraulic system. This device uses multiple cameras or sensors to acquire images of multiple angles of hydraulic facilities for more accurate analysis and judgment of the facilities.
The target hydraulic facility locking device is configured to acquire operation data of the target hydraulic system and perform data analysis on the data to judge a hydraulic facility possibly having a fault in the target hydraulic system as the target hydraulic facility. The device can discover abnormal conditions by monitoring the operation data of the water conservancy system in real time, and marks and tracks the abnormal conditions as a target water conservancy facility.
The image extraction device is configured to screen out a portion including the target hydraulic facility from the multi-angle image as a determination image based on the obtained target hydraulic facility. The device can accurately position the position and the size of the target facility in the image according to the position information of the target water conservancy facility and the characteristics of the multi-angle image, and extract the part containing the target facility.
The fault judging device is configured to obtain a judging result by using a preset fault judging model according to the judging image, taking the judging image as a model input and taking the operation parameters related to the target water conservancy facilities as influence factors, wherein the judging result represents the water conservancy facilities with faults of the target water conservancy system and the types of faults of the target water conservancy system. The device can judge the state of the target water conservancy facilities and the type of faults through the preset fault judging model and combining the information such as images, operation data and the like, so that measures can be taken in time to maintain and repair.
Example 2
On the basis of the above embodiment, the types of faults include: aging, damage and foreign body effects.
Wherein, the aging is usually due to long-time use of water conservancy facilities, and the aging of materials leads to the degradation of facility functions or the failure of normal operation; damage is typically due to accidents, harsh environments, etc. that cause damage or destruction of hydraulic facilities; foreign matter effects are often due to blockage or jamming of some foreign matter that results in the water conservancy facilities not functioning properly. These fault types all have different degrees of impact on the operation and stability of the hydraulic system, and therefore it is important to identify and classify them.
Example 3
On the basis of the above embodiment, the hydraulic facility in the target hydraulic system includes: water pipes, water pumps, water towers, sluice gates and reservoir dams.
These facilities play different roles in water conservancy systems, for example, water pipes and water pumps are important facilities for water transportation and pressurization, water storage towers are used for storing water resources, floodgates are used for controlling water flow and water level, and reservoir dams are used for regulating river water level and preventing floods and the like. Therefore, monitoring and maintaining the operational status of these hydraulic facilities is critical to the proper operation of the hydraulic system.
Specifically, the system can acquire multi-angle images of each water conservancy facility, including information of the inside and outside of a water pipe, blades and mechanical parts of a water pump, containers and water levels of a water storage tower, the form and structure of a sluice and a reservoir dam, and the like, through the multi-angle image acquisition device. Through the target hydraulic facility locking device, the system can conduct data analysis and fault identification on the hydraulic facilities, and the hydraulic facilities with possible faults serve as the target hydraulic facilities.
By means of the image extraction device and the fault determination device, the system can determine and classify faults of the target water conservancy facilities, including aging, damage, foreign matter influence and the like. Therefore, the system can be applied to various water conservancy systems with different types, improves the monitoring and predicting capability of water conservancy facilities, effectively prevents and solves the problem of faults, and improves the reliability and stability of the water conservancy systems.
For these different hydraulic facilities, the intelligent hydraulic monitoring system can adopt different monitoring means and fault diagnosis methods. For example, for water pipes and water pumps, monitoring devices such as a flowmeter, a pressure sensor, a vibration sensor and the like can be used for monitoring the running state of the water pipes and the water pumps in real time, and judging whether the water pipes and the water pumps have fault types such as aging, damage, foreign matter influence and the like or not through a fault diagnosis algorithm. For the water storage tower, the water storage capacity can be monitored through a water level sensor, and whether faults such as water source leakage or water leakage exist or not can be judged through water level change. For sluice and reservoir dam, can use devices such as fluviograph, inclination sensor and seismic sensor to monitor its deformation and displacement condition to combine image analysis technique to judge whether there is fault such as crack, landslide and dam body destruction.
Example 4
On the basis of the above embodiment, the multi-view image acquisition device includes a plurality of cameras installed at different positions, and the positions where the cameras are installed are calculated according to the positions of all hydraulic facilities in the target hydraulic system through a coordinate calculation model, and specifically includes: the method comprises the steps of taking the center of a target water conservancy system as an origin, establishing a three-dimensional space coordinate system, obtaining position coordinates of various water conservancy facilities in the target water conservancy system, and representing the position coordinates through a matrix: p= [ P1, P2, ], pn ]; wherein P is a coordinate matrix, pn is the coordinates of the water conservancy facilities, and n is the number of the water conservancy facilities; based on basic parameters of cameras, calculating an optimal installation position of each camera by using a preset camera optimal position determining algorithm and combining a coordinate matrix; wherein the basic parameters of different cameras are different from each other; the basic parameters include: camera field of view range θ, camera resolution M, and camera focal length K.
When there are a plurality of hydraulic facilities in the hydraulic system, in order to be able to monitor the running situation of every hydraulic facility comprehensively, this intelligent hydraulic monitoring system based on machine vision uses a plurality of cameras to acquire the image of a plurality of perspectives to improve the monitoring capacity and the accuracy of system.
The positions of the cameras are calculated through a coordinate calculation model according to the positions of all hydraulic facilities in the target hydraulic system. Specifically, by establishing a three-dimensional space coordinate system, the position coordinates of each hydraulic facility in the coordinate system can be obtained by taking the center of the target hydraulic system as the origin, and the position coordinates are represented by a matrix.
Then, the system calculates the optimal installation position of each camera by combining a coordinate matrix based on basic parameters of the camera, including the field of view range, resolution, focal length and the like of the camera and using a preset optimal position determination algorithm of the camera. Therefore, each camera can shoot the water conservancy facilities needing to be monitored, and can cover comprehensively and carry out monitoring and data acquisition without blind areas.
Meanwhile, the basic parameters of different cameras are different from each other. This is because different water conservancy facilities may have different characteristics and requirements, for example, the field of view range needs to be determined according to the size and position of the device, the resolution needs to be determined according to the monitoring requirements and the data processing capability, and the focal length needs to be calculated according to the shooting distance and the accuracy requirements. Therefore, basic parameters of different cameras in the system need to be adjusted according to actual conditions so as to adapt to the characteristics and monitoring requirements of different water conservancy facilities.
In a word, through this kind of camera mounting means based on coordinate calculation model, this intelligent water conservancy monitoring system can improve the comprehensiveness and the accuracy of monitoring, ensures that every water conservancy facility can all be monitored and data acquisition to prevention and solution potential trouble problem.
Example 5
On the basis of the above embodiment, the executing process of the camera optimal position determining algorithm includes: assuming that the coordinate of the ith camera is ci= (Xi, yi, zi), the angle of view is θi, the resolution is Mi, and the focal length is Ki, the range that can be observed by the camera is represented as a view cone, and the shape and the size of the view cone are related to the basic parameters of the camera; assuming that the coordinates of the jth hydraulic facility are sj= (Xj, yj, zj), the condition of the hydraulic facility in the field of view of the ith camera is expressed as:
(Xj-Xi)^2+(Yj-Yi)^2＜＝(Zj-Zi)*tan(θi/2)^2；
wherein tan (θi/2) is half of the field of view; according to the above conditions, the following constraints are constructed:
(Xj-Xi)^2+(Yj-Yi)^2＜＝(Zj-Zi)*tan(θi/2)^2，j＝1，2，...，n；
wherein n is the number of water conservancy facilities; converting the constraint conditions into a matrix form to obtain the following components: a C < = b; wherein A is an n×4 matrix, C is a 4×1 matrix, b is an n×1 matrix, and the specific form is:
A＝[X1-X2，Y1-Y2，Z1-Z2-K2*tan(θ2/2)；...；X1-Xn，Y1-Yn，Z1-Zn-Kn*tan(θn/2)]；
C＝[X1；Y1；Z1；1]；
b＝[-X1*X2-Y1*Y2+Z1*Z2+K2*tan(θ2/2)；...；-X1*Xn-Y1*Yn+Z1*Zn+Kn*tan(θn/2)]；
Wherein the symbol "x" represents a matrix multiplication; and solving the constraint conditions by using a linear programming method, so as to obtain the optimal installation position of each camera.
The basic parameters of different cameras are different from each other, because different water conservancy facilities may have different characteristics and requirements, for example, the field of view range needs to be determined according to the size and the position of equipment, the resolution needs to be determined according to the size and the distance of the water conservancy facilities to be monitored, the focal length needs to be determined according to the field of view range and the monitoring distance, and the like. Therefore, when the algorithm is used, basic parameters are required to be set and adjusted according to actual conditions so as to adapt to the characteristics and monitoring requirements of different water conservancy facilities, and the monitoring capability and accuracy of the system are improved.
In addition to the basic parameters described above, other factors need to be considered, such as the view shielding of the camera and the light conditions. To solve these problems, adjustments and optimizations may be made in connection with the actual situation, such as increasing the number of cameras, adjusting the angles and positions of the cameras, etc. In addition to the basic parameters described above, other factors need to be considered, such as the view shielding of the camera and the light conditions. To solve these problems, adjustments and optimizations may be made in connection with the actual situation, such as increasing the number of cameras, adjusting the angles and positions of the cameras, etc.
When the constraint a×c < =b is constructed, a linear programming method can be used to solve the optimal mounting position of the camera. The method comprises the following specific steps:
defining an objective function
An objective function, i.e. a function to be minimized or maximized, is defined. In this problem, our goal is to have the camera's field of view cover as many hydropower facilities as possible, so the objective function can be defined as: minimize (x), where x is a binary vector indicating whether each hydropower plant is covered by a camera, and sum (x) indicates the sum of the number of all covered hydropower plants.
Determining constraints
According to the constraint a x C < = b, we can convert it into a standard linear programming form, i.e. Ax < = b, where x is a decision variable representing the optimal mounting position of each camera, a is a constraint matrix, and b is a constraint vector.
Solving the linear programming problem
And solving the linear programming problem by using a linear programming solver to obtain the optimal installation position of each camera.
Analysis of the solution results
And analyzing the solving result, checking whether the optimal mounting position of each camera is reasonable, and further optimizing the layout and the visual field range of the cameras.
Example 6
On the basis of the above embodiment, the target water conservancy facility locking device comprises: the operation data acquisition unit is configured to acquire various operation data in the target water conservancy system through the sensor; the operation data at least comprises: water flow speed, water level, water pressure, water quality, water pump current, water pump voltage, water pump temperature, water pump vibration, sluice current, sluice voltage, sluice temperature and sluice vibration; and the locking unit is configured to perform data analysis on the operation data, and judge the hydraulic facilities possibly having faults in the target hydraulic system as the target hydraulic facilities.
Example 7
On the basis of the above embodiment, the locking unit performs data analysis on the operation data, and determines a hydraulic facility in the target hydraulic system, where the hydraulic facility may have a fault, and the method for serving as the target hydraulic facility includes: assuming that the operational data of the target water conservancy facility is x= [ X1, X2, ], xm where m is the data dimension; firstly, processing data by using a feature selection technology or a feature extraction technology to obtain feature vectors Z= [ Z1, Z2, ], wherein l is the feature quantity; then, constructing a group of feature sets S containing different feature subsets according to the feature vector Z; specifically, the feature set S is recursively constructed by: step 1: initializing a feature set S as an empty set; step 2: for each feature zi in the feature vector Z, adding it to the feature set S; step 3: for each subset Si in the feature set S, adding features which are not in Si in the feature vector Z into Si to obtain a new subset Si+1; step 4: repeating step 3 until si+1 contains all features; after obtaining the feature set S, constructing a fault judging model fi (Zi) for each feature subset Si epsilon S, wherein Zi is a feature vector corresponding to the feature subset Si; for each water conservancy facility in the target water conservancy system, calculating a corresponding feature vector Zi, and taking the feature vector Zi as the input of a model fi (Zi) to obtain a fault judgment result yi, wherein yi=1 indicates that the facility has a fault, and yi=0 indicates that the facility operates normally; and finally, combining all the fault judgment results to obtain a fault judgment result Y= [ Y1, Y2, ], wherein yi represents the fault judgment result of the ith water conservancy facility.
Feature selection techniques may select the most representative subset of features by evaluating the importance and relevance of the features. Common feature selection methods include variance selection, chi-square inspection, mutual information, model-based feature selection, and the like. In the water conservancy monitoring system, a proper feature selection method and an index can be selected according to field knowledge and actual requirements so as to obtain the most representative feature vector. The feature extraction technique can convert the original data into new feature vectors through mathematical transformation to improve the distinguishing and generalization capabilities of the features. Common feature extraction methods include principal component analysis, linear discriminant analysis, wavelet transformation, local binary pattern, image processing, and the like. In the water conservancy monitoring system, a proper feature extraction method and algorithm can be selected according to the actual data type and feature requirements so as to obtain the most distinguishable feature vector. Through processing of feature selection or feature extraction techniques, a feature vector z= [ Z1, Z2, ], where l is the feature quantity, can be obtained. The feature vector is an abstract representation of the original operation data, has higher differentiation and generalization capability, and can be used as input data of a fault determination model.
Example 8
On the basis of the above embodiment, the method for selecting, as the determination image, the portion including the target water conservancy facility from the multi-angle image based on the obtained target water conservancy facility includes: for the images of each angle, carrying out feature extraction by using a SIFT algorithm to obtain a series of feature points and corresponding feature descriptors of each image; matching feature points in images of different angles to determine their relative positions; estimating the position and the direction of the target water conservancy facilities in each image according to the matching result of the characteristic points; cutting each image according to the position and direction information of the target water conservancy facilities, and extracting a local image of the target water conservancy facilities; and (5) enhancing the brightness, contrast and color of the cut local image of the target water conservancy facility.
Assuming that an image at a certain angle is I (x, y), carrying out feature extraction on the image by using a SIFT algorithm to obtain a series of feature points and corresponding feature descriptors:
wherein, (x) i ，y i ) Representing the position of the feature point s i Representing the scale, θ i Indicates the direction, d i Representing the feature descriptors. N represents the number of feature points.
For images at different angles, the feature points are matched using the RANSAC algorithm to determine their relative positions:
wherein, the liquid crystal display device comprises a liquid crystal display device,and->Respectively representing the position coordinates of the feature points in the matching pairs in the images of different angles.
Estimating the position and the direction of the target water conservancy facilities in each image according to the matching result of the characteristic points:
wherein, the liquid crystal display device comprises a liquid crystal display device,the position and the direction of the target hydraulic facility in the image of the ith angle are represented, and K represents the internal reference matrix of the camera.
Cutting each image according to the position and direction information of the target hydraulic facility, and extracting a local image of the target hydraulic facility:
wherein (x ', y') represents the position of the target water conservancy facility in the image of the ith angle, I i Representing the original image of the i-th angle.
And (3) enhancing the brightness, contrast and color of the cut local image of the target water conservancy facility:
where f represents the enhancement function.
And then, according to the extracted partial image of the target water conservancy facility, the brightness, contrast and color are enhanced, so that the accuracy of fault judgment is improved. The method comprises the following specific steps:
brightness enhancement: for each extracted partial image, the average luminance Lmean and standard deviation Lstd thereof are calculated, and the luminance adjustment is performed on the image according to the following formula:
Lout＝(Lin-Lmean)*(Lmax/3*Lstd)+Lmean
Wherein Lin and Lout respectively represent the input and output luminance values, and Lmax is the maximum luminance value. The formula achieves the effect of enhancing the brightness of the image by mapping the brightness value of the image pixel to a certain range.
Contrast enhancement: the average gray value Gmean and the standard deviation Gstd of the pixel value of the partial image are calculated, and then the contrast adjustment is performed on the image according to the following formula:
Gout＝(Gin-Gmean)*(Gmax/3*Gstd)+Gmean
wherein Gin and Gout represent input and output pixel values, respectively, and Gmax is a maximum pixel value. This formula increases the contrast of the image by mapping the gray values of the image pixels into a range.
Color enhancement: for color images, the color of the image may be enhanced by enhancing the color saturation. Specifically, the RGB color space is converted into the HSV color space, the saturation S is adjusted, and then the image is converted back into the RGB color space. The specific formula for adjusting the saturation is:
Sout＝(Sin-Smean)*(Smax/3*Sstd)+Smean
wherein Sin and Sout represent the saturation values of the input and output, respectively, smean and Sstd are the average and standard deviation of the local image saturation,is the maximum saturation value. The formula improves the saturation of the image by mapping the saturation to a range.
Through the brightness, contrast and color enhancement processing, a local image with better fault determination capability can be obtained, so that the accuracy of the whole intelligent water conservancy monitoring system is improved.
Example 9
On the basis of the above embodiment, the fault determining device uses a preset fault determining model according to the determining image, inputs the determining image as the model, and uses the operation parameters related to the target water conservancy facilities as the influencing factors, and the method for obtaining the determining result includes: assuming n hydraulic facilities, the fault state of each facility is represented as a k-dimensional vector yi= (yi, 1, y) i，2 ，...，y i，k ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein y is i，j Represent the firsti whether a j-th fault has occurred at the facility; if y i，j =1 indicates that the ith facility has failed j, otherwise y i，j =0; m operating parameters are arranged, and the value range of each operating parameter is [0,1 ]]They are represented as an m X n matrix x= [ X ] 1 ，x2，...，xn]Where xi= (x 1, i, x2, i,) xm, i represents the value of the m operating parameters of the i-th facility; set I i Multi-angle image representing the ith facility, I i Is a three-dimensional tensor, expressed as an image of size h×w×c, where h represents the image height, w represents the image width, and c represents the number of channels; the input of the failure determination model includes a multi-angle image I i And m operating parameters x 1 ，x 2 ，...，x m The output of the failure determination model is two n×k matrices y= [ Y 1 ，y 2 ，...，y n ]And t= [ T ] 1 ，t 2 ，...，t n ]Wherein Y represents a fault status of each facility, and T represents a type of fault of each facility; let F= [ F 1 ，f 2 ，...，f n ]Representing a feature vector for each facility, where f i A feature vector representing an ith facility; the failure determination model may be expressed as: f=cnn (I i ；θ c ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein CNN represents convolutional neural network, θ c Parameters representing the network; the output of the fault determination model is an n x d feature matrix F, where d is the dimension of the feature vector; the fault determination model uses a fully connected neural network to combine the operating parameters X and the feature matrix F to generate a fault state vector y for each facility i And fault type vector t i The method comprises the steps of carrying out a first treatment on the surface of the Let z= [ Z ] 1 ，z 2 ，...，z n ]Representing an intermediate vector for each facility, where z i Is a d+m dimension vector, which represents the combination of the feature vector and the operation parameter vector of the ith facility, and can be obtained: z= [ concat (f) 1 ，x 1 )，concat(f 2 ，x 2 )，...，concat(f n ，x n )]The method comprises the steps of carrying out a first treatment on the surface of the Where concat represents the join operation of the vector, z i ＝concat(f i ，x i ) Representing the order of feature vectors and operating parameter vectorsConnecting to obtain a d+m-dimensional vector; the fully connected layer of the model can be expressed as: y is i ，t i ＝FCN(Z；θ f ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein θ is f Representing parameters of the fully connected layer, y i Is a 1 xk vector representing the fault state vector of the ith facility, t i Is a 1×3 vector representing a fault type vector of the i-th facility, wherein the 1 st element represents a probability of an aged fault, the 2 nd element represents a probability of a damaged fault, and the 3 rd element represents a probability of a foreign matter affecting the fault.
Example 10
On the basis of the previous embodiment, the system further comprises an early warning unit configured to send out an alarm to the far end according to the judging result.
And alarming so that the hydraulic facility can take measures in time to avoid or reduce the loss caused by the failure of the hydraulic facility. The specific implementation process is as follows:
firstly, the early warning unit analyzes the fault judging result, and determines the level and the content of the alarm according to different types of faults and severity. For example, for some severe faults, an alarm may need to be raised immediately, while containing detailed fault information and handling advice; for some minor faults, the alarm level may be reduced and the alarm content simplified.
Secondly, the early warning unit can send alarm information to the remote equipment, so that water conservancy management personnel or maintenance personnel can receive the alarm in time. In general, the alarm information is sent to related personnel through a network or a short message, so as to ensure that the related personnel can acquire the alarm information in time.
Finally, the early warning unit records the detailed information of each alarm so as to analyze and count in the future. Such information may include fault type, alarm level, time of occurrence, alarm content, etc., which may assist a water manager or maintainer in knowing the operational condition of the water utility and providing a reference for future maintenance work.
It should be noted that the apparatus (device) embodiments and the readable storage medium embodiments and the method embodiments described above belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments. The technical features in the method embodiment are applicable to the device embodiment correspondingly, and are not described herein.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof.
In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, and are not thereby limiting the scope of the claims of the present disclosure. Any modifications, equivalent substitutions and improvements made by those skilled in the art without departing from the scope and spirit of the present disclosure shall fall within the scope of the claims of the present disclosure.
Claims (7)
1. Intelligent water conservancy monitoring system based on machine vision, its characterized in that includes: the multi-view image acquisition device is configured to acquire multi-angle images of all water conservancy facilities aiming at all the water conservancy facilities of the target water conservancy system; the target water conservancy facility locking device is configured to acquire operation data of the target water conservancy system, perform data analysis on the operation data, and judge the water conservancy facility with faults in the target water conservancy system as the target water conservancy facility; an image extraction device configured to screen out a portion including the target water conservancy facility from the multi-angle image as a determination image based on the obtained target water conservancy facility; the fault judging device is configured to obtain a judging result by using a preset fault judging model according to the judging image, taking the judging image as a fault judging model input and taking the operation parameters related to the target water conservancy facilities as influence factors, wherein the judging result represents the water conservancy facilities with faults of the target water conservancy system and the types of faults of the target water conservancy system; the target water conservancy facility locking device comprises: the operation data acquisition unit is configured to acquire various operation data in the target water conservancy system through the sensor; the operation data at least comprises: water flow speed, water level, water pressure, water quality, water pump current, water pump voltage, water pump temperature, water pump vibration, sluice current, sluice voltage, sluice temperature and sluice vibration; the locking unit is configured to perform data analysis on the operation data, and judge the hydraulic facility with the fault in the target hydraulic system as the target hydraulic facility; the locking unit is used for carrying out data analysis on the operation data and judging the hydraulic facility with the fault in the target hydraulic system, and the method for serving as the target hydraulic facility comprises the following steps: let the operational data of the target water conservancy facility be x= [ X1, X2, ], xm where m is the data dimension; first, the data is processed by using a feature selection technique or a feature extraction technique to obtain feature vectors
Z＝[z1，z2，...，zl]Wherein l is the number of features; then, constructing a group of feature sets S containing different feature subsets according to the feature vector Z; specifically, the feature set S is recursively constructed by: step 1: initializing a feature set S as an empty set; step 2: for each feature zi in the feature vector Z, adding it to the feature set S; step 3: for each subset Si in the feature set S, adding features which are not in Si in the feature vector Z into Si to obtain a new subset Si+1; step 4: repeating step 3 until si+1 contains all features; after obtaining the feature set S, constructing a fault judging model fi (Zi) for each feature subset Si epsilon S, wherein Zi is a feature vector corresponding to the feature subset Si; for each water conservancy facility in the target water conservancy system, calculating a corresponding feature vector Zi, and taking the feature vector Zi as input of a fault judgment model fi (Zi) to obtain a fault judgment result yi, wherein yi=1 indicates that the facility has a fault, and yi=0 indicates that the facility operates normally; and finally, combining all fault judgment results to obtain fault judgment results Y= [ Y1, Y2, ], yn of all water conservancy facilities in the target water conservancy system ]Wherein yi represents the failure determination result of the ith hydraulic facility; the fault judging device uses a preset fault judging model according to a judging image, the judging image is used as a fault judging model to be input, and the operation parameters related to the target water conservancy facilities are used as influence factors, so that a judging result is obtained, and the method comprises the following steps: there are n hydraulic facilities, and the fault state of each facility is expressed as a k-dimensional vector yi= (y) i，1 ，y i，2 ，...，y i，k ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein y is i，j Indicating whether the ith facility has failed; if y i，j =1 indicates that the ith facility has failed j, otherwise y i，j =0; m operating parameters are arranged, and the value range of each operating parameter is [0,1 ]]They are represented as a matrix x= [ X1, X2, ], xn, of m X n]Where xi= (x 1, i, x2, i,) xm, i represents the value of the m operating parameters of the i-th facility; set I i Multi-angle image representing the ith facility, I i Is a three-dimensional tensor expressed as an image of size h x w x c, where h representsImage height, w represents image width, c represents channel number; the input of the failure determination model includes a multi-angle image I i And m operating parameters x 1 ，x 2 ，...，x m The output of the failure determination model is two n×k matrices y= [ Y 1 ，y 2 ，...，y n ]And t= [ T ] 1 ，t 2 ，...，t n ]Wherein Y represents a fault status of each facility, and T represents a type of fault of each facility; let F= [ F 1 ，f 2 ，...，f n ]Representing a feature vector for each facility, where f i A feature vector representing an ith facility; the failure determination model is expressed as:
F＝CNN(I i ；θ c ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein CNN represents convolutional neural network, θ c Parameters representing the network; the output of the fault determination model is an n x d feature matrix F, where d is the dimension of the feature vector; the fault determination model uses a fully connected neural network to combine the operating parameters X and the feature matrix F to generate a fault state vector y for each facility i And fault type vector t i The method comprises the steps of carrying out a first treatment on the surface of the Let z= [ Z ] 1 ，z 2 ，...，z n [ representing the intermediate vector of each facility, where z i Is a d+m dimension vector, which represents the combination of the feature vector and the operation parameter vector of the ith facility, and is obtained by:
Z＝[concat(f 1 ，x 1 )，concat(f 2 ，x 2 )，...，concat(f n ，x n )]the method comprises the steps of carrying out a first treatment on the surface of the Where concat represents the join operation of the vector, z i ＝concat(f i ，x i ) The characteristic vector and the operation parameter vector are sequentially connected to obtain a d+m-dimensional vector; the fully connected layer of the failure determination model is expressed as: y is i ，t i ＝FCN(Z；θ f ) The method comprises the steps of carrying out a first treatment on the surface of the Wherein θ is f Representing parameters of the fully connected layer, y i Is a 1 xk vector representing the fault state vector of the ith facility, t i Is a 1 x 3 vector representing the fault type vector of the ith facility, wherein the 1 st element represents the probability of an aging fault, the 2 nd element represents the probability of a failure fault, and the 3 rd element table Showing the probability that a foreign object affects a fault.
2. The system of claim 1, wherein the type of fault comprises: aging, damage and foreign body effects.
3. The system of claim 2, wherein the hydraulic facility in the target hydraulic system comprises: water pipes, water pumps, water towers, sluice gates and reservoir dams.
4. A system according to claim 3, wherein the multi-view image acquisition device comprises a plurality of cameras installed at different positions, and the positions where the cameras are installed are calculated according to the positions of various hydraulic facilities in the target hydraulic system through a coordinate calculation model, and specifically comprises: the method comprises the steps of taking the center of a target water conservancy system as an origin, establishing a three-dimensional space coordinate system, obtaining position coordinates of various water conservancy facilities in the target water conservancy system, and representing the position coordinates through a matrix:
p= [ P1, P2, ], pn ]; wherein P is a coordinate matrix, pn is the coordinates of the water conservancy facilities, and n is the number of the water conservancy facilities; based on basic parameters of cameras, calculating an optimal installation position of each camera by using a preset camera optimal position determining algorithm and combining a coordinate matrix; wherein the basic parameters of different cameras are different from each other; the basic parameters include: camera field of view range θ, camera resolution M, and camera focal length K.
5. The system of claim 4, wherein the camera best position determination algorithm is performed by: setting the coordinate of the ith camera as Ci= (Xi, yi, zi), the angle of view as thetai, the resolution as Mi and the focal length as Ki, the observable range of the camera is expressed as a view cone, and the shape and the size of the view cone are related to the basic parameters of the camera; let the coordinates of the j-th hydraulic facility be sj= (Xj, yj, zj), the condition of the hydraulic facility in the field of view of the i-th camera is expressed as:
(Xj-Xi)^2+(Yj-Yi)^2<＝(Zj-Zi)*tan(θi/2)^2；
wherein tan (θi/2) is half of the field of view; according to the above conditions, the following constraints are constructed:
(Xj-Xi)^2+(Yj-Yi)^2＜＝(Zj-Zi)*tan(θi/2)^2，j＝1，2，...，n；
wherein n is the number of water conservancy facilities; converting the constraint conditions into a matrix form to obtain: a C < = b; wherein A is an n×4 matrix, C is a 4×1 matrix, b is an n×matrix, and the specific form is:
A＝[X1-X2，Y1-Y2，z1-z2-K2*tan(θ2/2)；...；X1-Xn，Y1
Yn，Z1-Zn-Kn*tan(θn/2)]；
C＝[X1；Y1；Z1；1]；
b＝[-X1*X2-Y1*Y2+Z1*Z2+K2*tan(θ2/2)；...；-X1*Xn
Y1*Yn+Z1*Zn+Kn*tan(θn/2)]；
wherein the symbol "x" represents a matrix multiplication; and solving the constraint conditions by using a linear programming method, so as to obtain the optimal installation position of each camera.
6. The system according to claim 1, wherein the image extracting means screens out a portion including the target water conservancy facility from the multi-angle image based on the obtained target water conservancy facility, and the method for determining the image includes: for the images of each angle, carrying out feature extraction by using a SIFT algorithm to obtain a series of feature points and corresponding feature descriptors of each image; matching feature points in images of different angles to determine their relative positions; estimating the position and the direction of the target water conservancy facilities in each image according to the matching result of the characteristic points; cutting each image according to the position and direction information of the target water conservancy facilities, and extracting a local image of the target water conservancy facilities; and (5) enhancing the brightness, contrast and color of the cut local image of the target water conservancy facility.
7. The system of claim 6, further comprising an alert unit configured to alert the remote end based on the determination.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310370298.4A CN116310760B (en) | 2023-04-06 | 2023-04-06 | Intelligent water conservancy monitoring system based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310370298.4A CN116310760B (en) | 2023-04-06 | 2023-04-06 | Intelligent water conservancy monitoring system based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116310760A CN116310760A (en) | 2023-06-23 |
| CN116310760B true CN116310760B (en) | 2023-09-22 |
Family
ID=86820638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310370298.4A Active CN116310760B (en) | 2023-04-06 | 2023-04-06 | Intelligent water conservancy monitoring system based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116310760B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326783A (en) * | 2021-06-01 | 2021-08-31 | 浪潮软件科技有限公司 | Edge early warning method for water conservancy industry |
| CN114219687A (en) * | 2021-11-02 | 2022-03-22 | 三峡大学 | Intelligent identification method for potential construction safety hazards by fusing human-computer vision |
| CN115131325A (en) * | 2022-07-11 | 2022-09-30 | 国网河南省电力公司经济技术研究院 | Breaker fault operation and maintenance monitoring method and system based on image recognition and analysis |
| CN115409741A (en) * | 2022-11-01 | 2022-11-29 | 长江勘测规划设计研究有限责任公司 | Machine vision recognition algorithm for measuring sediment content by using river surface color difference |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111337768B (en) * | 2020-03-02 | 2021-01-19 | 武汉大学 | Deep parallel fault diagnosis method and system for dissolved gas in transformer oil |
-
2023
- 2023-04-06 CN CN202310370298.4A patent/CN116310760B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113326783A (en) * | 2021-06-01 | 2021-08-31 | 浪潮软件科技有限公司 | Edge early warning method for water conservancy industry |
| CN114219687A (en) * | 2021-11-02 | 2022-03-22 | 三峡大学 | Intelligent identification method for potential construction safety hazards by fusing human-computer vision |
| CN115131325A (en) * | 2022-07-11 | 2022-09-30 | 国网河南省电力公司经济技术研究院 | Breaker fault operation and maintenance monitoring method and system based on image recognition and analysis |
| CN115409741A (en) * | 2022-11-01 | 2022-11-29 | 长江勘测规划设计研究有限责任公司 | Machine vision recognition algorithm for measuring sediment content by using river surface color difference |
Non-Patent Citations (1)
| Title |
|---|
| "水利泵站电气设备故障自动排查研究";王善慈等;《自动化应用》(第4期);124-126 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116310760A (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Haurum et al. | A survey on image-based automation of CCTV and SSET sewer inspections | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN102673979B (en) | Method and device for judging deviation of conveying belt | |
| CN110992349A (en) | Underground pipeline abnormity automatic positioning and identification method based on deep learning | |
| CN113283344A (en) | Mining conveying belt deviation detection method based on semantic segmentation network | |
| CN111696320B (en) | Intelligent visual early warning system for early gas leakage | |
| CN112487356B (en) | Structural health monitoring data enhancement method | |
| CN106990109A (en) | A kind of underground belt foreign matter detection system judged based on multi-cam and method | |
| CN115507309B (en) | Gas leakage alarm method and system | |
| CN116433668A (en) | Intelligent hydraulic oil pipe oil leakage detection method | |
| CN113343998A (en) | Reading monitoring system and method for electric power mechanical meter, computer equipment and application | |
| CN115861307A (en) | Fascia gun power supply drive plate welding fault detection method based on artificial intelligence | |
| CN116310760B (en) | Intelligent water conservancy monitoring system based on machine vision | |
| CN116664846B (en) | Method and system for realizing 3D printing bridge deck construction quality monitoring based on semantic segmentation | |
| CN115965625B (en) | Instrument detection device based on visual recognition and detection method thereof | |
| CN117173130A (en) | Pipeline detection method, system, terminal equipment and storage medium | |
| CN112232235A (en) | Intelligent factory remote monitoring method and system based on 5G | |
| CN116259008A (en) | Water level real-time monitoring method based on computer vision | |
| CN112784914B (en) | Pipe gallery video intelligent attribute detection method and system based on cloud processing | |
| KR102281100B1 (en) | System and method for providing heat transporting pipe status information | |
| CN112529881A (en) | Method and device for identifying cable abnormity of electric control cabinet | |
| CN110309838A (en) | Video detection area contour of object based on exponential transform detects preprocess method | |
| JPH1042274A (en) | Abnormality monitoring method and device | |
| CN111767815A (en) | Tunnel water leakage identification method | |
| CN117541583B (en) | Drill rod corrosion monitoring method and system for explosion-proof monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |