CN114241423A - Intelligent detection method and system for river floaters - Google Patents
Intelligent detection method and system for river floaters Download PDFInfo
- Publication number
- CN114241423A CN114241423A CN202111611405.5A CN202111611405A CN114241423A CN 114241423 A CN114241423 A CN 114241423A CN 202111611405 A CN202111611405 A CN 202111611405A CN 114241423 A CN114241423 A CN 114241423A
- Authority
- CN
- China
- Prior art keywords
- neural network
- network model
- river
- module
- pulse
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent detection method and system for river course floaters, which are used for extracting river course video image data and performing data preprocessing; establishing a river channel floater data set; constructing a pulse neural network model; training the preprocessed riverway floating object data set, and converting the trained equivalent convolutional neural network model into a pulse neural network detection model; and acquiring river channel video test data in real time, and inputting the trained pulse neural network model to perform real-time intelligent detection on river channel floating objects. The method is suitable for detecting the real-time floater in various river channel scenes. The method for identifying the river channel floating objects by using the impulse neural network can provide a rapid and lightweight deep neural network prediction mode, can be applied to embedded equipment, mobile equipment and cheap computing environments, is easy to embed systems such as unmanned aerial vehicles and city monitoring, has the advantages of high detection speed, high precision and strong timeliness, and is easy to use in practice.
Description
Technical Field
The invention relates to the field of image analysis, in particular to an intelligent detection method and system for river floating objects.
Background
River channel environmental management is an important ring of urban environment improvement, and the mode of observing or monitoring on the spot river channel monitoring through manpower investigates the quantity, source, distribution of river channel floater, and is inefficient, and often because the improper selection of the time of decontaminating causes the problem of water-supply not smooth, the quality of decontaminating is low etc.. How to utilize mobile devices such as city monitoring system and unmanned aerial vehicle to carry out intelligent monitoring to the river course floater, provide the distribution situation of the audio-visual information of river course floater and river course floater in real time, be favorable to river administration personnel to make quick response according to the data analysis who provides. The river treatment efficiency is improved, and great economic value and practical significance are achieved.
In the past years, a plurality of methods are applied to detection of river course floating objects, and with rapid development of the fields of machine vision and deep learning, the intelligent detection precision of river course floating object detection is greatly improved, however, a deep network based on a convolutional neural network generally needs a computer with a high-performance GPU processor to operate, and is difficult to be embedded into cheap mobile equipment.
Disclosure of Invention
In order to solve the defects of the prior art and achieve the purposes of reducing the parameters of the neural network for detecting the river floaters, reducing the energy consumption and improving the efficiency, the invention adopts the following technical scheme:
an intelligent detection method for river floaters comprises the following steps:
s1, extracting river channel video image data and performing data preprocessing;
s2, establishing a river channel floating object data set, marking the position of a river channel floating object in river channel image data by using annotation software, generating a corresponding real label, and constructing a sample data set;
s3, constructing a pulse neural network model;
s4, training the preprocessed river floating object data set, and converting the trained equivalent convolutional neural network model into a pulse neural network detection model, comprising the following steps:
s41, performing equivalent convolutional neural network model training through the sample data set, predicting the real label, and obtaining a trained equivalent convolutional neural network model;
s42, converting the trained equivalent convolutional neural network model into a pulse neural network model, converting convolutional layers of the equivalent convolutional neural network model into synaptic connections similar to convolutional operation, converting pooling layers into synaptic connections for pooling operation, converting full-connection layers into synaptic connections for full-connection operation, integrating pulses by adopting IF neurons among different synapses, and converting bias of the equivalent convolutional neural network model into a pulse current form to be injected into corresponding neurons;
and S5, acquiring river channel video test data in real time, inputting the converted impulse neural network model, and carrying out real-time intelligent detection on river channel floating objects.
Further, in S1, river video stream data is collected, and frame image extraction is performed on the video stream at certain time intervals to obtain continuous multi-frame river image data.
Further, in S2, data expansion including random flipping and color enhancement is performed on the data set by using an image processing technique, so as to increase the number of samples.
Further, in the training of the equivalent convolutional neural network model in S41, the sample data set is input into the equivalent convolutional neural network model, the prediction label is output through calculation, loss calculation is performed on the output prediction label and the real label in the data set, the obtained loss value is reversely propagated into the equivalent convolutional neural network model, and the weight of the equivalent convolutional neural network model is updated until model fitting is performed, so that the trained equivalent convolutional neural network model is obtained.
Furthermore, the pulse neural network model converts a plurality of continuous images into pulse forms for input through a certain coding mode, so that the space-time information is better combined, the capability of the pulse neural network for processing space-time information data is enhanced, the characteristic information and the semantic information of the floaters with a plurality of scales are extracted, the characteristic information is fused, a group of characteristic graphs with different scales and levels are output, the output characteristics have high precision and rich semantic information, the detection precision of the floaters is improved, and the corresponding floaters are regressed and output to extract results.
Further, in S5, the operation S1 is performed on the real-time acquired river video data, and a group of multi-frame images are extracted each time in a sliding window form in the time dimension and input to the impulse neural network model for prediction, so as to obtain the extraction result of the floating objects in each frame of image of the video.
Further, the result of extracting the floating objects in S5 includes the specific position coordinates, size and number of the floating objects.
An intelligent detection system for river floaters comprises a data acquisition module, a data processing module, a model construction module, a model conversion module and a real-time detection module, wherein the model conversion module comprises an equivalent convolutional neural network model and a conversion module;
the data acquisition module is used for extracting video image data of the river channel and carrying out data preprocessing;
the data processing module is used for establishing a river channel floater data set, marking the position of river channel floaters in the river channel image data by using annotation software, generating a corresponding real label and constructing a sample data set;
the model construction module is used for constructing a pulse neural network model and predicting the real label to obtain a trained equivalent convolutional neural network model;
the equivalent convolutional neural network model predicts the real label through the sample data set to obtain a trained equivalent convolutional neural network model;
the conversion module is used for converting the trained equivalent convolutional neural network model into a pulse neural network model, converting convolutional layers of the equivalent convolutional neural network model into synaptic connections similar to convolutional operation, converting pooling layers into synaptic connections for pooling operation, converting full-connection layers into synaptic connections for full-connection operation, integrating pulses by adopting IF neurons among different synapses, and converting bias of the equivalent convolutional neural network model into a pulse current form to be injected into corresponding neurons;
and the real-time detection module acquires river channel video test data in real time, inputs the converted pulse neural network model and carries out real-time intelligent detection on river channel floating objects.
Furthermore, the data acquisition module acquires river channel video stream data, and performs frame image extraction on the video stream according to a certain time interval to obtain continuous multi-frame river channel image data.
Further, the pulse neural network model comprises a pulse coding module, an input module, a feature extraction module, a multi-scale feature fusion module and a regression output module;
the pulse coding module converts a plurality of frames of continuous images into a pulse form through a certain coding mode so as to better combine the spatiotemporal information and enhance the capacity of the pulse neural network for processing spatiotemporal information data;
the input module is used for inputting the coded pulse information to the feature extraction unit;
the characteristic extraction module is used for extracting characteristic information and semantic information of the floating objects with multiple scales;
the multi-scale feature fusion module outputs a group of feature maps with different scale levels, so that the output features have high precision and rich semantic information, and the detection precision of the floaters is improved;
and the regression output unit is used for outputting the corresponding floater extraction result. Including float size, number, and specific location coordinates.
The invention has the advantages and beneficial effects that:
the invention improves the real-time intelligent detection capability of the river course floaters, can quickly and accurately monitor the size, the quantity and the specific positions of the floaters in the river course, thereby quantitatively evaluating the accumulation condition of the river course floaters, providing reliable technical data support for the urban river course decontamination work, using a rapid and lightweight deep neural network prediction mode of the pulse neural network, eliminating the influence of external factors such as illumination conditions, climate change and the like on the detection, being easy to be used in embedded equipment, mobile equipment and a cheap computing environment, being applicable to unmanned aerial vehicles, urban monitoring and other systems, having the advantages of wide application range, high detection speed, high precision, strong timeliness, wide applicability and the like, being applicable to the real-time floaters detection of various river course scenes and having wide application value in practice.
Drawings
FIG. 1 is a process flow diagram of an embodiment of the present invention.
Fig. 2 is a schematic system configuration according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of a neural network model training process according to an embodiment of the present invention.
FIG. 4 is a schematic diagram of a model test process according to an embodiment of the present invention.
FIG. 5 is a schematic diagram of a model element according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present invention, are given by way of illustration and explanation only, not limitation.
The invention discloses an intelligent detection method and system for river floaters and an application technology, and as shown in figure 1, the method comprises the following steps:
a data extraction step, namely extracting river channel video image data and performing data preprocessing;
a sample construction step, namely establishing a river floating object data set;
a model construction step, namely constructing a pulse neural network model;
a model conversion step, training the preprocessed river floating object data set, and converting the trained equivalent convolutional neural network model into a pulse neural network detection model;
and a real-time detection step, namely acquiring river channel video test data in real time, and inputting the trained pulse neural network model to carry out real-time intelligent detection on river channel floaters.
The system structure is shown in fig. 2, and comprises:
the data acquisition module is used for extracting video image data of the river channel and carrying out data preprocessing;
the data processing module is used for establishing a river floating object data set;
the model building module is used for building a pulse neural network model;
the model conversion module is used for converting the trained equivalent convolutional neural network model into a pulse neural network detection model to obtain a floater detection pulse network model;
and the real-time detection module acquires river channel video test data in real time and inputs the trained pulse neural network model to perform real-time intelligent detection on river channel floaters.
As shown in fig. 3, the neural network model training process is schematically illustrated;
the method comprises the following steps: and extracting frame images of the video stream at certain time intervals to obtain continuous multi-frame river channel image data.
Step two: marking the positions of the river floats in the river image data in the data extraction step by using annotation software to generate corresponding tag files, wherein the tag files comprise the categories of the floats and coordinate information (x 1, x2, y1, y 2) of four points of an annotation frame, and performing data expansion in the modes of random overturning, color enhancement and the like on a data set by using an image processing technology to increase the number of samples and manufacture a sample data set.
Step three: constructing a river channel floater detection pulse network model, as shown in fig. 4, specifically includes: the device comprises a pulse coding module, an input module, a feature extraction module, a multi-scale feature fusion module and a regression output module.
(a) The pulse coding module is used for converting three frames of time continuous images into a pulse form in a certain coding mode;
(b) and the input module is used for inputting the coded pulse into the pulse network model.
(c) And the feature extraction module is used for extracting features of multiple scales and extracting semantic information of the input data.
(d) The multi-scale feature fusion module is used for specifically fusing three different scale levels to output feature maps, so that the output features have high precision and abundant semantic information, and the detection precision of floaters is improved.
(e) And the regression output module is used for specifically predicting and classifying position frames of the floating object information and outputting the corresponding size, number and specific position coordinates of the floating objects.
Training an equivalent convolutional neural network model, using the constructed sample data and performing model training, wherein the optional method comprises the following steps:
and carrying out continuous multi-frame grouping on the image data after data expansion in a time sequence, training an input network, calculating an output prediction label of the input image through an equivalent convolutional neural network, carrying out loss calculation on the output label and a real label in a data set, reversely transmitting the calculated loss value to the equivalent convolutional neural network, updating the weight of the equivalent convolutional neural network until model fitting is carried out, and obtaining an optimal equivalent convolutional neural network model.
Step four: and converting the optimal equivalent model into a river floating object detection pulse network model constructed in the third step, specifically, converting the convolution layer into synaptic connections similar to convolution operation, converting the pooling layer into synaptic connections for pooling operation, converting the full-connection layer into synaptic connections for full-connection operation, and integrating pulses by adopting IF neurons among different synapses. And converting the bias of the equivalent convolutional neural network into a pulse current form and injecting the pulse current form into the corresponding neuron.
As shown in fig. 5, the model test flow is as follows:
the method comprises the following steps: acquiring a river channel video data stream to be detected in real time;
step two: performing frame image extraction on the river channel video data stream according to a certain time interval to obtain continuous multi-frame river channel image data;
step two: extracting a group of multi-frame images of the river channel image obtained in the step one each time in a sliding window mode on a time dimension, and inputting the images into an impulse neural network model for prediction;
step three: the river course floater detection pulse network model outputs the floater extraction result in each frame of image of the river course video to be detected, wherein the floater extraction result comprises the size, the number and the position information of the floaters, and the position information is specifically the position coordinates (x 1, x2, y1 and y 2) of the image where the floater detection frame is located.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. An intelligent detection method for river floaters is characterized by comprising the following steps:
s1, extracting river channel video image data and performing data preprocessing;
s2, establishing a river channel floater data set, marking the position of river channel floaters in river channel image data, generating a corresponding real label, and constructing a sample data set;
s3, constructing a pulse neural network model;
s4, training the preprocessed river floating object data set, and converting the trained equivalent convolutional neural network model into a pulse neural network detection model, comprising the following steps:
s41, performing equivalent convolutional neural network model training through the sample data set, predicting the real label, and obtaining a trained equivalent convolutional neural network model;
s42, converting the trained equivalent convolutional neural network model into a pulse neural network model, converting convolutional layers of the equivalent convolutional neural network model into synaptic connections of convolutional operation, converting pooling layers into synaptic connections of pooling operation, converting full-connection layers into synaptic connections of full-connection operation, integrating pulses by adopting neurons among different synapses, and converting bias of the equivalent convolutional neural network model into a pulse current form to be injected into corresponding neurons;
and S5, acquiring river channel video test data in real time, inputting the converted impulse neural network model, and carrying out real-time intelligent detection on river channel floating objects.
2. The method according to claim 1, wherein in S1, the method comprises collecting video stream data of the river, and extracting frames of the video stream at certain time intervals to obtain continuous multi-frame image data of the river.
3. The method according to claim 1, wherein in S2, data expansion including random inversion and color enhancement is performed on the data set.
4. The method according to claim 1, wherein the training of the equivalent convolutional neural network model in S41 is performed by inputting a sample data set into the equivalent convolutional neural network model, outputting a prediction tag through calculation, performing loss calculation on the output prediction tag and a real tag in the data set, back-propagating the obtained loss value into the equivalent convolutional neural network model, and updating the weight of the equivalent convolutional neural network model until model fitting is performed to obtain the trained equivalent convolutional neural network model.
5. The method according to claim 1, wherein the pulse neural network model encodes and converts a plurality of continuous images into pulse form for input, extracts feature information and semantic information of the floats with a plurality of scales, fuses the feature information, outputs a group of feature maps with different scales and levels, and outputs corresponding float extraction results in regression.
6. The method according to claim 1, wherein in S5, S1 is performed on the real-time acquired river video data, and a group of multi-frame images are extracted each time in a form of a sliding window in a time dimension and input to the impulse neural network model for prediction, so as to obtain an extraction result of the floating objects in each frame of image of the video.
7. The method according to claim 5, wherein the result of the extraction of the floating objects in S5 includes the specific position coordinates, size and number of the floating objects.
8. The utility model provides a river course floater intellectual detection system, includes data acquisition module, data processing module, model construction module, model conversion module and real-time detection module, and model conversion module includes equivalent convolution neural network model and conversion module, its characterized in that:
the data acquisition module extracts the video image data of the river channel and performs data preprocessing;
the data processing module is used for establishing a river channel floater data set, marking the position of river channel floaters in the river channel image data, generating a corresponding real label and constructing a sample data set;
the model construction module is used for constructing a pulse neural network model and predicting a real label to obtain a trained equivalent convolutional neural network model;
the equivalent convolutional neural network model predicts the real label through the sample data set to obtain a trained equivalent convolutional neural network model;
the conversion module is used for converting the trained equivalent convolutional neural network model into a pulse neural network model, converting convolutional layers of the equivalent convolutional neural network model into synaptic connections of convolutional operation, converting pooling layers into synaptic connections of pooling operation, converting full-connection layers into synaptic connections of full-connection operation, integrating pulses by adopting neurons among different synapses, and converting bias of the equivalent convolutional neural network model into a pulse current form to be injected into the corresponding neurons;
the real-time detection module acquires river channel video test data in real time, inputs the converted pulse neural network model and carries out real-time intelligent detection on river channel floaters.
9. The system according to claim 8, wherein the data acquisition module acquires river video stream data and extracts frame images of the video stream at certain time intervals to obtain continuous multi-frame river image data.
10. The intelligent detection system for the river course floater according to claim 8, wherein the pulse neural network model comprises a pulse coding module, an input module, a feature extraction module, a multi-scale feature fusion module and a regression output module;
the pulse coding module is used for converting a plurality of frames of continuous images into a pulse form through coding;
the input module is used for inputting the coded pulse information to the feature extraction unit;
the characteristic extraction module is used for extracting characteristic information and semantic information of the floating objects with multiple scales;
the multi-scale feature fusion module outputs a group of feature maps with different scale levels;
and the regression output unit is used for outputting a corresponding floater extraction result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111611405.5A CN114241423A (en) | 2021-12-27 | 2021-12-27 | Intelligent detection method and system for river floaters |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111611405.5A CN114241423A (en) | 2021-12-27 | 2021-12-27 | Intelligent detection method and system for river floaters |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114241423A true CN114241423A (en) | 2022-03-25 |
Family
ID=80763339
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111611405.5A Pending CN114241423A (en) | 2021-12-27 | 2021-12-27 | Intelligent detection method and system for river floaters |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114241423A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN117782205A (en) * | 2023-12-18 | 2024-03-29 | 甘肃省武威生态环境监测中心 | River channel environment detection system and early warning method |
-
2021
- 2021-12-27 CN CN202111611405.5A patent/CN114241423A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115249339A (en) * | 2022-06-10 | 2022-10-28 | 广州中科云图智能科技有限公司 | River floating object identification system, method, equipment and storage medium |
| CN115249339B (en) * | 2022-06-10 | 2024-05-28 | 广州中科云图智能科技有限公司 | River float recognition system, method, equipment and storage medium |
| CN117782205A (en) * | 2023-12-18 | 2024-03-29 | 甘肃省武威生态环境监测中心 | River channel environment detection system and early warning method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ye et al. | A review on deep learning-based structural health monitoring of civil infrastructures | |
| CN109815785A (en) | A kind of face Emotion identification method based on double-current convolutional neural networks | |
| CN114693615A (en) | Deep learning concrete bridge crack real-time detection method based on domain adaptation | |
| CN110569843B (en) | Intelligent detection and identification method for mine target | |
| CN113436169A (en) | Industrial equipment surface crack detection method and system based on semi-supervised semantic segmentation | |
| CN114241423A (en) | Intelligent detection method and system for river floaters | |
| US20220315243A1 (en) | Method for identification and recognition of aircraft take-off and landing runway based on pspnet network | |
| Cao et al. | MCS-YOLO: A multiscale object detection method for autonomous driving road environment recognition | |
| Gu et al. | Real-Time vehicle detection and tracking using deep neural networks | |
| CN116486240A (en) | Application of image recognition algorithm in intelligent inspection method of unmanned aerial vehicle of power transmission line | |
| Yang et al. | C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks | |
| CN116206112A (en) | Remote sensing image semantic segmentation method based on multi-scale feature fusion and SAM | |
| Lin et al. | Optimal CNN-based semantic segmentation model of cutting slope images | |
| Xingxin et al. | Adaptive auxiliary input extraction based on vanishing point detection for distant object detection in high-resolution railway scene | |
| Quan et al. | CrackViT: a unified CNN-transformer model for pixel-level crack extraction | |
| Gao et al. | Robust lane line segmentation based on group feature enhancement | |
| CN117520664A (en) | Public opinion detection method and system based on graphic neural network | |
| CN115083229B (en) | Intelligent recognition and warning system of flight training equipment based on AI visual recognition | |
| CN115830309A (en) | SGAN road crack image segmentation method based on Deeplabv3+ | |
| CN111259762B (en) | Pantograph abnormity detection method | |
| CN110427920B (en) | Real-time pedestrian analysis method oriented to monitoring environment | |
| Li et al. | Multi-scale feature extraction and fusion net: Research on UAVs image semantic segmentation technology | |
| Ashraf et al. | Efficient Pavement Crack Detection and Classification Using Custom YOLOv7 Model | |
| Li et al. | Ctmu-net: an improved u-net for semantic segmentation of remote-sensing images based on the combined attention mechanism | |
| Song et al. | Autonomous identification of bridge concrete cracks using unmanned aircraft images and improved lightweight deep convolutional networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |