CN116563944A - Abnormal behavior detection method, device, computer equipment and storage medium - Google Patents
Abnormal behavior detection method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116563944A CN116563944A CN202310470030.8A CN202310470030A CN116563944A CN 116563944 A CN116563944 A CN 116563944A CN 202310470030 A CN202310470030 A CN 202310470030A CN 116563944 A CN116563944 A CN 116563944A
- Authority
- CN
- China
- Prior art keywords
- image information
- abnormal behavior
- information
- visible light
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 206010000117 Abnormal behaviour Diseases 0.000 title claims abstract description 156
- 238000001514 detection method Methods 0.000 title claims abstract description 106
- 238000012549 training Methods 0.000 claims abstract description 79
- 230000004927 fusion Effects 0.000 claims abstract description 46
- 238000004590 computer program Methods 0.000 claims abstract description 25
- 238000004458 analytical method Methods 0.000 claims abstract description 14
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000000034 method Methods 0.000 abstract description 23
- 230000000391 smoking effect Effects 0.000 abstract description 11
- 230000008569 process Effects 0.000 abstract description 9
- 238000012545 processing Methods 0.000 description 27
- 239000011159 matrix material Substances 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 230000003321 amplification Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 239000003571 electronic cigarette Substances 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000010687 lubricating oil Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Social Psychology (AREA)
- Psychiatry (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Image Analysis (AREA)
Abstract
The application relates to an abnormal behavior detection method, an abnormal behavior detection device, computer equipment, a storage medium and a computer program product, and model training is performed based on visible light training image information and infrared training image information to obtain a preset double-flow fusion target detection model. In the actual detection process, visible light image information and infrared image information can be obtained simultaneously, and analysis is carried out by combining with a trained preset double-flow fusion target detection model, so that whether current target personnel have abnormal behaviors such as smoking or not is judged, and finally, alarm prompt information is output under the condition of detecting the abnormal behaviors. According to the scheme, the advantages of predicting the abnormal behavior by the visible light image and predicting the abnormal behavior by the infrared light are fused, and the abnormal behavior of the target person can be still effectively detected under the condition that partial shielding or a longer distance occurs, so that the detection precision is high.
Description
Technical Field
The present invention relates to the field of detection technologies, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for detecting abnormal behavior.
Background
With the continuous development of the power grid technology, the types and the quantity of electric loads in the power grid are more and more, the power grid operation is more and more complex and various, and the safe operation of the power transmission line is important for ensuring the power supply reliability of the power grid. Most equipment and instruments in the transmission line are subjected to daily maintenance, and various lubricating oils are required to be used, and are extremely easy to cause fire when encountering open fire, so that serious safety accidents are caused.
At present, an infrared identification technology is generally adopted to detect whether staff has abnormal behaviors such as smoking or not, so as to avoid fire disaster. However, when a situation such as occlusion occurs, the abnormal behavior is often difficult to detect, and the conventional abnormal behavior detection method has a disadvantage of low detection accuracy.
Disclosure of Invention
Based on this, it is necessary to provide an abnormal behavior detection method, apparatus, computer device, storage medium, and computer program product with high detection accuracy in view of the above-described technical problems.
An abnormal behavior detection method, comprising: obtaining visible light image information and infrared image information corresponding to a target person; obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; the preset double-flow fusion target detection model is obtained by carrying out model training according to visible light training image information and infrared training image information; judging whether the target person has abnormal behaviors or not according to the abnormal behavior position information and the temperature information; and if the target personnel have abnormal behaviors, outputting alarm prompt information.
According to the abnormal behavior detection method, model training is performed based on the visible light training image information and the infrared training image information at the same time, and the preset double-flow fusion target detection model is obtained. In the actual detection process, visible light image information and infrared image information can be obtained simultaneously, and analysis is carried out by combining with a trained preset double-flow fusion target detection model, so that whether current target personnel have abnormal behaviors such as smoking or not is judged, and finally, alarm prompt information is output under the condition of detecting the abnormal behaviors. According to the scheme, the advantages of predicting the abnormal behavior by the visible light image and predicting the abnormal behavior by the infrared light are fused, and the abnormal behavior of the target person can be still effectively detected under the condition that partial shielding or a longer distance occurs, so that the detection precision is high.
In one embodiment, the determining manner of the preset double-flow fusion target detection model includes: acquiring visible light training image information and infrared training image information; preprocessing, splicing and extracting features of the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images; performing encoder coding according to the characteristic image and the position coding to obtain a first output value; performing feature reduction, prediction head network prediction and proportional network matching according to the first output value to obtain a second output value; performing gradient return loss calculation according to the second output value to obtain trained network parameters; and updating the double-flow target detection model according to the trained network parameters to obtain a preset double-flow fusion target detection model.
In one embodiment, the encoder includes at least one of a multi-headed attention mechanism network, a residual network, a layer normalization network, and a feed forward network.
In one embodiment, the feature extraction of the visible training image information and the infrared training image information respectively includes: and respectively extracting the characteristics of the visible light training image information and the infrared training image through a lightweight neural network.
In one embodiment, the obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model includes: performing model analysis according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model to obtain abnormal behavior position information; and carrying out temperature inversion according to the infrared image information to obtain temperature information.
In one embodiment, the determining whether the target person has abnormal behavior according to the abnormal behavior position information and the temperature information includes: if the abnormal behavior position information is matched with preset abnormal position information, determining that the target person has abnormal behavior; or if the temperature information which is larger than or equal to the preset temperature threshold exists, determining that the target person has abnormal behaviors.
An abnormal behavior detection apparatus comprising: the image acquisition module is used for acquiring visible light image information and infrared image information corresponding to the target personnel; the model matching module is used for obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; the preset double-flow fusion target detection model is obtained by carrying out model training according to visible light training image information and infrared training image information; the abnormal judgment module is used for judging whether the target person has abnormal behaviors or not according to the abnormal behavior position information and the temperature information; and the alarm prompt module is used for outputting alarm prompt information if the target personnel have abnormal behaviors.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the above-described abnormal behavior detection method when the computer program is executed.
A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the abnormal behavior detection method described above.
A computer program product comprising a computer program which, when executed by a processor, implements the steps of the abnormal behavior detection method described above.
Drawings
FIG. 1 is a diagram of an application environment of a method for detecting abnormal behavior in one embodiment of the present application;
FIG. 2 is a flow chart of a method for detecting abnormal behavior according to an embodiment of the present application;
FIG. 3 is a flow chart illustrating abnormal behavior detection in one embodiment of the present application;
FIG. 4 is a flowchart illustrating a method for detecting abnormal behavior according to another embodiment of the present application;
FIG. 5 is a block diagram illustrating an abnormal behavior detection apparatus according to one embodiment of the present application;
fig. 6 is an internal structural diagram of a computer device in one embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The abnormal behavior detection method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. The image acquisition device 102 is in communication connection with the processing device 104 through wired or wireless communication. The processing device 104 may be a terminal, a server or a server cluster, and is not specifically limited as long as it has a corresponding image data processing function. In particular, the terminal may be, but is not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and is not particularly limited.
In more detail, in one embodiment, the abnormal behavior detection method is applied to a power grid, and the image acquisition device 102 is disposed on the power grid and is used for acquiring visible light image information and infrared image information of a power grid worker and sending the visible light image information and the infrared image information to the processing device 104. The processing device 104 may be disposed on the power grid or at a remote end, so long as it is ensured that the visible light image information and the infrared image information sent by the image acquisition device 102 can be received in time, and whether the power grid staff has abnormal behaviors (such as smoking) in the working process can be detected. In other embodiments, the abnormal behavior detection method may also be applied to other scenarios, for example, scenarios that require open flame detection or smoking detection, and the like, and is not limited in particular.
In one embodiment, as shown in fig. 2, there is provided an abnormal behavior detection method, which is applied to a power grid and is illustrated by the processing device 104 in fig. 1, including the following steps:
step 202, obtaining visible light image information and infrared image information corresponding to a target person.
Specifically, the target person is the person needing to detect whether abnormal behaviors exist. The visible light image information is the image information obtained when the image acquisition is carried out based on visible light, and the infrared image information is the image information obtained when the image acquisition is carried out based on infrared light, namely the image information corresponding to the infrared image. The power grid is provided with an image acquisition device, the image acquisition device can acquire images in real time to obtain visible light image information and infrared image information, and the visible light image information and the infrared image information are uploaded to the processing device in a wired or wireless communication mode.
It should be noted that the type of image capturing device is not exclusive, and in one embodiment, may be an image capturing device of a type in which visible light image capturing and infrared image capturing are integrated, and in another embodiment, may also include two image capturing devices, one of which performs visible light image capturing and the other of which performs infrared image capturing, which is not particularly limited.
It will be appreciated that the particular type of abnormal behavior is not exclusive, as long as the abnormal behavior causes a change in the behavior of the target person, and also causes a change in the temperature of the area in which the target person is located. For example, in one more detailed embodiment, the abnormal behavior includes smoking (including but not limited to electronic cigarettes, etc.).
In one embodiment, the visible light image information is image information obtained by labeling each pixel point in the visible light image according to a coordinate system, and the infrared image information is image information obtained by labeling each pixel point in the infrared image according to the coordinate system.
It should be noted that the visible light image information and the infrared image information may be directly acquired by the image acquisition device, that is, after the image acquisition device acquires the infrared image and the visible light image, the visible light image information and the infrared image information are respectively labeled, and sent to the processing device. In another embodiment, the image acquisition device may acquire the infrared image and the visible light image and then directly send the infrared image and the visible light image to the processing device, and when the processing device performs further labeling, the visible light image information and the infrared image information are acquired, which is not particularly limited.
And 204, obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model.
Specifically, a preset double-flow fusion target detection model is obtained by carrying out model training according to visible light training image information and infrared training image information. The preset double-flow fusion target detection model is based on a preset detection model capable of carrying out behavior prediction on double-flow data information (visible light image information and infrared image information), and represents the corresponding relation of the visible light image information, the infrared image information and abnormal behavior position information. Therefore, in the actual application process, after the visible light image information and the infrared image information are input into a preset double-flow fusion target detection model to be predicted, abnormal behavior position information can be output; and further analysis is carried out by combining the infrared image information, so that the temperature actually corresponding to each pixel point in the infrared image can be obtained.
It can be appreciated that in another embodiment, after obtaining the visible light image information and the infrared image information, the processing device may perform preprocessing (including denoising, image enhancement processing, etc.) on the visible light image information and the infrared image information to ensure the subsequent detection accuracy and the detection reliability, and then perform matching analysis by combining the preprocessed visible light image information and the preprocessed infrared image information, and finally perform alarm prompt after detecting that abnormal behavior occurs.
And 206, judging whether the target person has abnormal behaviors according to the abnormal behavior position information and the temperature information.
Specifically, after obtaining the abnormal behavior position information and the temperature information of the target person, the processing device performs further analysis and judgment by combining the abnormal behavior position information and the temperature information, so as to determine whether the target user has abnormal behaviors currently, for example, whether the target user has smoking behaviors currently or not.
And step 208, outputting alarm prompt information if the target personnel have abnormal behaviors.
Specifically, the processing device analyzes the abnormal behavior position information and the temperature information, and outputs alarm prompt information to remind the target person to stop the abnormal behavior or remind the monitoring person that the current target person has the abnormal behavior after judging that the target person has the abnormal behavior.
It should be noted that the manner in which the processing device outputs the alarm prompt information is not unique, and in one embodiment, the alarm prompt information may be output to the alarm device, and may be output through the alarm device in a manner of sound, light, text, or vibration, so as to inform the target person or the monitoring person.
It can be understood that if the processing device determines that the target person does not have abnormal behaviors at the moment, the processing device returns to execute the operation of acquiring the visible light image information and the infrared image information corresponding to the target person, and performs analysis and judgment of the next round again, so that when the target person has abnormal behaviors, alarm prompt information can be timely output for reminding.
According to the abnormal behavior detection method, model training is performed based on the visible light training image information and the infrared training image information at the same time, and the preset double-flow fusion target detection model is obtained. In the actual detection process, visible light image information and infrared image information can be obtained simultaneously, and analysis is carried out by combining with a trained preset double-flow fusion target detection model, so that whether current target personnel have abnormal behaviors such as smoking or not is judged, and finally, alarm prompt information is output under the condition of detecting the abnormal behaviors. According to the scheme, the advantages of predicting the abnormal behavior by the visible light image and predicting the abnormal behavior by the infrared light are fused, and the abnormal behavior of the target person can be still effectively detected under the condition that partial shielding or a longer distance occurs, so that the detection precision is high.
Referring to fig. 3 in combination, in one embodiment, a method for determining a dual-stream fusion target detection model is preset, including: acquiring visible light training image information and infrared training image information; preprocessing, splicing and extracting features of the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images; performing encoder coding according to the characteristic image and the position coding to obtain a first output value; performing feature reduction, prediction head network prediction and direct proportion network matching according to the first output value to obtain a second output value; performing gradient return loss calculation according to the second output value to obtain trained network parameters; and updating the double-flow target detection model (namely, the double-flow Yolov5 model) according to the trained network parameters to obtain a preset double-flow fusion target detection model.
Specifically, the visible light training image information is image information obtained by labeling each pixel point in a visible light image for training according to a coordinate system, and the infrared training image information is image information obtained by labeling each pixel point in an infrared image for training according to the coordinate system.
For example, in one more detailed embodiment, the visible training image information or the infrared training image information corresponding to the target person may be labeled (x, y, w, h), where (x, y) represents the center point coordinates of the target person region, w represents the width of the target person region, h represents the height of the target person region, and a category thereof is labeled L and represents the target person.
The obtained visible light training image information and infrared training image information are preprocessed, the specific processing mode is not unique, and the method is selected according to actual requirements, for example, data denoising, image enhancement or amplification processing and the like, and the method is not limited in detail.
And then, combining the obtained visible light training image information and infrared training image information to perform feature extraction, namely extracting coordinate position parameters corresponding to feature points in the visible light training image information and the infrared training image information, and splicing the coordinate position parameters corresponding to the two image information together to obtain a final feature image and position codes of all pixel points in the feature image. In more detail, in one embodiment, the Position Encoding (PE) is calculated as follows:
wherein pos represents the position of the pixel point in the feature image, i represents the dimension of the feature map, d model Representing the vector dimensions of the dual stream object detection model setup.
After that, the obtained feature image and the position code are input to an encoder for encoding processing, and a first output value is obtained. Specifically, the encoder includes six encoders of the same type connected in sequence of the transducer network. In the first encoder, a feature image X is input pos And Position Encoding (PE) to obtain an output value, inputting the output value to a next encoder, outputting an output value by the next encoder, and so on, wherein the last encoder outputs a first output value. It will be appreciated that in other embodiments, the number of encoders may be provided in other numbers, as may be selected in particular in conjunction with actual requirements.
In the processing procedure of the position encoding and the encoder, the data in the original matrix form is processed into the form of column vectors, so that the first output value needs to be subjected to feature reduction to the same matrix size as the feature layer after feature extraction, and the output value is obtained. And then sequentially inputting the output value into a prediction head network for prediction and matching with a direct proportion network to obtain a second output value. Finally, the second output value and the gradient return loss function are combined to calculate, so that the corresponding network parameters at the moment, namely the trained network parameters, are obtained, the trained network parameters are used for updating the double-flow target detection model, and the trained double-flow target detection model (namely the double-flow fusion Yolov5 model) is obtained and stored and is used as a preset double-flow fusion target detection model.
In one embodiment, the encoder includes at least one of a multi-headed attention mechanism network, a residual network, a layer normalization network, and a feed forward network.
Specifically, for a multi-head attention mechanism network, a concept of Query, key and Value is introduced, query (Q) is the meaning of the Query, key (K) is a Key and is used for comparing with Query to be queried to obtain a score (correlation or similarity) and multiplying Value (V) by a final result. The parameter Q of its input is the output of the last encoder (the feature image X if the current encoder is the first encoder pos ) Plus position coding (PE), and matrix W Q The result of the multiplication, K, is the output of the last encoder plus the position code (PE), and the matrix W K The result of the multiplication, V, is the output of the last encoder plus the position code (PE), and the matrix W V The result of the multiplication.
The multi-head attention mechanism network calculates through the multi-head attention mechanism, and the calculation formula is as follows:
wherein the multi-headed attention mechanism network is used to calculate correlations,in order to change the Attention moment array into standard normal distribution, the multi-head Attention mechanism uses a plurality of different parameters Q, K and V to calculate results of Attention for merging (concat), then performs dimension adjustment through matrix multiplication, and adjusts the dimension to be consistent with the dimension of the input value of the encoder.
The feed-forward network is a two-layer fully connected layer, the activation function of the first layer is Relu, the second layer does not use the activation function, and the dimension of the output matrix finally obtained by the feed-forward network is consistent with the dimension of the input matrix. The calculation formula is as follows:
output＝max(0，XW 1 + 1 )W 2 + 2
where X represents an input, W1 represents a matrix in full connection, b1 represents a bias term, max () represents a function taking the maximum value, W2 represents a matrix in full connection, and b2 represents a bias term.
In more detail, in one embodiment, for the above-described encoders for encoding, each encoder includes a multi-headed attention mechanism network, a residual network, a layer normalization network, a feed forward network, a residual network, and a layer normalization network connected in sequence.
In one embodiment, feature extraction is performed on visible training image information and infrared training image information, respectively, including: and respectively extracting the characteristics of the visible light training image information and the infrared training image through a lightweight neural network.
Specifically, in the scheme of the embodiment, two lightweight neural networks are arranged in the dual-flow target detection model, when feature extraction is performed, feature extraction is performed by adopting one lightweight neural network for visible light training image information and infrared training image information respectively, a visible light feature image and an infrared feature image after feature extraction are obtained, then the two images are spliced, and subsequent operation is performed. In more detail, in one embodiment, the lightweight neural network is specifically a MobileNet V2 type network, each MobileNet V2 network is responsible for extracting a characteristic of data, and the MobileNet V2 network is lighter, so that the test speed can be ensured in the practical application process.
Referring to fig. 4, in one embodiment, step 204 includes steps 402 and 404.
And step 402, performing model analysis according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model to obtain abnormal behavior position information.
And step 404, performing temperature inversion according to the infrared image information to obtain temperature information.
Specifically, the preset double-flow fusion target detection model characterizes the corresponding relation between the visible light image information and the infrared image information and the abnormal behavior position information, so that the abnormal behavior position of the current target personnel can be finally confirmed by analyzing according to the preset double-flow fusion target detection model and the acquired visible light image information and infrared image information. And the temperature information can be obtained by carrying out temperature inversion according to the infrared images only, so that the temperature information corresponding to different pixel points in different infrared images can be obtained. Finally, the processing device combines the abnormal behavior position information and the temperature information to perform abnormal behavior analysis.
It should be noted that, in one embodiment, after the visible light image information and the infrared image information are input into the preset double-flow fusion target detection model to perform prediction, a predicted bounding box is output, after that, a mode of maximum suppression is required to be used to suppress repeated boxes, and the output result is taken as a final prediction box to obtain abnormal behavior position information of the target personnel.
In one embodiment, determining whether the target person has abnormal behavior according to the abnormal behavior position information and the temperature information includes: if the abnormal behavior position information is matched with the preset abnormal position information, determining that the target person has abnormal behaviors; or if the temperature information which is larger than or equal to the preset temperature threshold exists, determining that the target person has abnormal behaviors.
Specifically, after the processing device obtains the abnormal behavior position information and the temperature information, the processing device respectively matches the abnormal behavior position information with preset abnormal position information, compares and analyzes the temperature information of each pixel point with a preset temperature threshold value, and if the abnormal behavior position information is matched with the preset abnormal position information or the temperature information which is greater than or equal to the preset temperature threshold value exists, the target person is considered to have abnormal behaviors at the moment, so that alarm prompt information is output.
In order to facilitate understanding of the technical solutions of the present application, the present application is explained below in connection with more detailed embodiments.
Firstly, image acquisition is carried out on a smoker through an image acquisition device (a camera which can integrate visible light and infrared acquisition), and the acquired visible light image and infrared image are marked to obtain visible light training image information and infrared training image information. And then denoising, image enhancement and image amplification are carried out on the visible light training image information and the infrared training image information, so that the processed visible light training image information and infrared training image information are obtained. And respectively inputting the processed visible light training image information and infrared training image information into corresponding MobileNet V2 networks, extracting features, and splicing and encoding the data output after the features of the two MobileNet V2 networks are extracted to obtain feature images and corresponding position codes. And then inputting the characteristic image and the position code into an encoder to sequentially perform coding processing, and finally outputting a first output value.
And then the first output is restored to the same size as the feature layer after the feature extraction module, and is input into the prediction head module for prediction, and then is transmitted to the positive example matching module for matching, so that a second output value is obtained. And finally, calculating the classification regression loss by combining the second output value, and updating network parameters through gradient return to obtain a trained double-flow target detection model, namely a preset double-flow fusion target detection model.
In the actual application process, the camera acquires visible light image information and infrared image information in real time, inputs the visible light image information and the infrared image information into a preset double-flow fusion target detection model for prediction, outputs a predicted boundary frame, suppresses repeated frames by using a non-maximum suppression mode, and takes an output result as a final prediction frame to obtain abnormal behavior position information. Meanwhile, temperature inversion can be performed according to the infrared image information, and the temperature information of each pixel point in the infrared image can be determined. Finally, when the abnormal behavior position information is detected to be matched with the preset abnormal position information or the temperature information which is larger than or equal to the preset temperature threshold value exists, alarm prompt information is output to prompt the smoker to exist currently or warn the smoker to stop smoking.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an abnormal behavior detection device for realizing the abnormal behavior detection method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiment of the device for detecting abnormal behavior provided in the following may be referred to the limitation of the method for detecting abnormal behavior in the above description, and will not be repeated here.
In one embodiment, as shown in fig. 5, there is provided an abnormal behavior detection apparatus including: an image acquisition module 502, a model matching module 504, an anomaly determination module 506, and an alarm prompting module 508, wherein:
the image acquisition module 502 is configured to acquire visible light image information and infrared image information corresponding to a target person; the model matching module 504 is configured to obtain abnormal behavior position information and temperature information according to visible light image information, infrared image information and a preset double-flow fusion target detection model; the method comprises the steps that a preset double-flow fusion target detection model is obtained through model training according to visible light training image information and infrared training image information; the abnormality determination module 506 is configured to determine whether the target person has abnormal behavior according to the abnormal behavior location information and the temperature information; the alarm prompt module 508 is configured to output alarm prompt information if the target person has abnormal behavior.
In one embodiment, the model matching module 504 is further configured to perform model analysis according to the visible light image information, the infrared image information, and the preset double-flow fusion target detection model, so as to obtain abnormal behavior position information; and carrying out temperature inversion according to the infrared image information to obtain temperature information.
In one embodiment, the alarm prompting module 508 is further configured to determine that the target person has abnormal behavior if the abnormal behavior location information matches with the preset abnormal location information; or if the temperature information which is larger than or equal to the preset temperature threshold exists, determining that the target person has abnormal behaviors.
The respective modules in the above-described abnormal behavior detection apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
The abnormal behavior detection device performs model training based on the visible light training image information and the infrared training image information at the same time to obtain a preset double-flow fusion target detection model. In the actual detection process, visible light image information and infrared image information can be obtained simultaneously, and analysis is carried out by combining with a trained preset double-flow fusion target detection model, so that whether current target personnel have abnormal behaviors such as smoking or not is judged, and finally, alarm prompt information is output under the condition of detecting the abnormal behaviors. According to the scheme, the advantages of predicting the abnormal behavior by the visible light image and predicting the abnormal behavior by the infrared light are fused, and the abnormal behavior of the target person can be still effectively detected under the condition that partial shielding or a longer distance occurs, so that the detection precision is high.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of abnormal behavior detection. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 6 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
obtaining visible light image information and infrared image information corresponding to a target person; obtaining abnormal behavior position information and temperature information according to visible light image information, infrared image information and a preset double-flow fusion target detection model; judging whether the target personnel has abnormal behaviors or not according to the abnormal behavior position information and the temperature information; if the target person has abnormal behaviors, outputting alarm prompt information.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
obtaining visible light image information and infrared image information corresponding to a target person; obtaining abnormal behavior position information and temperature information according to visible light image information, infrared image information and a preset double-flow fusion target detection model; judging whether the target personnel has abnormal behaviors or not according to the abnormal behavior position information and the temperature information; if the target person has abnormal behaviors, outputting alarm prompt information.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
obtaining visible light image information and infrared image information corresponding to a target person; obtaining abnormal behavior position information and temperature information according to visible light image information, infrared image information and a preset double-flow fusion target detection model; judging whether the target personnel has abnormal behaviors or not according to the abnormal behavior position information and the temperature information; if the target person has abnormal behaviors, outputting alarm prompt information.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.
The computer equipment, the storage medium and the computer program product perform model training based on the visible light training image information and the infrared training image information at the same time to obtain a preset double-flow fusion target detection model. In the actual detection process, visible light image information and infrared image information can be obtained simultaneously, and analysis is carried out by combining with a trained preset double-flow fusion target detection model, so that whether current target personnel have abnormal behaviors such as smoking or not is judged, and finally, alarm prompt information is output under the condition of detecting the abnormal behaviors. According to the scheme, the advantages of predicting the abnormal behavior by the visible light image and predicting the abnormal behavior by the infrared light are fused, and the abnormal behavior of the target person can be still effectively detected under the condition that partial shielding or a longer distance occurs, so that the detection precision is high.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. An abnormal behavior detection method, comprising:
obtaining visible light image information and infrared image information corresponding to a target person;
obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; the preset double-flow fusion target detection model is obtained by carrying out model training according to visible light training image information and infrared training image information;
judging whether the target person has abnormal behaviors or not according to the abnormal behavior position information and the temperature information;
and if the target personnel have abnormal behaviors, outputting alarm prompt information.
2. The abnormal behavior detection method according to claim 1, wherein the determining manner of the preset double-flow fusion target detection model includes:
acquiring visible light training image information and infrared training image information;
preprocessing, splicing and extracting features of the visible light training image information and the infrared training image information respectively to obtain corresponding feature images and position codes of the feature images;
performing encoder coding according to the characteristic image and the position coding to obtain a first output value;
performing feature reduction, prediction head network prediction and proportional network matching according to the first output value to obtain a second output value;
performing gradient return loss calculation according to the second output value to obtain trained network parameters;
and updating the double-flow target detection model according to the trained network parameters to obtain a preset double-flow fusion target detection model.
3. The abnormal behavior detection method according to claim 2, wherein the encoder comprises at least one of a multi-headed attention mechanism network, a residual network, a layer normalization network, and a feed forward network.
4. The abnormal behavior detection method according to claim 2, wherein feature extraction is performed on the visible light training image information and the infrared training image information, respectively, comprising:
and respectively extracting the characteristics of the visible light training image information and the infrared training image through a lightweight neural network.
5. The abnormal behavior detection method according to claim 1, wherein the obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model includes:
performing model analysis according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model to obtain abnormal behavior position information;
and carrying out temperature inversion according to the infrared image information to obtain temperature information.
6. The abnormal behavior detection method according to any one of claims 1 to 5, wherein the determining whether the target person has abnormal behavior based on the abnormal behavior position information and the temperature information includes:
if the abnormal behavior position information is matched with preset abnormal position information, determining that the target person has abnormal behavior; or alternatively, the first and second heat exchangers may be,
if the temperature information which is larger than or equal to the preset temperature threshold exists, determining that the target person has abnormal behaviors.
7. An abnormal behavior detection apparatus, comprising:
the image acquisition module is used for acquiring visible light image information and infrared image information corresponding to the target personnel;
the model matching module is used for obtaining abnormal behavior position information and temperature information according to the visible light image information, the infrared image information and a preset double-flow fusion target detection model; the preset double-flow fusion target detection model is obtained by carrying out model training according to visible light training image information and infrared training image information;
the abnormal judgment module is used for judging whether the target person has abnormal behaviors or not according to the abnormal behavior position information and the temperature information;
and the alarm prompt module is used for outputting alarm prompt information if the target personnel have abnormal behaviors.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the abnormal behavior detection method of any one of claims 1 to 6 when the computer program is executed.
9. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the abnormal behavior detection method of any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the abnormal behavior detection method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310470030.8A CN116563944A (en) | 2023-04-24 | 2023-04-24 | Abnormal behavior detection method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310470030.8A CN116563944A (en) | 2023-04-24 | 2023-04-24 | Abnormal behavior detection method, device, computer equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563944A true CN116563944A (en) | 2023-08-08 |
Family
ID=87487217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310470030.8A Pending CN116563944A (en) | 2023-04-24 | 2023-04-24 | Abnormal behavior detection method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563944A (en) |
-
2023
- 2023-04-24 CN CN202310470030.8A patent/CN116563944A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109544396B (en) | Account recommendation method and device, server, terminal and storage medium | |
| CN112884092B (en) | AI model generation method, electronic device, and storage medium | |
| CN111860140B (en) | Target event detection method, device, computer equipment and storage medium | |
| JP2018117338A (en) | Automated object and activity tracking in live video feed | |
| JP6352512B1 (en) | Signal processing apparatus, signal processing method, signal processing program, and data structure | |
| CN117115743A (en) | Mining safety production monitoring system and method thereof | |
| JP2018142137A (en) | Information processing device, information processing method and program | |
| CN112508137A (en) | Transformer abnormality detection method and device, computer equipment and storage medium | |
| CN117033039A (en) | Fault detection method, device, computer equipment and storage medium | |
| Chen et al. | Safety Helmet‐Wearing Detection System for Manufacturing Workshop Based on Improved YOLOv7 | |
| CN108898067B (en) | Method and device for determining association degree of person and object and computer-readable storage medium | |
| CN116563944A (en) | Abnormal behavior detection method, device, computer equipment and storage medium | |
| CN115271117A (en) | Method and device for diagnosing faults of internal equipment of converter station and computer equipment | |
| CN114241354A (en) | Warehouse personnel behavior identification method and device, computer equipment and storage medium | |
| US11470098B2 (en) | Terminal device and controlling method thereof | |
| CN114238968A (en) | Application program detection method and device, storage medium and electronic equipment | |
| CN118314123B (en) | Welding radiographic defect detection method, device, computer equipment and medium | |
| CN117454241A (en) | Analysis method, device and computer equipment for monitoring alarm event | |
| CN117557108A (en) | Training method and device for intelligent identification model of power operation risk | |
| CN117151873A (en) | Abnormality prompting method, abnormality prompting device, computer equipment and storage medium | |
| CN117011513A (en) | Fault location method, apparatus, device, storage medium and computer program product | |
| Sun et al. | Remaining useful life prognostics of bearings based on convolution attention networks and enhanced transformer | |
| CN118155383A (en) | Early warning processing method, early warning processing device, computer equipment and storage medium | |
| CN115457482A (en) | Image recognition method, device and system and computer equipment | |
| CN116995812A (en) | Monitoring method and system for power transmission line terminal equipment, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |