CN117011729A - Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle - Google Patents

Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle Download PDF

Info

Publication number
CN117011729A
CN117011729A CN202311107380.4A CN202311107380A CN117011729A CN 117011729 A CN117011729 A CN 117011729A CN 202311107380 A CN202311107380 A CN 202311107380A CN 117011729 A CN117011729 A CN 117011729A
Authority
CN
China
Prior art keywords
data
human body
unmanned aerial
format
aerial vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311107380.4A
Other languages
Chinese (zh)
Inventor
徐亮
林昶荣
王志敏
任晓波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Huasairuifei Intelligent Technology Co ltd
Original Assignee
Shenzhen Huasairuifei Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Huasairuifei Intelligent Technology Co ltd filed Critical Shenzhen Huasairuifei Intelligent Technology Co ltd
Priority to CN202311107380.4A priority Critical patent/CN117011729A/en
Publication of CN117011729A publication Critical patent/CN117011729A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Remote Sensing (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the technical field of unmanned aerial vehicles, in particular to a method and a system for detecting abnormal behaviors of a human body of an unmanned aerial vehicle. According to the scheme, the unmanned aerial vehicle is controlled to position the human body according to the acquired environmental data so as to acquire image data of the human body, then the acquired image data is subjected to data processing through a pre-trained data processing model so as to obtain a target format image file, and finally the image data in the target format image file is subjected to recognition processing through a pre-trained recognition model so as to obtain an abnormal recognition result representing abnormal behaviors of the human body, so that the abnormal behaviors of the human body are recognized. By adopting the method and the system for detecting the abnormal behavior of the human body, the detection can be realized without dead angles, the image data of the human body can be obtained, the specific abnormal behavior identification is facilitated, the sensitivity and the robustness of the abnormal behavior identification can be improved, and the identification result is more accurate and comprehensive.

Description

Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle
Technical Field
The application relates to the technical field of unmanned aerial vehicles, in particular to a method and a system for detecting abnormal behaviors of a human body of an unmanned aerial vehicle.
Background
The human body abnormal behavior detection is a technology for monitoring and analyzing human body behaviors in real time, and can be widely applied to security scenes. With the development of intelligent home and internet of things, more and more families begin to use security equipment for monitoring. The traditional human body abnormal behavior detection technology generally depends on a fixed monitoring camera, and blind areas and dead angles exist in the method, so that all areas cannot be comprehensively monitored. The mobile security is realized by a rail-mounted inspection robot, a wheel-type patrol robot or a mobile security robot, but the manufacturing cost is too high, the deployment is difficult, and the problem of competing with people exists. Therefore, it is necessary to study a detection apparatus and method capable of ensuring high detection accuracy while being flexibly movable and easy to maintain.
Disclosure of Invention
The method and the system for detecting the abnormal behavior of the human body of the unmanned aerial vehicle effectively solve the problem that the equipment in the prior art cannot accurately and comprehensively detect the abnormal behavior of the human body.
According to a first aspect, in one embodiment, a method for detecting abnormal behavior of a human body of an unmanned aerial vehicle is provided, including:
acquiring environmental data, wherein the environmental data comprises at least one of infrared data, audio data and video data;
controlling the unmanned aerial vehicle to position a human body according to the environmental data so as to acquire image data of the human body;
inputting the image data into a pre-trained data processing model for data processing to obtain a target format image file;
and inputting the target format image file into a pre-trained recognition model for recognition processing so as to recognize an abnormal recognition result used for representing abnormal behaviors of the human body in the image data.
In an implementation manner, after the identifying the abnormal identification result used for characterizing the abnormal behavior of the human body in the image data, the method further includes:
and controlling the unmanned aerial vehicle to make an avoidance action and sending out an alarm.
In an implementation manner, the inputting the image data into a pre-trained data processing model for data processing to obtain the image file in the target format includes:
cutting out an interested region image in the image data, and storing the interested region image;
intercepting the region-of-interest image frame by frame to obtain a plurality of pictures;
labeling each picture to enable the pictures to generate a labeling file in a first format;
and receiving a format conversion instruction, and converting the annotation file in the first format into an image file in the second format.
In an implementation manner, the inputting the target format image file into a pre-trained recognition model for recognition processing includes:
calculating the second format image file through the cascade cavity volume and the activation function to obtain the spatial attention weight of the second format image file;
performing weighted screening on the spatial attention weight of the second format image file to obtain a feature map based on the spatial attention weight;
channel attention learning is carried out on the feature map based on the spatial attention weight, and a fusion feature map is obtained;
and calculating the fusion feature map through a network model to obtain an abnormal recognition result.
In an implementation manner, the weighted filtering of the high-resolution feature map is performed on the spatial attention weight of the second format image file to obtain a feature map based on the spatial attention weight, and the calculation formula is as follows:
where z is a feature map based on spatial attention weight, y is txt format image file, x is high resolution feature map, DConv is a hole convolution function, and Sigmoid is an activation function.
In an implementation manner, the channel attention learning is performed on the feature map based on the spatial attention weight to obtain a fused feature map, and the calculation formula is as follows:
in the formula, r is a fusion characteristic diagram, relu is an activation function, avg is an average function, and expansion_as is an expansion data function.
According to a second aspect, in one embodiment, a system for detecting abnormal behavior of a human body of an unmanned aerial vehicle is provided, including:
the acquisition module is used for acquiring environment data, wherein the environment data comprises infrared data, audio data and video data;
the positioning module is used for controlling the unmanned aerial vehicle to position a human body according to the environmental data so as to acquire image data of the human body;
the data processing module is used for inputting the image data into a pre-trained data processing model for data processing so as to obtain a target format image file;
the identification module is used for inputting the target format image file into a pre-trained identification model for identification processing so as to identify abnormal data in the image data, wherein the abnormal data are used for representing abnormal behaviors of a human body.
In one implementation, the system further includes an alarm module;
and the alarm module is used for controlling the unmanned aerial vehicle to make an avoidance action and simultaneously giving out an alarm when the abnormal data in the image data are identified.
In one implementation, the data processing module includes:
the clipping unit is used for clipping the region of interest image in the image data and storing the region of interest image;
the frame intercepting unit is used for intercepting the region of interest image frame by frame to obtain a plurality of pictures;
the labeling unit is used for labeling each picture so that the picture generates a labeling file in a first format;
and the format conversion unit is used for receiving a format conversion instruction and converting the annotation file in the first format into the image file in the second format.
According to a third aspect, an embodiment provides a computer readable storage medium having stored thereon a program executable by a processor to implement the method described above.
According to the method and the system for detecting the abnormal behavior of the human body of the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to position the human body according to the acquired environmental data so as to acquire the image data of the human body, then the acquired image data is subjected to data processing through a pre-trained data processing model so as to acquire a target format image file, and finally the image data in the target format image file is subjected to recognition processing through a pre-trained recognition model so as to acquire an abnormal recognition result representing the abnormal behavior of the human body, so that the recognition of the abnormal behavior of the human body is realized. By adopting the human body abnormal behavior detection method and system, the unmanned aerial vehicle is used for collecting the image data of the human body, so that the unmanned aerial vehicle can realize no dead angle, monitor in all directions and acquire the image data of the human body, and the collected image data is preprocessed to be helpful for carrying out abnormal behavior recognition in a targeted manner, and the recognition module trained in advance is used for carrying out recognition processing, so that the sensitivity and the robustness of the abnormal behavior recognition can be improved, and the recognition result is more accurate and comprehensive.
Drawings
Fig. 1 is a flowchart of a method for detecting abnormal behaviors of an unmanned aerial vehicle according to the present embodiment;
fig. 2 is a flowchart of image data processing provided in the present embodiment;
FIG. 3 is a flowchart for identifying processed data according to the present embodiment;
fig. 4 is a diagram of an algorithm structure of an identification model according to the present embodiment;
fig. 5 is a schematic diagram of an identification operation structure for performing identification processing on image data according to the present embodiment;
fig. 6 is a block diagram of the unmanned aerial vehicle human body abnormal behavior detection system according to the present embodiment;
fig. 7 is a block diagram of a data processing module according to the present embodiment.
Reference numerals: 10. an acquisition module; 20. a positioning module; 30. a data processing module; 31. a cutting unit; 32. a frame capturing unit; 33. a labeling unit; 34. a format conversion unit; 40. and an identification module.
Detailed Description
The application will be described in further detail below with reference to the drawings by means of specific embodiments. Wherein like elements in different embodiments are numbered alike in association. In the following embodiments, numerous specific details are set forth in order to provide a better understanding of the present application. However, one skilled in the art will readily recognize that some of the features may be omitted, or replaced by other elements, materials, or methods in different situations. In some instances, related operations of the present application have not been shown or described in the specification in order to avoid obscuring the core portions of the present application, and may be unnecessary to persons skilled in the art from a detailed description of the related operations, which may be presented in the description and general knowledge of one skilled in the art.
Furthermore, the described features, operations, or characteristics of the description may be combined in any suitable manner in various embodiments. Also, various steps or acts in the method descriptions may be interchanged or modified in a manner apparent to those of ordinary skill in the art. Thus, the various orders in the description and drawings are for clarity of description of only certain embodiments, and are not meant to be required orders unless otherwise indicated.
The numbering of the components itself, e.g. "first", "second", etc., is used herein merely to distinguish between the described objects and does not have any sequential or technical meaning. The term "coupled" as used herein includes both direct and indirect coupling (coupling), unless otherwise indicated.
In the prior art, when a technology for detecting abnormal behaviors of a human body is applied to a room, the following disadvantages generally exist: first, the complexity of the indoor environment may affect the accuracy of the technique. As various obstacles and other interference factors, such as furniture, electrical appliances, light, etc., may exist in the room, these factors may interfere with the field of view and sensing effect of the camera, resulting in erroneous judgment or missing report, which provides a greater challenge for algorithm accuracy. Secondly, a large number of high-quality monitoring devices, including high-definition cameras, intelligent sensors and the like, are required to be used for detecting abnormal behaviors of the human body, and have high cost, professional installation and maintenance are required, so that the deployment cost and the operation cost of the system are increased. Therefore, when the human body abnormal behavior detection method is applied to the indoor mobile security equipment, how to optimize the algorithm effect of human body abnormal behavior detection in the indoor scene and how to replace the traditional monitoring mode by using the unmanned plane technology need to be considered.
In order to solve the problems and further realize detection of abnormal behaviors of the human body by the indoor unmanned aerial vehicle, the application provides a detection method of the abnormal behaviors of the human body by the indoor unmanned aerial vehicle. According to the method, firstly, an abnormal behavior detection algorithm is optimized, and meanwhile, reasonable logic is provided for the application of the unmanned aerial vehicle in an indoor scene, so that a user can detect abnormal behaviors of a human body indoors by using the unmanned aerial vehicle. In addition, in order to realize abnormal behavior identification, the application aims at utilizing the visual angle of the unmanned aerial vehicle, monitoring in real time in an indoor scene, detecting the behavior of the attack unmanned aerial vehicle, defining an abnormal behavior as the attack unmanned aerial vehicle, and focusing on analyzing and detecting the abnormal behavior. By finding out such behavior in time and avoiding, the unmanned aerial vehicle is protected as much as possible or the shooting time is prolonged.
The following specifically describes a method and a system for detecting abnormal behaviors of the unmanned aerial vehicle in detail.
As shown in fig. 1, the method for detecting abnormal behaviors of a human body of an unmanned aerial vehicle provided in this embodiment includes the following steps:
step 100: environmental data including at least one of infrared data, audio data, and video data is acquired.
The flight speed and the track of the unmanned aerial vehicle are controlled to detect possible abnormal behaviors in detail. Specifically, the infrared data can be acquired through the infrared sensor, the audio data can be acquired through the sound sensor, and the video data can be acquired through the video sensor.
Step 200: and controlling the unmanned aerial vehicle to position the human body according to the environmental data so as to acquire image data of the human body.
In the step, image data related to a person is acquired from the acquired infrared data, audio data and/or video data, and after the image data related to the person is acquired, the human body is positioned by controlling the flying speed and track of the unmanned aerial vehicle so as to accurately acquire the image data of the human body.
Step 300: inputting the image data into a pre-trained data processing model for data processing to obtain the target format image file.
In the step, before the acquired human body image data is subjected to data processing, in order to obtain a better detection effect, the application establishes an own experimental data set aiming at abnormal behaviors in a specific scene. From two aspects: firstly, defining abnormal behaviors, namely aiming at attack behaviors of the unmanned aerial vehicle; secondly, in the action track of a person, many other actions are often accompanied, so that a person's person-none (p-none for short) needs to be defined, i.e. no specific action of the person is identified. Such considerations are based primarily on the need for video surveillance, from human localization to behavior recognition, where the first step is to locate in order to monitor the video more clearly.
And then the image data is processed, and the image data is input into a pre-trained data processing model for data processing, and because the optimized identification model is a labeling file with a specific format, the image data is required to be processed first to obtain the image file with the target format.
Step 400: inputting the target format image file into a pre-trained recognition model for recognition processing so as to recognize an abnormal recognition result used for representing abnormal behaviors of the human body in the image data.
In the step, after the image data are processed in the step, a target format image file is obtained, the target format image file is input into a pre-trained recognition model, invalid information in the target format image file, such as small object feature information, is filtered out due to optimization of a recognition algorithm of the recognition model, so that attention to an interested region is improved, the recognition model is used for recognizing the interested region, after abnormal behaviors are recognized, recognition results output by the recognition model are displayed in a form of a labeling frame, the abnormal behaviors are regarded as a category, and names and confidence of the category are displayed in the labeling frame.
In this embodiment, the unmanned aerial vehicle is controlled to position the human body according to the acquired environmental data to acquire image data of the human body, then the acquired image data is subjected to data processing through a pre-trained data processing model to obtain a target format image file, and finally the image data in the target format image file is subjected to recognition processing through a pre-trained recognition model to obtain an abnormal recognition result representing abnormal behaviors of the human body, so that the abnormal behaviors of the human body are recognized. By adopting the human body abnormal behavior detection method, the unmanned aerial vehicle is used for collecting the image data of the human body, so that no dead angle can be realized, the image data of the human body can be monitored in an all-round way, the collected image data is preprocessed, the abnormal behavior can be identified in a targeted way, and the recognition processing is carried out through the recognition module trained in advance, so that the sensitivity and the robustness of the abnormal behavior recognition can be improved, and the recognition result is more accurate and comprehensive.
The system for detecting the abnormal behavior of the human body is arranged on the unmanned aerial vehicle, the unmanned aerial vehicle can be applied to various indoor and outdoor scenes, and the system is more focused on indoor applications, such as families, prisons, museums, exhibition halls, data centers and other places needing security. By the arrangement, the detection method has expandability and flexibility, and can be adjusted and optimized according to different requirements so as to adapt to the actual conditions of different occasions. The solution of the application is therefore a very promising technology, which may lead to a more efficient, intelligent and reliable solution for the field of security monitoring.
In addition, in this embodiment, after identifying the abnormal identification result used for characterizing the abnormal behavior of the human body in the image data, the method further includes the following steps: and controlling the unmanned aerial vehicle to make an avoidance action and sending out an alarm.
Specifically, after the abnormal behavior is identified, the identification module sends an abnormal identification result to the alarm module, and the alarm module is triggered after receiving the signal, so that a warning is sent to an administrator and related information is provided so as to take corresponding actions, and further indoor safety can be effectively guaranteed.
As shown in fig. 2, in step 300, the image data is input into a pre-trained data processing model for data processing to obtain a target format image file, which specifically includes the following steps:
step 310: and cutting out the region of interest image in the image data, and storing the region of interest image.
Step 320: and intercepting the image of the region of interest frame by frame to obtain a plurality of pictures.
Step 330: and labeling each picture to enable the picture to generate a labeling file in a first format.
Step 340: and receiving a format conversion instruction, and converting the annotation file in the first format into the image file in the second format.
In this embodiment, specifically, image data of related behaviors needs to be acquired first. In practice, detecting abnormal behavior is mainly dependent on analyzing a certain state of a certain behavior and distinguishing it from normal behavior, so we can directly analyze the behavior state in video images. We can crop the image and save the region of interest image at a rate of 3 frames for 1 second using the ffmpeg tool. Then, the image of the region of interest is intercepted frame by frame to obtain a plurality of pictures. And then labeling the pictures, wherein a LabelImg labeling tool can be adopted in the label calibration process. When LabelImg is used, generating a catalog according to the name of an initial folder and a specified format, firstly creating a folder, and creating a folder named JPEGImages in the folder to store pictures to be marked; creating a label folder named as Annogens storage label; finally, a txt file named predefined_class.txt is created to store the class name to be annotated, and the finally generated annotation file is in xml format (i.e. first format). Then, upon receiving the instruction for the number conversion, the xml format (first format) is converted into txt format (second format).
In the application, the data processing model acquires a large amount of data through network crawling keyword pictures to manufacture a data set, meanwhile, a fixed camera is used for collecting abnormal behavior state data and acquiring partial data from recorded videos of products, and in the training stage, the collected image data are arranged in the same way as the data processing steps in the using stage, and the images are cut and stored. Then, to ensure the robustness of the data set, we should acquire image data of various scenes and different persons as much as possible. In the image screening process, frame interception, calibration and other operations are required to be carried out so as to obtain a plurality of calibrated pictures. Finally, generating a catalog according to the naming of the initial folder and a designated format, firstly creating a folder, and creating a folder named JPEGImages in the folder to store pictures to be marked; creating a label folder named as Annogens storage label; finally, a txt file named predefined_class.txt is created to store the class name to be annotated, and the finally generated annotation file is in xml format. In addition, in order to prevent over fitting and increase the robustness of the algorithm model, a data enhancement method is also adopted, mainly performing horizontal operation, vertical transformation and diagonal mirror transformation on the image, so as to increase the number of samples, considering that the data set samples may be too few in the training stage.
As shown in fig. 3, in step 400, the target format image file is input into a pre-trained recognition model for recognition processing, and specifically includes the following steps:
step 410: and calculating the second-format image file through the cascade hole volume and the activation function to obtain the spatial attention weight of the second-format image file.
Step 420: and carrying out weighted screening on the spatial attention weight of the second format image file to obtain a feature map based on the spatial attention weight.
Step 430: and carrying out channel attention learning on the feature map based on the spatial attention weight to obtain a fusion feature map.
Step 440: and calculating the fusion feature map through the network model to obtain an abnormal recognition result.
In this embodiment, specifically, in order to improve the attention of the region of interest, noise in the feature needs to be filtered out. In this embodiment, a YOLOv5 network model is mainly adopted and optimized, and a cross-stage attention feature filtering module is embedded in the YOLOv5 network model. Specifically, the main idea is that a low-resolution feature map, namely a second format image file, is calculated and extracted through a cascade cavity convolution and Sigmoid activation function, after the spatial attention weight of the second format image file is obtained, the spatial attention weight is applied to the weighted screening of a high-resolution feature map, and the feature map based on the spatial attention weight is obtained, so that the feature utilization efficiency is improved. And fitting complex correlations among channels by using the feature graphs with different spatial attention weights through channel attention learning, and further carrying out feature fusion on a plurality of features with different weights to obtain a fused feature graph. And finally, carrying out recognition calculation on the fusion feature map through the YOLOv5 network model to obtain an abnormal recognition result. As shown in fig. 4, the algorithm structure diagram of the YOLOv5 network model for identifying and calculating the fusion feature map is shown. Specifically, after the calculation is finished, if a labeling frame appears in the picture, abnormal behavior is considered, otherwise, abnormal behavior is not generated.
In this embodiment, specifically, the spatial attention weight of the second format image file is weighted and filtered to obtain a feature map based on the spatial attention weight, and the calculation formula is as follows:
where z is a feature map based on spatial attention weight, y is txt format image file, x is high resolution feature map, DConv is a hole convolution function, and Sigmoid is an activation function.
Referring to the recognition algorithm schematic of fig. 5, where y is a low resolution feature map, as input to the cross-stage attention feature filter module, from the same level corresponding to this stage. X as another input, results from downsampling of the high resolution feature map. First, the low-resolution feature map y is subjected to hole convolution with a hole rate of 3 and calculated by using a Sigmoid activation function, and is multiplied by the input feature map X to obtain a feature map z based on the spatial attention weight.
In this embodiment, channel attention learning is performed on a feature map based on spatial attention weights to obtain a fused feature map, and the calculation formula is as follows:
in the formula, r is a fusion characteristic diagram, relu is an activation function, avg is an average function, and expansion_as is an expansion data function.
In particular, feature map z has aggregated spatial information by weighting X filters, but channel attention is ignored. Where z itself already contains a priori knowledge of some spatial attention mechanism, so that performing subsequent channel attention operations on the basis of z can further integrate features in both the spatial and channel dimensions, complementarily applied to the input feature map. And z, adjusting the channel weight through the channel attention to obtain the feature r to be fused. Then, the original input y and the processed feature r are subjected to a stitching operation in accordance with the channel dimension. The expressive force of the region of interest can be improved by adopting an attention mechanism, and the characteristics with significance in the channel and space dimensions can be effectively extracted. By the design, important features and invalid features can be focused, so that effective flow of feature information in the model is enhanced, and the YOLOv5 network model can realize end-to-end training.
As shown in fig. 6, the system for detecting abnormal behaviors of an unmanned aerial vehicle provided in this embodiment includes an acquisition module 10, a positioning module 20, a data processing module 30, and an identification module 40. The acquiring module 10 is configured to acquire environmental data, where the environmental data includes infrared data, audio data, and video data; the positioning module 20 is used for controlling the unmanned aerial vehicle to position the human body according to the environmental data so as to acquire image data of the human body; the data processing module 30 is used for inputting the image data into a pre-trained data processing model for data processing so as to obtain a target format image file; the recognition module 40 is configured to input the target format image file into a pre-trained recognition model for recognition processing, so as to recognize abnormal data in the image data, where the abnormal data is used to characterize abnormal behaviors of the human body.
In the system for detecting abnormal behaviors of the human body of the unmanned aerial vehicle in the embodiment, the positioning module 20 controls the unmanned aerial vehicle to position the human body according to the environmental data acquired by the acquisition module 10 so as to acquire image data of the human body, then the data processing module 30 performs data processing on the acquired image data through a pre-trained data processing model so as to acquire a target format image file, and finally the recognition module 40 performs recognition processing on the image data in the target format image file through the pre-trained recognition model so as to acquire an abnormal recognition result representing the abnormal behaviors of the human body, thereby realizing recognition on the abnormal behaviors of the human body. By adopting the human body abnormal behavior detection system, the unmanned aerial vehicle is used for collecting the image data of the human body, so that no dead angle can be realized, the image data of the human body can be monitored in an all-round way, the collected image data is preprocessed, the abnormal behavior can be identified in a targeted way, and the recognition module 40 trained in advance is used for carrying out recognition processing, so that the sensitivity and the robustness of the abnormal behavior recognition can be improved, and the recognition result is more accurate and comprehensive. In this embodiment, the descriptions of the acquisition module 10, the positioning module 20, the data processing module 30, and the identification module 40 are described in detail in the above method for detecting abnormal behaviors of the unmanned aerial vehicle, and the description of this embodiment is omitted here.
In addition, the system in the embodiment further comprises an alarm module. The alarm module is used for controlling the unmanned aerial vehicle to make an avoidance action and simultaneously giving out an alarm when abnormal data in the image data are identified.
Specifically, after the abnormal behavior is identified, the identification module 40 sends the abnormal identification result to the alarm module, and the alarm module is triggered after receiving the signal, so as to warn the administrator and provide relevant information to take corresponding action, thereby effectively helping to ensure indoor safety.
Referring to fig. 7, the data processing module 30 in the present embodiment includes a clipping unit 31, a frame capturing unit 32, a labeling unit 33, and a format conversion unit 34. Specifically, the cropping unit 31 is configured to crop the region of interest image in the image data, and store the region of interest image; the frame capturing unit 32 is configured to capture the image of the region of interest frame by frame, so as to obtain a plurality of pictures; the labeling unit 33 is configured to label each picture, so that the picture generates a labeling file in a first format; the format conversion unit 34 is configured to receive an instruction for format conversion, and convert the annotation file in the first format into the image file in the second format.
In this embodiment, specifically, image data of related behaviors needs to be acquired first. In practice, detecting abnormal behavior is mainly dependent on analyzing a certain state of a certain behavior and distinguishing it from normal behavior, so we can directly analyze the behavior state in video images. The cropping unit 31 crops the image at a speed of 1 second and 3 frames by using a ffmpeg tool and saves the region of interest image. Then, the region of interest image is cut out frame by the frame cutting out unit 32, resulting in a plurality of pictures. Next, the pictures are marked by the marking unit 33, and a LabelImg marking tool may be used in the label marking process. When LabelImg is used, generating a catalog according to the name of an initial folder and a specified format, firstly creating a folder, and creating a folder named JPEGImages in the folder to store pictures to be marked; creating a label folder named as Annogens storage label; finally, a txt file named predefined_class.txt is created to store the class name to be annotated, and the finally generated annotation file is in xml format (i.e. first format). Then, after receiving the instruction of the number conversion, the xml format (first format) is converted into txt format (second format) by the format conversion unit 34.
A computer readable storage medium is provided in this embodiment, where a program is stored on the medium, and the program can be executed by a processor to implement the method described above, and this embodiment is not described herein in detail.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by a computer program. When all or part of the functions in the above embodiments are implemented by means of a computer program, the program may be stored in a computer readable storage medium, and the storage medium may include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc., and the program is executed by a computer to realize the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above can be realized. In addition, when all or part of the functions in the above embodiments are implemented by means of a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and the program in the above embodiments may be implemented by downloading or copying the program into a memory of a local device or updating a version of a system of the local device, and when the program in the memory is executed by a processor.
The foregoing description of the application has been presented for purposes of illustration and description, and is not intended to be limiting. Several simple deductions, modifications or substitutions may also be made by a person skilled in the art to which the application pertains, based on the idea of the application.

Claims (10)

1. The method for detecting the abnormal behavior of the human body of the unmanned aerial vehicle is characterized by comprising the following steps of:
acquiring environmental data, wherein the environmental data comprises at least one of infrared data, audio data and video data;
controlling the unmanned aerial vehicle to position a human body according to the environmental data so as to acquire image data of the human body;
inputting the image data into a pre-trained data processing model for data processing to obtain a target format image file;
and inputting the target format image file into a pre-trained recognition model for recognition processing so as to recognize an abnormal recognition result used for representing abnormal behaviors of the human body in the image data.
2. The method for detecting abnormal behavior of a human body of an unmanned aerial vehicle according to claim 1, wherein after the recognition of the abnormal recognition result for characterizing the abnormal behavior of the human body in the image data, further comprises:
and controlling the unmanned aerial vehicle to make an avoidance action and sending out an alarm.
3. The method for detecting abnormal behaviors of a human body of an unmanned aerial vehicle according to claim 1, wherein the inputting the image data into a pre-trained data processing model for data processing to obtain a target format image file comprises:
cutting out an interested region image in the image data, and storing the interested region image;
intercepting the region-of-interest image frame by frame to obtain a plurality of pictures;
labeling each picture to enable the pictures to generate a labeling file in a first format;
and receiving a format conversion instruction, and converting the annotation file in the first format into an image file in the second format.
4. The method for detecting abnormal behaviors of a human body of an unmanned aerial vehicle according to claim 3, wherein the step of inputting the target format image file into a pre-trained recognition model for recognition processing comprises the following steps:
calculating the second format image file through the cascade cavity volume and the activation function to obtain the spatial attention weight of the second format image file;
performing weighted screening on the spatial attention weight of the second format image file to obtain a feature map based on the spatial attention weight;
channel attention learning is carried out on the feature map based on the spatial attention weight, and a fusion feature map is obtained;
and calculating the fusion feature map through a network model to obtain an abnormal recognition result.
5. The method for detecting abnormal behaviors of a human body of an unmanned aerial vehicle according to claim 4, wherein the weighted screening of the high-resolution feature map is performed on the spatial attention weight of the image file in the second format to obtain a feature map based on the spatial attention weight, and the calculation formula is as follows:
where z is a feature map based on spatial attention weight, y is txt format image file, x is high resolution feature map, DConv is a hole convolution function, and Sigmoid is an activation function.
6. The method for detecting abnormal behaviors of an unmanned aerial vehicle according to claim 5, wherein the channel attention learning is performed on the feature map based on the spatial attention weight to obtain a fused feature map, and the calculation formula is as follows:
in the formula, r is a fusion characteristic diagram, relu is an activation function, avg is an average function, and expansion_as is an expansion data function.
7. A system for detecting abnormal human body behavior of an unmanned aerial vehicle, comprising:
the acquisition module is used for acquiring environment data, wherein the environment data comprises infrared data, audio data and video data;
the positioning module is used for controlling the unmanned aerial vehicle to position a human body according to the environmental data so as to acquire image data of the human body;
the data processing module is used for inputting the image data into a pre-trained data processing model for data processing so as to obtain a target format image file;
the identification module is used for inputting the target format image file into a pre-trained identification model for identification processing so as to identify abnormal data in the image data, wherein the abnormal data are used for representing abnormal behaviors of a human body.
8. The system for detecting abnormal human body behavior of an unmanned aerial vehicle according to claim 7, wherein the system further comprises an alarm module;
and the alarm module is used for controlling the unmanned aerial vehicle to make an avoidance action and simultaneously giving out an alarm when the abnormal data in the image data are identified.
9. The system for detecting abnormal behavior of a human body of an unmanned aerial vehicle according to claim 7, wherein the data processing module comprises:
the clipping unit is used for clipping the region of interest image in the image data and storing the region of interest image;
the frame intercepting unit is used for intercepting the region of interest image frame by frame to obtain a plurality of pictures;
the labeling unit is used for labeling each picture so that the picture generates a labeling file in a first format;
and the format conversion unit is used for receiving a format conversion instruction and converting the annotation file in the first format into the image file in the second format.
10. A computer readable storage medium, characterized in that the medium has stored thereon a program, which is executable by a processor to implement the method of any of claims 1-6.
CN202311107380.4A 2023-08-29 2023-08-29 Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle Pending CN117011729A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311107380.4A CN117011729A (en) 2023-08-29 2023-08-29 Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311107380.4A CN117011729A (en) 2023-08-29 2023-08-29 Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle

Publications (1)

Publication Number Publication Date
CN117011729A true CN117011729A (en) 2023-11-07

Family

ID=88576331

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311107380.4A Pending CN117011729A (en) 2023-08-29 2023-08-29 Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle

Country Status (1)

Country Link
CN (1) CN117011729A (en)

Similar Documents

Publication Publication Date Title
Elharrouss et al. A review of video surveillance systems
US11735018B2 (en) Security system with face recognition
US9560323B2 (en) Method and system for metadata extraction from master-slave cameras tracking system
CN108230594B (en) Method for generating alarm in video monitoring system
CN101918989B (en) Video surveillance system with object tracking and retrieval
KR101425505B1 (en) The monitering method of Intelligent surveilance system by using object recognition technology
US20160019427A1 (en) Video surveillence system for detecting firearms
JP6013923B2 (en) System and method for browsing and searching for video episodes
CN107122743B (en) Security monitoring method and device and electronic equipment
WO2021095351A1 (en) Monitoring device, monitoring method, and program
US11210529B2 (en) Automated surveillance system and method therefor
KR20160014413A (en) The Apparatus and Method for Tracking Objects Based on Multiple Overhead Cameras and a Site Map
KR102107957B1 (en) Cctv monitoring system for detecting the invasion in the exterior wall of building and method thereof
KR20160093253A (en) Video based abnormal flow detection method and system
CN113283859A (en) Edge platform system applied to edge computing management
CN117011729A (en) Method and system for detecting abnormal behaviors of human body of unmanned aerial vehicle
US20220406066A1 (en) Surveillance system and method for automatically executing a security function and system and method for generating a synthetic training data set
KR20130047131A (en) Method and system for surveilling contents of surveillance using mobile terminal
US11854266B2 (en) Automated surveillance system and method therefor
KR100902275B1 (en) Cctv system for intelligent security and method thereof
Nishanthini et al. Smart Video Surveillance system and alert with image capturing using android smart phones
Kent et al. An autonomous sensor module based on a legacy CCTV camera
den Hollander et al. Automatically assessing properties of dynamic cameras for camera selection and rapid deployment of video content analysis tasks in large-scale ad-hoc networks
VICTORY VIDEO SURVEILLANCE USING MOTION DETECTION
Patil et al. Integrating Artificial Intelligence with Camera Systems for Automated Surveillance and Analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination