CN113392760A - Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction - Google Patents
Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction Download PDFInfo
- Publication number
- CN113392760A CN113392760A CN202110658501.9A CN202110658501A CN113392760A CN 113392760 A CN113392760 A CN 113392760A CN 202110658501 A CN202110658501 A CN 202110658501A CN 113392760 A CN113392760 A CN 113392760A
- Authority
- CN
- China
- Prior art keywords
- model
- unsafe
- video
- submodule
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Abstract
The invention discloses a video-based system and a video-based method for identifying unsafe behaviors of non-navigation-stop construction, wherein the system comprises a cloud computing center, a plurality of edge computing nodes and a plurality of front-end sensing devices; the front-end sensing equipment is positioned at an access end of the system, and the cloud computing center is positioned at a far end; the cloud computing center is internally provided with an artificial intelligent model training platform, the platform is used for repeatedly training an unsafe behavior algorithm to obtain an unsafe behavior model by importing a video image data set collected by front-end sensing equipment from the site, and then directly loading the model to an edge computing node, so that the real-time video image from a front-end sensing layer camera is subjected to artificial intelligent analysis, the trained unsafe behavior is recognized, the alarm information of the result is sent to an alarm device of the front-end equipment, and the unsafe behavior is warned. The system and the method intelligently realize the intelligent judgment and warning of unsafe behaviors and can meet the requirement of real-time problem solution on the instant response of intelligent analysis.
Description
Technical Field
The invention relates to an airport, in particular to a video-based system and a video-based method for identifying unsafe behaviors of non-navigation-stop construction.
Background
With the continuous and high-speed development of civil aviation industry in China, the problems of unbalance, incoordination and unsustainability in the development of the industry still stand out, the guarantee capability of an airport is insufficient, and the construction speed lags behind the development requirement, which is one of the problems to be solved urgently. According to the statistics of airport departments of civil aviation, as late as 2020, 241 transport airports, 341 general airports and 523 navigation enterprises are in total in China.
The airport reconstruction and extension project is multiple, the task is heavy, and under the consideration of factors such as the normal rate of airplane flights, the credit of the airport, social economy and the like, the airport cannot be completely closed for special construction, so the non-stop construction becomes the best choice. According to the regulations on the operation safety management of civil airports, non-navigation construction refers to construction in a flight area under the condition that an airport is not closed or is closed in part of time intervals and aircrafts are received and released according to a flight plan. The non-stop construction becomes a normal state under the implementation of the current airport reconstruction and extension project, and in the face of the characteristics of narrow and small field, complex pipelines, large information amount, short effective operation time, more rush operations at night, long construction period, more involved departments and the like, effective technology and management means are adopted to ensure the safe operation of the existing facilities of the airport, reduce the influence of the non-stop construction on the normal operation of the airport to the maximum extent and ensure the safe operation, air defense safety, pipeline safety and construction safety of the airport in the reconstruction and extension process.
Construction safety management is taken as a worldwide problem, and airport non-stop construction safety management and control are no exception. The research related to the construction safety management can be classified into two types, one is management-driven, and the research analyzes the mechanism behind the accident from the self-reason of the accident, and is expected to reduce the cause of the accident from the perspective of organization management or behavior management. The other type is driven by technology, and the research usually adopts high-tech scientific technology to enhance education and supervision on construction safety so as to improve the safety. Computer vision and AI intelligence technologies have begun to be used in job site safety management to improve job site safety.
The video image system is widely deployed in an airport non-stop construction site, is still in the standardized construction stage of basic hardware equipment at present, and is mainly used for recording, transmitting, storing and displaying a construction process and monitoring targets such as personnel, equipment, materials and the like.
The system mainly based on video monitoring adopts a mode of manually judging whether abnormal events occur through a monitor, and has the defects of insufficient intelligent degree, low monitoring efficiency and lack of deep information perception and self-adaptive learning capacity. Meanwhile, the traditional cluster cloud computing service cannot meet the requirement of airport non-stop construction safety risk perception under time delay sensitivity, and effective association and real-time transmission of perception data and decision data are difficult to realize.
Disclosure of Invention
The invention aims to provide a video-based system and a video-based method for identifying unsafe behaviors of non-stop construction, which can realize intelligent identification and warning of various unsafe construction behaviors in the business scene of the non-stop construction of an airport.
In order to achieve the purpose, the invention adopts the technical scheme that:
a system and a method for identifying unsafe behaviors of non-navigation-stop construction based on videos comprise a cloud computing center, a plurality of edge computing nodes and a plurality of front-end sensing devices;
the front-end sensing equipment is positioned at the front end or the access end of the system, the cloud computing center is positioned at the far end of the system, and the edge computing node is positioned between the front-end sensing equipment and the cloud computing center;
the front-end sensing equipment is directly connected with the edge computing node nearby through CAN/RJ45/RS485/RS232, and the edge computing node is remotely connected with the cloud computing center through 5G/4G/LAN/WLAN;
the front-end sensing equipment is provided with a video image acquisition module and a result warning module;
the edge computing nodes have an intelligent unsafe behavior judgment function, the cloud computing center has a deep learning function, and model training of unsafe behavior recognition is achieved;
the cloud computing center is internally provided with an artificial intelligent model training platform, the training platform directly loads the model to an edge computing node after repeatedly training an unsafe behavior model by importing a video image data set of a non-navigation-stop construction unsafe behavior example collected by front-end sensing equipment from the site, so that the real-time video image from a front-end sensing layer camera is subjected to artificial intelligent analysis, the trained unsafe behavior is identified, and the alarm information of the result is sent to an alarm device of the front-end equipment to warn the unsafe behavior; and carrying out sample marking on the identified image, and providing the image for an artificial intelligence model training platform to train.
Preferably, the airport non-stop construction identification warning system adopts a 'cloud-edge-end' network system, and the network structure comprises a front-end sensing layer, an edge computing layer, a cloud computing layer and an intelligent application layer.
Preferably, the front-end sensing layer is located at the front end or the access end of the network system and is responsible for acquiring field video image data and warning unsafe behaviors; the access terminal of the front end sensing layer comprises various cameras, warning device equipment, a signal collecting sensor and a signal controller;
the cloud computing layer is positioned at the far end of the system; the cloud computing layer is responsible for processing and storing global information, bears computing tasks which cannot be executed by the edge computing layer, and issues service rules, algorithm models and standard APIs (application programming interfaces) for development and docking of various applications to the edge computing layer.
Preferably, the edge computing layer is located between the front-end sensing device and the cloud computing center; the edge calculation layer is responsible for summarizing unstructured video data and Internet of things data sent by each front-end sensing device, preprocessing the unstructured video data and the Internet of things data, triggering corresponding actions according to set rules, uploading a processing structure and related data to a cloud end, and triggering a front-end alarm device to alarm unsafe behavior results;
the edge computing node deploys one side close to a data source according to actual needs to provide nearest-end service nearby;
the intelligent application layer provides vertical application services such as object identification, behavior identification and safety management scenes for users by utilizing structural data analyzed and processed and combining specific business requirements and application models, and mainly provides intelligent identification and safety warning application services of unsafe behaviors.
Preferably, an edge management module, a video cloud platform module, an artificial intelligence module and an internet of things platform module are arranged in the cloud computing center;
the edge management module comprises a node management submodule, an assembly management submodule, a service management submodule and a service arrangement submodule so as to realize the resource monitoring function of edge calculation;
the video cloud platform module comprises a video storage sub-module, a video distribution sub-module, a video processing sub-module and a quality diagnosis sub-module so as to realize the function of structured data of video images;
the artificial intelligence module comprises an algorithm market submodule, a model training submodule, a sample marking submodule and a model distributing submodule so as to realize the intelligent management function of the model algorithm;
the Internet of things platform module comprises an equipment management submodule, a connection management submodule, a data analysis submodule, a data management submodule, a rule engine submodule and a user management submodule so as to realize the management function of various kinds of equipment.
Preferably, an edge computing network computing storage module, an operating system integrated platform module, a Docker container module, an application program module, an equipment management module and a scheduling management module are arranged in the edge computing node;
the operating system integrated platform module comprises a network submodule, a calculation submodule, a storage submodule and an operating system submodule so as to realize the function of an edge calculation support platform;
the Docker container module is used for realizing light and modularized use of a plurality of application programs in the virtual machine;
the application program module comprises n unsafe behavior identification application programs so as to realize identification of various unsafe behaviors. The device management module is used for realizing the management function of the access device;
the scheduling management module is used for realizing task scheduling functions, including task scheduling of the cloud computing center and the edge computing nodes and task scheduling between the edge computing nodes.
Preferably, the artificial intelligence model training platform performs model training by using an unsafe behavior model training method, and the unsafe behavior model training method includes the following steps:
a1, combing the safety standard items related to non-navigation construction for the first time, and defining n types of unsafe behavior activities:
the unsafe behaviors are characterized in that various requirements on construction safety in the current common airport operation and engineering specifications are combed according to the requirements on operation safety, air defense safety, pipeline safety and construction safety, whether the requirements in the specifications are met or not is detected in real time and efficiently by utilizing a computer vision technology, and the safety factor of non-navigation-stop construction is increased;
a2, video image acquisition and preprocessing: the method comprises the steps of carrying out noise removal, gray level conversion and geometric correction preprocessing on a received video image shot by a camera, and reducing the influence of a non-navigation construction environment on video analysis;
a3, case collection: collecting unsafe behavior cases in the preprocessed images according to the n classes of unsafe behavior activities defined in the step A1;
a4, image sample labeling: manually marking a target rectangular frame of the unsafe behavior case, performing normalization processing and protecting relevant information of the target rectangular frame, wherein the relevant information comprises an initial position, a width and a height;
a5, image sample storage: storing the marked image samples to form a training video image data set;
a6, training a model;
a7, data post-processing: carrying out some post-processing and sorting on the result obtained by the convolution network, and storing for the next step;
a8, feedback (weight adjustment): the method has the advantages that the method has the function of adjusting the weights obtained by the convolutional network, optimizes the cost function by using a gradient descent method, and adjusts the weight of each layer of convolutional network from back to front step by step, so that the generalization capability of an output model is greatly improved;
a9, model output: and after continuous feedback, model training and data processing, optimizing the scheme to obtain a final model result.
Preferably, the model training in a6 is the core of the output of the unsafe behavior recognition model, and the module comprises a plurality of convolution layers and a plurality of down-sampling and nonlinear transformations; performing first-level and first-level abstraction on a training video image data set to finally obtain model characteristics with certain discrimination capability on images except for training; after a series of processing is carried out on an input video image, information is gradually reduced, valuable information is stored after the processing of a convolution network, redundant information is abandoned, namely image characteristics are abstracted, and a large amount of image characteristic information interacts with each other to jointly obtain a final model.
Preferably, the step of identifying the unsafe behavior algorithm includes the following steps:
b1, model distribution;
b2, model loading: the loaded model is the weight value of each network layer obtained by training the unsafe behavior model training method, the weight value comprises millions of weight value parameters, the unsafe behavior recognition model in the whole recognition process is loaded to the edge computing node in the initial stage of the system, and the unsafe behavior recognition model is dynamically updated according to the lapse of time and the rules of the model distribution submodule;
b3, behavior recognition: after the loading is finished, a behavior recognition network layer is added so as to restrain a recognition result, the behavior recognizes the position of the marked object, and simultaneously predicts the category of the object, so that the confirmation of the credibility of the detection recognition is realized;
b4, image annotation: after the behaviors are identified, outputting the unsafe behaviors as a result of construction without stopping the navigation, and labeling the identified network image;
b5, image sample storage: the identified image and the marked image are subjected to background coding storage, the unsafe behavior identification model is dynamically updated according to the rule of the model distribution submodule, and the updated image sample is deleted at regular time;
b6, providing samples for model training: the stored image samples provide samples for unsafe behavior model training;
b7, data post-processing: the obtained image sample and the model sample can be retrieved through a standard retrieval tool, and the recorded specific event can be queried in real time;
and B8, outputting the updated model sample, and feeding the output model back to the unsafe behavior model training method.
The invention has the beneficial effects that:
the invention provides a video-based system and a video-based method for identifying unsafe behaviors of non-stop construction, which are used for intelligently identifying and warning various unsafe construction behaviors in a business scene of non-stop construction of an airport and have great practical value. By arranging the cloud computing center, the edge computing nodes and the front-end sensing devices, more accuracy of warning identification and identification sensing of colleges and universities are achieved. A brand-new artificial intelligence model training platform is arranged in the computing center, the platform collects a video image data set of an unsafety behavior example of non-navigation-stop construction by leading in a camera of on-site front-end sensing equipment, and after an unsafety behavior algorithm is repeatedly trained to obtain an unsafety behavior model, the model is directly loaded to an edge computing node, so that accurate and efficient artificial intelligence analysis is carried out on a real-time video image from the camera of the front-end sensing equipment, the trained unsafety behavior is identified, and the alarm information of the result is sent to an alarm device of the front-end equipment to warn the unsafety behavior; and carrying out sample labeling on the identified image, and providing the labeled image for an artificial intelligence model training platform to train. The system and the method can intelligently realize the intelligent judgment and warning of unsafe behaviors and meet the requirement of real-time problem solution on the instant response of intelligent analysis.
Drawings
In order to clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can obtain other drawings without creative efforts.
Fig. 1 is a schematic diagram of a cloud-edge-end network system architecture of an airport non-stop construction identification warning system based on video images.
Fig. 2 is a system logic framework diagram of the airport non-stop construction identification warning system based on video images.
FIG. 3 is a block diagram of a method of unsafe behavior model training.
Fig. 4 is a block diagram of a method of identifying unsafe behavior algorithm.
Detailed Description
The invention provides a video-based system and a video-based method for identifying unsafe behaviors of non-navigation-stop construction, and the invention is further described in detail below in order to make the purpose, the technical scheme and the effect of the invention clearer and clearer. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention is described in detail below with reference to the accompanying drawings:
example 1
With reference to fig. 1 to 4, a system and method for identifying unsafe behaviors of non-navigation-stop construction based on video includes a cloud computing center, a plurality of edge computing nodes, and a plurality of front-end sensing devices. The front-end sensing equipment is positioned at the front end or the access end of the system, the cloud computing center is positioned at the far end of the system, and the edge computing node is positioned between the front-end sensing equipment and the cloud computing center; the front-end sensing equipment is directly connected with the edge computing node nearby through CAN/RJ45/RS485/RS232, and the edge computing node is remotely connected with the cloud computing center through 5G/4G/LAN/WLAN.
The front-end sensing equipment is provided with a video image acquisition module and a result warning module; the edge computing nodes have an unsafe behavior intelligent judgment function, the cloud computing center has a deep learning function, and model training of unsafe behavior recognition is achieved. The cloud computing center is internally provided with an artificial intelligent model training platform, the training platform is used for repeatedly training an unsafe behavior algorithm to obtain an unsafe behavior model by importing a video image data set of a non-navigation-stop construction unsafe behavior example collected by front-end sensing equipment from the site, and then directly loading the model to an edge computing node, so that the real-time video image from a front-end sensing layer camera is subjected to artificial intelligent analysis, the trained unsafe behavior is identified, and the alarm information of the result is sent to an alarm device of the front-end equipment to warn the unsafe behavior; and carrying out sample marking on the identified image, and providing the image for an artificial intelligence model training platform to train.
A cloud-edge-end network system is adopted in the airport non-stop construction identification warning system, and the network structure comprises a front-end sensing layer, an edge computing layer, a cloud computing layer and an intelligent application layer. The front end sensing layer is positioned at the front end or the access end of the network system and is responsible for acquiring field video image data and warning unsafe behaviors; the access terminal of the front end sensing layer comprises various cameras, warning device equipment, a signal collecting sensor and a signal controller;
the cloud computing layer is positioned at the far end of the system; the cloud computing layer is responsible for processing and storing global information, bears computing tasks which cannot be executed by the edge computing layer, and provides standard API for issuing business rules, algorithm models and development and docking of various applications to the edge computing layer.
The edge computing layer is positioned between the front-end sensing equipment and the cloud computing center; the edge computing layer is responsible for summarizing unstructured video data and internet of things data sent by the front-end sensing devices, preprocessing the unstructured video data and the internet of things data, triggering corresponding actions according to set rules, uploading processing structures and related data to the cloud, and triggering the front-end warning device to warn unsafe behavior results. The edge computing node deploys one side close to the data source according to actual needs and provides nearest-end service nearby; .
The intelligent application layer provides vertical application services such as object identification, behavior identification and safety management scenes for users by utilizing the structural data analyzed and processed and combining specific business requirements and application models, and mainly provides intelligent identification and safety warning application services of unsafe behaviors.
An edge management module, a video cloud platform module, an artificial intelligence module and an Internet of things platform module are arranged in the cloud computing center; the edge management module comprises a node management submodule, an assembly management submodule, a service management submodule and a service arrangement submodule so as to realize the resource monitoring function of edge calculation. The video cloud platform module comprises a video storage sub-module, a video distribution sub-module, a video processing sub-module and a quality diagnosis sub-module so as to realize the function of structured data of video images.
The artificial intelligence module comprises an algorithm market submodule, a model training submodule, a sample labeling submodule and a model distribution submodule so as to realize the intelligent management function of the model algorithm. The Internet of things platform module comprises an equipment management submodule, a connection management submodule, a data analysis submodule, a data management submodule, a rule engine submodule and a user management submodule so as to realize the management function of various kinds of equipment.
An edge computing network computing storage module, an operating system integrated platform module, a Docker container module, an application program module, an equipment management module and a scheduling management module are arranged in the edge computing node. The operation system integrated platform module comprises a network submodule, a calculation submodule, a storage submodule and an operation system submodule so as to realize the function of an edge calculation support platform;
the system comprises a Docker container module, a virtual machine and a control module, wherein the Docker container module is used for realizing light and modularized use of a plurality of application programs in the virtual machine; the application program module comprises n unsafe behavior identification application programs so as to realize identification of various unsafe behaviors. The device management module is used for realizing the management function of the access device. The scheduling management module is used for realizing task scheduling functions, including task scheduling of the cloud computing center and the edge computing nodes and task scheduling between the edge computing nodes and the edge computing nodes.
Example 2
With reference to fig. 1 to 4, the artificial intelligence model training platform performs model training by using an unsafe behavior model training method, and the unsafe behavior model training method includes the following steps:
a1, combing the safety standard items related to non-navigation construction for the first time, and defining n types of unsafe behavior activities:
the unsafe behaviors are characterized in that various requirements on construction safety in the current common airport operation and engineering specifications are combed according to the requirements on operation safety, air defense safety, pipeline safety and construction safety, whether the requirements in the specifications are met or not is detected in real time and efficiently by utilizing a computer vision technology, and the safety factor of non-navigation-stop construction is increased;
a2, video image acquisition and preprocessing: the method comprises the steps of carrying out noise removal, gray level conversion and geometric correction preprocessing on a received video image shot by a camera, and reducing the influence of a non-navigation construction environment on video analysis;
a3, case collection: collecting unsafe behavior cases in the preprocessed images according to the n classes of unsafe behavior activities defined in the step A1;
a4, image sample labeling: manually marking a target rectangular frame of the unsafe behavior case, performing normalization processing and protecting relevant information of the target rectangular frame, wherein the relevant information comprises an initial position, a width and a height;
a5, image sample storage: storing the marked image samples to form a training video image data set;
a6, training a model;
a7, data post-processing: carrying out some post-processing and sorting on the result obtained by the convolution network, and storing for the next step;
a8, feedback (weight adjustment): the method has the advantages that the method has the function of adjusting the weights obtained by the convolutional network, optimizes the cost function by using a gradient descent method, and adjusts the weight of each layer of convolutional network from back to front step by step, so that the generalization capability of an output model is greatly improved;
a9, model output: and after continuous feedback, model training and data processing, optimizing the scheme to obtain a final model result.
The model training in A6 is the core of the output of the unsafe behavior recognition model, and the model comprises a plurality of convolution layers and a plurality of down-sampling and nonlinear transformations; carrying out first-level and first-level abstraction on a training video image data set to finally obtain model characteristics with certain discrimination capability on some images outside training; the input video image is processed in a series of ways, the information is gradually reduced, after the input video image is processed by a convolution network, valuable information is stored, redundant information is abandoned, namely, the image characteristics are abstracted, and the interaction of a large amount of image characteristic information jointly obtains a final model.
The identification step of the unsafe behaviour algorithm comprises the following steps:
b1, model distribution;
b2, model loading: the loaded model is the weight value of each network layer obtained by training the unsafe behavior model training method, the weight value comprises millions of weight value parameters, the unsafe behavior recognition model in the whole recognition process is loaded to the edge computing node in the initial stage of the system, and the unsafe behavior recognition model is dynamically updated according to the lapse of time and the rules of the model distribution submodule;
b3, behavior recognition: after the loading is finished, a behavior recognition network layer is added so as to restrain a recognition result, the behavior recognizes the position of the marked object, and simultaneously predicts the category of the object, so that the confirmation of the credibility of the detection recognition is realized;
b4, image annotation: after the behaviors are identified, outputting the unsafe behaviors as a result of construction without stopping the navigation, and labeling the identified network image;
b5, image sample storage: the identified image and the marked image are subjected to background coding storage, the unsafe behavior identification model is dynamically updated according to the rule of the model distribution submodule, and the updated image sample is deleted at regular time;
b6, providing samples for model training: the stored image samples provide samples for unsafe behavior model training;
b7, data post-processing: the obtained image sample and the model sample can be retrieved through a standard retrieval tool, and the recorded specific event can be queried in real time;
and B8, outputting the updated model sample, and feeding the output model back to the unsafe behavior model training method.
Example 3
With reference to fig. 1 and 2, a video image-based airport non-stop construction recognition warning system is applied to a large airport in a certain city in China, and the system comprises a cloud computing center arranged in an airport logistics management and control area, a plurality of edge computing nodes arranged in a construction area and a plurality of front-end sensing devices arranged in the construction area. The front-end sensing equipment is positioned at the front end or the access end of the system, the cloud computing center is positioned at the far end of the system, and the edge computing node is positioned between the front-end sensing equipment and the cloud computing center; the front-end sensing equipment is directly connected with the edge computing node nearby through a CAN/RJ45/RS485/RS232, and the edge computing node is remotely connected with the cloud computing center through a 5G/4G/LAN/WLAN.
An artificial intelligent model training platform is arranged in the cloud computing center, the platform collects a video image data set of an unsafety behavior example of the non-navigation construction by leading in a camera of on-site front-end sensing equipment, and after an unsafety behavior algorithm is repeatedly trained to obtain an unsafety behavior model, the model is directly loaded to an edge computing node, so that the artificial intelligent analysis is carried out on a real-time video image from the camera of the front-end sensing equipment, the trained unsafety behavior is identified, and the alarm information of the result is sent to an alarm device of the front-end equipment to warn the unsafety behavior; and simultaneously, carrying out sample labeling on the identified image, and providing the labeled image for an artificial intelligence model training platform to train.
According to the technical scheme, intelligent judgment and warning of unsafe behaviors can be intelligently achieved, real-time response and real-time problem solving of intelligent analysis can be achieved, and the edge computing node is deployed at one side close to a data source to provide nearest-end service nearby.
With reference to fig. 2, a new cloud-edge-end network structure is adopted in the airport non-stop construction recognition warning system based on video images, and the system comprises a front-end sensing layer, an edge computing layer, a cloud computing layer 3 and an intelligent application layer.
Wherein, the front end sensing layer: the nerve endings of the whole system are positioned at the front end (or access end) of the system and are responsible for collecting the field video image data and warning the unsafe behavior. Besides various cameras and warning equipment, the access terminal of the front-end sensing layer also comprises various sensors, controllers and other Internet of things equipment.
A cloud computing layer: the system is located at the far end of the system and mainly comprises an edge management module, a video cloud platform, an artificial intelligence module and an internet of things platform, processes and stores global information, undertakes computing tasks (including unsafe behavior recognition model training and the like) which cannot be executed by an edge layer, and issues service rules and algorithm models to the edge layer and provides standard API for development and docking of various applications. Besides IT equipment such as various servers and networks, the cloud computing center also comprises power system equipment, air-conditioning refrigeration equipment and the like.
Edge calculation layer: the system is positioned between the front-end sensing equipment and the cloud computing center, mainly comprises a network, computing, storage and operation system integrated platform module, a Docker container module, an application program module, an equipment management module, a scheduling management module and the like, and is used for summarizing and preprocessing unstructured video data and Internet of things data sent by the front-end sensing equipment. The edge computing layer triggers corresponding actions (the actions are unsafe behavior identification) according to set rules, meanwhile, the processing structure and related data are uploaded to the cloud, and the front-end alarm equipment is triggered to alarm according to the unsafe behavior results. The edge computing node set network, computing, storing and applying integrated platform equipment is deployed at one side close to a data source according to actual needs, and nearest-end service is provided nearby.
And (3) intelligent application layer: the structured data analyzed and processed is combined with specific business requirements and application models, vertical application services such as object recognition, behavior recognition, safety management and other scenes are provided for users, and the system mainly provides intelligent recognition and safety warning application services for unsafe behaviors.
For each module in the cloud computing layer, the module composition and functions are as follows:
the edge management module comprises a node management submodule, an assembly management submodule, a service management submodule and a service arrangement submodule so as to realize the resource monitoring function of edge calculation.
The video cloud platform module comprises a video storage sub-module, a video distribution sub-module, a video processing sub-module and a quality diagnosis sub-module so as to realize the function of structured data of video images.
The artificial intelligence module comprises an algorithm market submodule, a model training submodule, a sample labeling submodule and a model distribution submodule so as to realize the intelligent management function of the model algorithm.
The Internet of things platform comprises an equipment management submodule, a connection management submodule, a data analysis submodule, a data management submodule, a rule engine submodule and a user management submodule so as to realize the management function of various kinds of equipment.
In the airport non-stop construction identification warning system based on the video images, an integrated platform module, a Docker container module, an application program module, an equipment management module, a scheduling management module and the like are arranged in the edge computing node.
The integrated platform module comprises a network submodule, a calculation submodule, a storage submodule and an operating system submodule so as to realize the function of the edge calculation support platform.
The Docker container module is used for realizing light and modularized use of a plurality of application programs in the virtual machine.
The application program module comprises n unsafe behavior identification application programs so as to realize identification of various unsafe behaviors. The device management module is used for realizing the management function of the access device.
The scheduling management module is used for realizing task scheduling functions, including task scheduling of the cloud computing center and the edge computing nodes and task scheduling between the edge computing nodes.
In the above system for intelligently identifying and warning the unsafe behaviors of the non-stop construction of the airport based on the video images, the front-end sensing equipment comprises a plurality of types, and each type comprises a plurality of types. The different types of the front-end sensing equipment are cameras, warning devices, other Internet of things equipment and the like.
Example 4
With reference to fig. 1 to fig. 3, in the airport non-stop construction recognition warning system based on video images, there are two stages for recognition warning, one of which is an unsafe behavior recognition model training stage.
Unsafe behavior model training, comprising the following:
1) firstly, carding safety standard items related to non-stop construction, defining n types of unsafe behaviors Actioni, wherein i is more than 0 and less than or equal to n.
The unsafe behaviors are characterized in that various requirements on construction safety in the current common airport operation and engineering specifications are combed according to the requirements on operation safety, air defense safety, pipeline safety and construction safety, whether the requirements in the specifications are met or not is detected in real time and efficiently by utilizing a computer vision technology, and the safety factor of non-navigation-stop construction is increased; the definition input for such unsafe behavior is open, adding or subtracting self-defined action categories as needed.
Because the number of construction elements is large, the number of construction specifications for each element is also large. The national standard and the industry standard are mainly combed, and the method comprises the following steps: the safety control system comprises a civil airport operation safety management regulation, a building construction safety inspection standard, a building machinery use safety technical regulation, a building engineering construction site electricity supply and utilization safety regulation, a building engineering construction site fire safety technical regulation, a construction site temporary building technical regulation and the like.
The field management system considers that any production site has five basic elements, namely Man (Man), Machine (Machine), Material (Material), Method (Method) and Environment (Environment), namely a 4M1E management Method. Corresponding to non-stop construction management, people refer to all personnel participating in project production, machines refer to equipment tools used in the production process, materials refer to raw materials or semi-finished products required by engineering production, methods refer to production processes or methods, and rings refer to the environment in which production is performed. Each element in a construction scene is complex and changeable, and taking two elements of people and machines as an example, hundreds of people can simultaneously carry out different operations on one construction site.
Taking the supporting project of the north freight transportation area and the flying area of the three-stage extension project at a certain airport in Zhengzhou as an example, the peak period of the non-navigation-stop construction reaches 384 persons/day; various machines are used for construction, and examples of the machine related to the earth and stone include an excavator, a bulldozer, a scraper, a road roller, and a battering ram.
Similarly, taking the supporting project of the north freight transportation area and the flying area of the three-stage extension project at a certain airport in Zhengzhou as an example, the peak time of non-navigation-stop construction reaches 147 platforms/day. The invention mainly combs: personnel management code, mechanical equipment management code, and materials management code. See tables 1, 2, 3 for details.
Table 1: non-navigation construction personnel management standard
Table 2: mechanical equipment management standard for non-stop construction
Table 3: material management standard article for non-navigation construction
2) Video image acquisition and preprocessing: and preprocessing the received video image shot by the camera such as noise removal, gray level conversion, geometric correction and the like is performed, so that the influence of the non-navigation construction environment on video analysis is reduced.
3) Case collection: according to the defined n-type unsafe behavior Actioni (included in tables 1, 2 and 3) of 1) in the embodiment, unsafe behavior cases are collected in the preprocessed image.
4) And image sample labeling: the steps are mainly to manually mark a target rectangular frame of the unsafe behavior case, to perform normalization processing and to keep the relevant information (initial position, width, height, etc.) of the target rectangular frame.
5) And storing the marked image samples to form a training video image data set Datai, wherein i is more than 0 and less than or equal to n.
6) And model training: the step is the core of outputting an unsafe behavior recognition model, the module comprises a plurality of layers of convolution layers, a plurality of down-sampling, nonlinear transformation and the like, one-level and one-level abstraction is carried out on a training video image data set Datai, and finally model characteristics (stored in a model weight mode) with certain discrimination capability on some images except for training are obtained. The input video image is subjected to a series of processing, the representative information is gradually reduced, the valuable information is stored after the convolution network processing, and some redundant information is abandoned, namely, the image characteristics are abstracted, and a robust model output Modeli is trained together through the interaction of a large amount of image characteristic information, wherein i is more than 0 and less than or equal to n.
7) And data post-processing: the submodule is also a specific link in the training process, and is mainly used for carrying out some post-processing on the result obtained by the convolutional network for the next step.
8) And feedback (weight adjustment): the step plays a role in adjusting the weight obtained by the convolutional network, the cost function is optimized by using a gradient descent method, and the weight of each layer of convolutional network is adjusted step by step from back to front, so that the generalization capability of the output model is greatly improved.
9) Model Modeli output: this step is the final output result.
Example 5
With reference to fig. 1 to 4, in the airport non-stop construction identification warning system based on video images, there are two stages of identification warning, one of which is an unsafe behavior determination stage.
The unsafe behavior identification comprises the following contents:
1) and loading a model: the unsafe behavior recognition model Modeli means that the weighted value of each network layer obtained by training in the training stage of the unsafe behavior recognition model has millions of parameters, the unsafe behavior recognition model Modeli in the whole recognition process is loaded to the edge computing node in the initial stage of the system, and dynamic updating is carried out according to the rules of the model distribution submodule.
2) And behavior recognition: the behavior recognition method is almost the same as a convolutional network in a training link, only a Detection layer is added so as to restrain a recognition result, the position of an object can be accurately marked at the same time and the category of the object can be predicted at the same time under the condition of double management of the behavior recognition, and the two layers supplement each other, so that the reliability of Detection recognition is improved.
3) And the identification result is as follows: and (4) predicting the unsafe behavior Actioni of the non-stop construction by the final output result, and on one hand, sending the identified alarm information to an alarm device of the front-end equipment to warn the unsafe behavior. And simultaneously, carrying out sample marking on the identified image, and carrying out coding storage. Therefore, the method can provide samples for unsafe behavior model training, and can also query the recorded specific events in real time through a standard retrieval tool.
It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.
Claims (9)
1. A video-based system and method for identifying unsafe behaviors of non-navigation-stop construction are characterized in that:
the system comprises a cloud computing center, a plurality of edge computing nodes and a plurality of front-end sensing devices;
the front-end sensing equipment is positioned at the front end or the access end of the system, the cloud computing center is positioned at the far end of the system, and the edge computing node is positioned between the front-end sensing equipment and the cloud computing center;
the front-end sensing equipment is directly connected with the edge computing node nearby through CAN/RJ45/RS485/RS232, and the edge computing node is remotely connected with the cloud computing center through 5G/4G/LAN/WLAN;
the front-end sensing equipment is provided with a video image acquisition module and a result warning module;
the edge computing nodes have an intelligent unsafe behavior judgment function, the cloud computing center has a deep learning function, and model training of unsafe behavior recognition is achieved;
the cloud computing center is internally provided with an artificial intelligent model training platform, the training platform directly loads the model to an edge computing node after repeatedly training an unsafe behavior model by importing a video image data set of a non-navigation-stop construction unsafe behavior example collected by front-end sensing equipment from the site, so that the real-time video image from a front-end sensing layer camera is subjected to artificial intelligent analysis, the trained unsafe behavior is identified, and the alarm information of the result is sent to an alarm device of the front-end equipment to warn the unsafe behavior; and carrying out sample marking on the identified image, and providing the image for an artificial intelligence model training platform to train.
2. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 1, wherein the system comprises: the airport non-stop construction identification warning system adopts a cloud-edge-end network system, and the network structure comprises a front-end sensing layer, an edge computing layer, a cloud computing layer and an intelligent application layer.
3. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 2, wherein: the front end sensing layer is positioned at the front end or the access end of the network system and is responsible for acquiring field video image data and warning unsafe behaviors; the access terminal of the front end sensing layer comprises various cameras, warning device equipment, a signal collecting sensor and a signal controller;
the cloud computing layer is positioned at the far end of the system; the cloud computing layer is responsible for processing and storing global information, bears computing tasks which cannot be executed by the edge computing layer, and issues service rules, algorithm models and standard APIs (application programming interfaces) for development and docking of various applications to the edge computing layer.
4. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 2, wherein: the edge computing layer is positioned between the front-end sensing equipment and the cloud computing center; the edge calculation layer is responsible for summarizing unstructured video data and Internet of things data sent by each front-end sensing device, preprocessing the unstructured video data and the Internet of things data, triggering corresponding actions according to set rules, uploading a processing structure and related data to the cloud, and triggering the front-end warning device to warn an unsafe behavior result;
the edge computing node deploys one side close to a data source according to actual needs to provide nearest-end service nearby;
the intelligent application layer provides vertical application services such as object identification, behavior identification and safety management scenes for users by utilizing structural data analyzed and processed and combining specific business requirements and application models, and mainly provides intelligent identification and safety warning application services of unsafe behaviors.
5. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 1, wherein the system comprises: the cloud computing center is internally provided with an edge management module, a video cloud platform module, an artificial intelligence module and an Internet of things platform module;
the edge management module comprises a node management submodule, an assembly management submodule, a service management submodule and a service arrangement submodule so as to realize the resource monitoring function of edge calculation;
the video cloud platform module comprises a video storage sub-module, a video distribution sub-module, a video processing sub-module and a quality diagnosis sub-module so as to realize the function of structured data of video images;
the artificial intelligence module comprises an algorithm market submodule, a model training submodule, a sample labeling submodule and a model distribution submodule so as to realize the intelligent management function of the model algorithm;
the Internet of things platform module comprises an equipment management submodule, a connection management submodule, a data analysis submodule, a data management submodule, a rule engine submodule and a user management submodule so as to realize the management function of various kinds of equipment.
6. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 1, wherein the system comprises: the edge computing node is internally provided with an edge computing network computing storage module, an operating system integrated platform module, a Docker container module, an application program module, an equipment management module and a scheduling management module;
the operating system integrated platform module comprises a network submodule, a calculation submodule, a storage submodule and an operating system submodule so as to realize the function of an edge calculation support platform;
the Docker container module is used for realizing light and modularized use of a plurality of application programs in the virtual machine;
the application program module comprises n unsafe behavior identification application programs so as to realize identification of various unsafe behaviors. The device management module is used for realizing the management function of the access device;
the scheduling management module is used for realizing task scheduling functions, including task scheduling of the cloud computing center and the edge computing nodes and task scheduling between the edge computing nodes and the edge computing nodes.
7. The system and method for identifying unsafe behaviors of non-navigation-stop construction based on video according to claim 1, wherein the system comprises: the artificial intelligence model training platform carries out model training through an unsafe behavior model training method, and the unsafe behavior model training method comprises the following steps:
a1, combing the safety standard items related to non-navigation construction for the first time, and defining n types of unsafe behavior activities:
the unsafe behaviors are characterized in that various requirements on construction safety in the current common airport operation and engineering specifications are combed according to the requirements on operation safety, air defense safety, pipeline safety and construction safety, whether the requirements in the specifications are met or not is detected in real time and efficiently by utilizing a computer vision technology, and the safety factor of non-navigation-stop construction is increased;
a2, video image acquisition and preprocessing: the method comprises the steps of carrying out noise removal, gray level conversion and geometric correction preprocessing on a received video image shot by a camera, and reducing the influence of a non-navigation construction environment on video analysis;
a3, case collection: collecting unsafe behavior cases in the preprocessed images according to the n classes of unsafe behavior activities defined in the step A1;
a4, image sample labeling: manually marking a target rectangular frame of the unsafe behavior case, performing normalization processing and protecting relevant information of the target rectangular frame, wherein the relevant information comprises an initial position, a width and a height;
a5, image sample storage: storing the marked image samples to form a training video image data set;
a6, training a model;
a7, data post-processing: carrying out some post-processing and sorting on the result obtained by the convolution network, and storing for the next step;
a8, feedback (weight adjustment): the method has the advantages that the method has the function of adjusting the weights obtained by the convolutional network, optimizes the cost function by using a gradient descent method, and adjusts the weight of each layer of convolutional network from back to front step by step, so that the generalization capability of an output model is greatly improved;
a9, model output: and after continuous feedback, model training and data processing, optimizing the scheme to obtain a final model result.
8. The system and method for unsafe behavior identification of non-navigation construction based on video according to claim 7, wherein: the model training in A6 is the core of the output of the unsafe behavior recognition model, and the model comprises a plurality of convolution layers and a plurality of down-sampling and nonlinear transformations; performing first-level and first-level abstraction on a training video image data set to finally obtain model characteristics with certain discrimination capability on images except for training; after a series of processing is carried out on an input video image, information is gradually reduced, valuable information is stored after the processing of a convolutional network, redundant information is abandoned, namely image characteristics are abstracted, and a final model is obtained through interaction of a large amount of image characteristic information.
9. The system and method for unsafe behavior identification of non-navigation construction based on video according to claim 8, wherein: the identification step of the unsafe behavior algorithm comprises the following steps:
b1, model distribution;
b2, model loading: the loaded model is the weighted value of each network layer obtained by training the unsafe behavior model training method, the weighted value comprises millions of weighted value parameters, the unsafe behavior recognition model in the whole recognition process is loaded to the edge computing node in the initial stage of the system, and the unsafe behavior recognition model is dynamically updated according to the lapse of time and the rules of the model distribution submodule;
b3, behavior recognition: after the loading is finished, a behavior recognition network layer is added so as to restrain a recognition result, the behavior recognizes the position of the marked object, and simultaneously predicts the category of the object, so that the confirmation of the credibility of the detection recognition is realized;
b4, image annotation: after the behaviors are identified, outputting the unsafe behaviors as a result of construction without stopping the navigation, and labeling the identified network image;
b5, image sample storage: carrying out background coding storage on the identified image and the labeled image, carrying out dynamic updating value of the unsafe behavior identification model according to the rule of the model distribution submodule, and deleting the updated image sample at regular time;
b6, providing samples for model training: the stored image samples provide samples for unsafe behavior model training;
b7, data post-processing: the obtained image sample and the model sample can be retrieved through a standard retrieval tool, and the recorded specific event can be queried in real time;
and B8, outputting the updated model sample, and feeding the output model back to the unsafe behavior model training method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110658501.9A CN113392760A (en) | 2021-06-15 | 2021-06-15 | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110658501.9A CN113392760A (en) | 2021-06-15 | 2021-06-15 | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113392760A true CN113392760A (en) | 2021-09-14 |
Family
ID=77620907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110658501.9A Pending CN113392760A (en) | 2021-06-15 | 2021-06-15 | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113392760A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114531439A (en) * | 2021-11-29 | 2022-05-24 | 杭州安脉盛智能技术有限公司 | Instrument data cloud edge cooperative acquisition and processing system and method based on image recognition |
| CN114743407A (en) * | 2022-03-10 | 2022-07-12 | 北京首都国际机场股份有限公司 | Runway taxiway navigation stop management method, system, electronic device and medium |
| CN114785792A (en) * | 2022-06-17 | 2022-07-22 | 东云睿连(武汉)计算技术有限公司 | Cloud edge collaborative video two-way analysis device and method |
| CN116578323A (en) * | 2023-07-14 | 2023-08-11 | 湖南睿图智能科技有限公司 | Deep learning algorithm iteration method based on Yun Bian cooperation |
| CN117540883A (en) * | 2024-01-10 | 2024-02-09 | 山东鲁轻安全评价技术有限公司 | AI-based security risk identification analysis system and method |
| CN117649594A (en) * | 2024-01-30 | 2024-03-05 | 深圳市震有智联科技有限公司 | Edge fusion all-in-one machine based on edge calculation and identification method thereof |
-
2021
- 2021-06-15 CN CN202110658501.9A patent/CN113392760A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114531439A (en) * | 2021-11-29 | 2022-05-24 | 杭州安脉盛智能技术有限公司 | Instrument data cloud edge cooperative acquisition and processing system and method based on image recognition |
| CN114743407A (en) * | 2022-03-10 | 2022-07-12 | 北京首都国际机场股份有限公司 | Runway taxiway navigation stop management method, system, electronic device and medium |
| CN114785792A (en) * | 2022-06-17 | 2022-07-22 | 东云睿连(武汉)计算技术有限公司 | Cloud edge collaborative video two-way analysis device and method |
| CN114785792B (en) * | 2022-06-17 | 2022-09-16 | 东云睿连(武汉)计算技术有限公司 | Cloud-edge collaborative video double-path analysis device and method |
| CN116578323A (en) * | 2023-07-14 | 2023-08-11 | 湖南睿图智能科技有限公司 | Deep learning algorithm iteration method based on Yun Bian cooperation |
| CN117540883A (en) * | 2024-01-10 | 2024-02-09 | 山东鲁轻安全评价技术有限公司 | AI-based security risk identification analysis system and method |
| CN117540883B (en) * | 2024-01-10 | 2024-04-09 | 山东鲁轻安全评价技术有限公司 | AI-based security risk identification analysis system and method |
| CN117649594A (en) * | 2024-01-30 | 2024-03-05 | 深圳市震有智联科技有限公司 | Edge fusion all-in-one machine based on edge calculation and identification method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113392760A (en) | Video-based system and method for identifying unsafe behaviors of non-navigation-stop construction | |
| CN104555627B (en) | Elevator Internet of Things operation Control management system and operation management method thereof | |
| CN106447107B (en) | Maintenance method based on aircraft structure health monitoring | |
| CN106628948B (en) | The method, apparatus and system of coal mine leather belt machine speed regulation | |
| CN112785458A (en) | Intelligent management and maintenance system for bridge health big data | |
| CN115097788A (en) | Intelligent management and control platform based on digital twin factory | |
| CN109559008A (en) | Construction monitoring method, apparatus and system | |
| CN102880802A (en) | Fatal danger fountainhead analysis and evaluation method for safety production cloud service platform system facing industrial and mining enterprises | |
| CN108985714A (en) | A kind of construction site wisdom control integrated system | |
| CN102929827A (en) | Wireless sensor data acquisition cluster for industrial-and-mining-enterprise-oriented safety production cloud service platform | |
| CN102930372A (en) | Data analysis method for association rule of cloud service platform system orienting to safe production of industrial and mining enterprises | |
| CN114297935A (en) | Airport terminal building departure optimization operation simulation system and method based on digital twin | |
| CN115456343B (en) | Intelligent airport evaluation index system construction and evaluation method | |
| CN111814687A (en) | Flight support node intelligent identification system | |
| CN111056258B (en) | Method and device for intelligently adjusting conveyor belt | |
| CN113627784A (en) | Enterprise asset management intelligent decision-making system based on industrial internet | |
| CN114841660A (en) | Enterprise intelligent safety management and control cloud platform based on field information | |
| CN114565282A (en) | Intelligent city management system based on unmanned patrol and implementation method | |
| CN114357795A (en) | Digital twinning system for full-automatic container wharf loading and unloading along shore | |
| CN111160537B (en) | Crossing traffic police force resource scheduling system based on ANN | |
| CN116468287A (en) | Intelligent park control system based on digital twinning | |
| CN102915482A (en) | Safety production process control and management method for cloud service platforms of industrial and mining enterprises | |
| CN102903009A (en) | Malfunction diagnosis method based on generalized rule reasoning and used for safety production cloud service platform facing industrial and mining enterprises | |
| Statsenko et al. | Developing software and hardware for automation of ground urban transport traffic management | |
| CN112258367A (en) | Monitoring processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |