CN118760996A - Smart yard multi-level safety control device with no blind spots - Google Patents

Abstract

The invention discloses a smart yard non-blind area multi-level safety management and control device, which comprises: the storage yard all-element non-blind area sensing module based on cloud edge cooperation comprises a sensing module with multi-source sensor cooperation and a storage yard multi-element non-blind area sensing Bian Yun cooperation module; the storage yard historical accident information analysis module based on data mining comprises a processing module of accident-oriented video data and a storage yard accident classification analysis module based on a social network analysis method; the storage yard safety management and control module based on the multi-level strategy comprises a multi-element real-time planning management and control module based on production safety constraint, an advanced early warning module integrating multi-production safety state identification and a storage yard emergency quick response and emergency processing module. The invention can improve the safety and efficiency of the traditional storage yard, reduce the risk of accidents and ensure the safety of operators and equipment.

Description

Smart yard multi-level safety control device with no blind spots

Technical Field

The present invention relates to dry bulk material yard safety technology, and in particular to a multi-level safety management and control device for a smart yard without blind spots.

Background Art

In the dry bulk material yard environment, due to frequent operations, the status of the material pile is updated very quickly, and the position changes frequently. When the stacker-reclaimer cuts and reclaims the dry bulk material pile (such as coal pile), the collapse caused by the material will also change the size and shape of the pile. The operating environment of the dry bulk cargo yard is complex. In the process of vehicles, workers, and mobile machinery working together, there are easy to be blind spots and human operating errors, resulting in personal injury to operators and damage to operating equipment.

There have been many systematic studies on the management and control of coal mines and hazardous chemicals, and great efforts have been invested, and initial results have been achieved. There has been insufficient research on dry bulk material yards, and a systematic system has not yet been formed. In view of the current situation where there are many dry bulk material yards, few studies, and frequent accidents, it is of great significance to build a dry bulk cargo safety production accident prevention and control service platform by drawing on the safety management and control technology of coal mines and hazardous chemicals to ensure personal safety and property safety.

Traditional storage yards have the following technical problems that need to be solved, including:

1) Blind spot detection problem: Traditional yard operation vehicles have blind spots, making it difficult to achieve all-round detection. This leads to safety hazards during vehicle driving and operation, and is prone to accidents. To solve this problem, all-round detection technology is needed to eliminate blind spots and improve operation safety.

2) Accident information mining problem: The accident data of traditional storage yards is huge, and manual review and analysis is time-consuming and error-prone. In order to better understand the causes and influencing factors of accidents, it is necessary to use video automatic processing and information extraction technology to analyze and mine historical accident videos and related data to discover the laws and potential risk factors of accidents, and provide a basis for accident prevention and safety management.

3) Safety management and control strategy issues: Traditional yard safety management has certain limitations and lacks a multi-level safety management and control strategy. In order to improve the safety of the yard, it is necessary to formulate a multi-level safety management and control strategy, including pre-prevention, in-process warning and post-processing. This includes the division of dangerous areas, route planning, the establishment of early warning systems, the formulation of accident templates, etc., to comprehensively manage and control the safety risks of the yard.

Solving these technical problems can improve the safety and efficiency of traditional storage yards, reduce the risk of accidents, and ensure the safety of operators and equipment.

Summary of the invention

The technical problem to be solved by the present invention is to provide a multi-level safety management and control device for a smart storage yard without blind spots in view of the defects in the prior art.

The technical solution adopted by the present invention to solve the technical problem is: a multi-level safety control device for a smart storage yard without blind spots, comprising:

The full-factor blind-spot perception module of the storage yard based on cloud-edge collaboration includes a multi-source sensor collaborative perception module and a multi-factor blind-spot perception edge-cloud collaboration module of the storage yard;

The yard multi-factor non-blind-zone perception edge-cloud collaboration module includes a yard perception device, a yard edge node and a yard cloud server;

The multi-source sensor collaborative perception module is used to perform environmental detection and target recognition using a multi-factor blind-spot-free perception edge-cloud collaborative module for a storage yard based on multi-source sensor collaboration, thereby achieving multi-factor blind-spot-free perception of the storage yard;

A data mining-based storage yard historical accident information analysis module, including an accident-oriented video data processing module and a storage yard accident classification analysis module based on social network analysis;

The accident-oriented video data processing module is used to identify abnormal events based on the video data in the dry bulk material yard;

The storage yard accident classification and analysis module based on social network analysis is used to classify historical storage yard accidents and analyze the causes of historical storage yard accidents;

The yard safety management and control module based on multi-level strategies includes a multi-factor real-time planning and control module based on production safety constraints, an advanced warning module integrating multiple production safety status identification, and a rapid response and emergency handling module for sudden accidents in the yard;

The multi-factor real-time planning and control module based on production safety constraints is used to divide the dangerous areas for vehicle and personnel movement according to the relevant technical personnel of the yard, establish a 2D grid map with dangerous area identification in combination with the perception system, and use the multi-objective path planning algorithm based on the improved A* algorithm to perform multi-objective path planning in the yard;

The advanced warning module integrating multiple production safety status identification is used to monitor the identification of abnormal events in real time. If it is judged to be a dangerous state, it will promptly alarm and take corresponding measures;

The rapid response and emergency handling module for sudden accidents in the storage yard divides the accident types according to the accident template, and performs corresponding emergency handling according to the results of the storage yard accident classification analysis module according to the accident type, so as to effectively deal with different types of accidents.

According to the above scheme, the perception module of the multi-source sensors performs environmental detection and target recognition;

The steps used are as follows:

1) By deploying a variety of sensors, comprehensive detection of various elements in the yard is carried out to achieve full-factor perception of the yard;

2) Various visual sensors are deployed in the yard for calibration and synchronization;

3) Fuse the information from different sensors to complete target detection.

According to the above scheme, in step 1), various elements in the storage yard are comprehensively detected by deploying various sensors to achieve full-element perception of the storage yard; including:

1.1) Active cameras, infrared cameras, video cameras, and laser radar equipment are integrated to collect data on the yard environment;

1.2) Install a facial recognition system and electronic fence at the entrance and exit to manage and record the entry and exit of personnel;

1.3) Deploy visual sensors in every corner of the entire yard to achieve real-time monitoring of the yard scene; add LiDAR-assisted perception in the stockpile area to achieve real-time modeling of the stockpile, monitor the number of stacked items, stacking height, type and status of goods;

1.4) An infrared camera is placed in the corner above the stockpile area, with the field of view facing the stockpile area, to achieve real-time detection of the heat distribution of objects in the stockpile yard;

1.5) Place a PTC active camera in the middle of the work area to identify objects, people or vehicles that appear in the picture and track them;

1.6) The overhead crane is used for real-time detection to sense and locate objects around the operating vehicle and the vehicle itself, providing a perception basis for vehicle collision avoidance and route planning. A wheel speed meter is installed above the overhead crane connected to the grab bucket to detect the speed of the hook.

According to the above scheme, in step 2), multiple visual sensors are deployed in the yard for calibration and synchronization; the details are as follows:

By moving the calibration vehicle and placing a checkerboard on the roof, sensors with the same field of view are jointly calibrated; when jointly calibrating sensors with the same field of view, the overlapping field of view is enhanced;

2.1) According to the principle of visual camera imaging, calculate the corner coordinates of the camera coordinate system of the mobile calibration vehicle and obtain the direction vector of the chessboard edge For the laser radar on the mobile calibration vehicle, within a scanning cycle T, the candidate point set of the calibration plate plane is obtained Then, for the candidate points Obtain local smooth features, expressed as:

If ss>T ₀ , the laser radar scanning beam suddenly changes here, and the point cloud Add smooth feature point set Construct the edge point set of the calibration plate; T ₀ is the set threshold;

2.2) Assume that the plane normal vectors in the yard camera coordinate system and the laser radar coordinate system are and The edge point feature vectors in the camera coordinate system and the lidar coordinate system are and The objective function H is established using the geometric feature constraints of the calibration plate:

in, is the rotation matrix from the visual camera coordinate system to the lidar coordinate system; is the translation matrix from the visual camera coordinate system to the lidar coordinate system; Represents surface feature constraints, represents line feature constraints, Represents corner feature constraint;

The closed-form solution of the rotation matrix is obtained by jointly solving the three geometric feature constraints, and the translation vector is obtained by substituting it into the constraint equation, thereby completing the external parameter calibration between different sensors.

2.3) By jointly calibrating the external parameters of sensors with the same field of view, the joint calibration of multi-source heterogeneous sensors in a large area of the yard can be achieved.

According to the above scheme, in step 3), the information of different sensors is fused to complete the target detection; the details are as follows:

3.1) Encode the data collected by the visual sensor in the yard. The encoding method is: perform local sequential color information (RGB) smear encoding on the point cloud within the cone of vision formed by the 2D detection frame, and convert the original point cloud information (x, y, z, r) into the code (x, y, z, r, S, R, G, B), where x, y, z of the point cloud are spatial position information, r is the intensity value information of the point cloud, S is the recommended channel, and R, G, B are the color channels corresponding to the point cloud;

3.2) After uniform transformation of the point cloud, project it onto the 2D image:

In the formula, matrix _C1 is the camera intrinsic parameter matrix, _C2 is the extrinsic parameter homogeneous transformation matrix, which is obtained through the previous multi-sensor joint calibration; s is the recommended channel, _XL is the information of the point cloud in the lidar coordinate system, and the mapping relationship between the point cloud P( _xL , _yL , _zL ) in the lidar coordinate system and its position p(u, v) in the plane image coordinate system is obtained by projection.

3.3) Object detection;

Convert the point cloud into a sparse pseudo image in the form of a vertical “column”, and then use a 2D convolutional network to detect the pseudo image and predict the 3D detection box;

The input point cloud is processed using a 2D convolutional network to obtain a 3D bounding box for multi-factor target recognition. The 3D bounding box is defined by parameters (x, y, z, w, l, h, θ), where x, y, z are the coordinates of the center of the bounding box, w, l, h are the width, length, and height of the bounding box, respectively, and θ is the target orientation.

In the convolutional network, the regression loss between the target and the anchor box is:

Among them, the subscript gt represents the value in the true box, the subscript a represents the value in the predicted box, the regression loss is used to identify the position, size and direction of the yard target, and then the classification loss is used to give the category of the target.

According to the above scheme, the accident-oriented video data processing module identifies abnormal events based on the video data in the dry bulk material yard; the method used is as follows:

1) Identify abnormal blocks of video data in the yard;

The accident video data in the dry bulk material yard is segmented into short-time video segments. The original event video sequence of the yard is preprocessed. The video image frames of the yard are segmented into multiple two-dimensional image units using a sliding window. The adjacent two-dimensional image units of consecutive T frames are stacked to form a three-dimensional space-time cube as the yard accident sampling block and the corresponding feature information is extracted. The abnormal blocks of the video data in the yard are identified through the PCANet network:

Among them, I is a unit vector with the same dimension and size as the yard accident sampling block x _t,s ; is the processed storage yard accident sampling block; max _1≤t≤T (G _t ) is the maximum gradient value of the current gradient storage yard accident feature cube in the time and space dimensions; min _1≤tT (G _t ) is the minimum gradient value in the time and space dimensions;

2) Detect and identify abnormal events in the yard based on the identified abnormal blocks;

2.1) Solve using PCA algorithm The first L ₁ largest features of the covariance matrix to the corresponding feature vector are used as PCA filters, where L ₁ corresponds to the number of required filters; for each yard accident gradient unit G _t , the first layer outputs L ₁ convolution feature maps, and the second layer uses convolution filters to generate L ₂ feature maps for each feature map, and then calculates the standard deviation of the histogram feature as the apparent abnormality score of the yard block feature:

Among them, s _app (i,j) is the abnormal score of the yard performance, and v*i,j+(δ) represents the height value corresponding to the δth interval of the histogram feature:

2.2) Sum the optical flow values I _p of all pixels in the yard block to obtain N _f is the number of pixels in the yard block;

2.3) The motion anomaly score and the appearance anomaly score are fused into _scon = _αsmot + β(1- _sapp ), and an anomaly threshold is set. The fusion of the yard anomaly score is compared with the threshold to achieve detection and discrimination of abnormal events in the yard;

According to the difference in score types, the category of abnormal accidents in the yard can be determined, such as stockpile collapse, abnormal movement patterns, operating activities, etc., so as to identify and recognize the abnormal types of abnormal video segments of the yard.

According to the above scheme, the yard accident classification analysis module based on social network analysis method classifies historical yard accidents and analyzes the causes of historical yard accidents; the method used is as follows:

The causal factors of the accidents in the storage yard are represented as nodes, and the causal relationship between the network nodes constitutes a semantic network of dry bulk material storage yard safety accidents, and the semantic network of storage yard safety accidents is established;

The causal factors of the yard accident, _Xi , include: human factors, equipment failure, environmental factors, yard layout, and operation management;

The above yard causal factors are defined as nodes of the semantic network of dry bulk material yard safety accidents. The relationship between yard accident causal factors is defined as edges. When two representative causal factors appear together in a yard accident, there is a certain correlation. The more common frequencies there are, the closer the relationship is. By collecting data and counting the number of dry bulk material yard accidents, it is assumed that there are K accidents in total, and (X _i , X _j ) represents that factor i and factor j appear together once.

The yard safety accident semantic network is evaluated based on the betweenness centrality of the yard safety accident semantic network, the proximity centrality of the yard safety accident semantic network and the characteristic vector centrality of the yard safety accident semantic network, which are calculated as follows:

(1) The intermediate centrality of the semantic network of storage yard safety accidents

The ratio of the number of shortest paths passing through the yard-causing node i to the total number of paths is the betweenness centrality of the yard-causing node i. Assuming that _Pjk is the number of shortcuts between the yard-causing node j and the yard-causing node k, and _Pjk (i) is the number of shortcuts between the two yard-causing nodes that include the yard-causing node i, then:

(2) Closeness centrality of the semantic network of storage yard safety accidents

The sum of the shortcut distances of other yard-causing nodes connected to the yard-causing node i is the proximity centrality of the yard-causing node i; define d(i,j) as the shortest path distance between i and j, then:

(3) Centrality of the semantic network feature vector of storage yard safety accidents

The number of nodes connected around a yard cause node affects the status and importance of the yard cause node, and is also affected by the importance of these connected yard cause nodes, which can be expressed as:

Where c is a proportional constant, a _ij = 1 if and only if i is connected to j, otherwise it is 0.

By establishing a network diagram, the more yard cause edges a yard cause node has, the greater its role is. This factor has a greater role in the entire yard safety accident semantic network diagram, and the stronger its ability to influence other yard cause nodes is. Frequent item sets and strong association rules between accident causes are mined from the data. By calculating the intermediate centrality of the yard accident network, the proximity centrality of the yard accident network, and the characteristic vector centrality of the yard accident network, a yard accident network with a higher centrality means that it is on multiple shortest paths of other yard cause nodes, a yard accident network with a higher proximity centrality means that it is closer to all other yard cause nodes, and a yard accident network with a higher characteristic vector centrality means that the importance of the yard cause nodes connected to it is high. This is used to classify and analyze the causes of yard accidents.

According to the above scheme, the multi-factor real-time planning and control module based on production safety constraints adopts a multi-objective path planning algorithm based on the improved A* algorithm to perform multi-objective path planning in the yard; the steps are as follows:

The multi-objective path planning of the yard is expressed as: G = (N, c), G represents the entire yard search space, N represents the yard target node set, and several target point sets f(S,G _i ) represents the estimated value of the starting point S relative to the target point G _i (i＝1,2,3,...,n) calculated by the heuristic evaluation function; when a yard accident occurs, the weight coefficient k _i is set by weighing the priorities between different targets to adjust the search order;

1) Create a global Open and Close list for the multi-objective path planning of the yard, create the starting search node S of the yard goal, use the public list Goals to store the yard planning goal nodes, and establish an Open and Close list for each yard planning goal point, recorded as G _op (i) and G _cl (i);

2) Put the starting point S of the yard target into G _op (i), expand the nodes adjacent to point S into G _op (i), calculate the estimated value of the starting point corresponding to the target point in each G _op (i), that is, calculate f(S, _Gi ) respectively according to the heuristic evaluation function, and at the same time, sort the calculated estimated values in ascending order in each G _op (i), take the first value in the list and put it into the global Open list, then sort the data in the global Open list according to the weight coefficient k _i , take the yard target node with the smallest estimated value as the starting point of the next step, put the original starting point into the global Close list, and clear each G _op (i) and G _cl (i) list;

3) After reaching a target point, execute 2) repeatedly to guide to a target point, set it as _Ga (1≤a≤n), then delete _Ga from the Goals list, and delete the _Gop (i) and _Gcl (i) lists at the same time; the remaining yard target points _Gop (i) and _Gcl (i) lists continue to participate in the next step;

4) Termination condition: determine whether the Open list is empty or the Goals list is empty, that is, all the target paths to be planned for the storage yard have been allocated.

According to the above scheme, the advanced warning module integrating multiple production safety status identification is used to monitor the identification of abnormal events in real time. If it is judged to be a dangerous state, it will promptly alarm and take corresponding measures;

The collaborative detection module is used to monitor the recognition results of abnormal events in real time to identify unsafe stacking methods, abnormal sounds and abnormal movements in the yard. If the sensor threshold in the recognition of abnormal events exceeds the safety threshold, an alarm signal is issued; if the sensor threshold in the recognition of abnormal events exceeds the danger alarm threshold, the corresponding equipment is stopped and controlled.

According to the above scheme, the rapid response and emergency handling module for sudden accidents in the storage yard divides the accident types according to the accident template, and performs corresponding emergency handling according to the results of the storage yard accident classification analysis module according to the accident type, so as to effectively deal with different types of accidents.

Obtain basic data from previous accident cases and post-incident conditions at the storage yard, and perform structured processing on the data to build a structured storage yard emergency response template generation model;

Determine the current accident type, match it to the processing template for that accident type, and obtain the emergency processing decision;

Construct a structured yard emergency treatment template generation model. The specific process is as follows:

The scenarios of the yard emergency are divided into s _k *(x ₁ ,x ₂ ,…,x _n ),(y ₁ ,y ₂ ,…,y _n )+, where x _n is the environmental attribute of the yard sub-event, including the temperature, pressure, relative humidity and wind speed of the yard emergency; is the state attribute of the yard sub-event;

A yard emergency event is composed of multiple yard sub-events, and similarity calculation is performed from two levels: yard sub-events and yard sub-event sets;

The similarity of the emergency level of the storage yard is:

The similarity of the number of sub-events in the yard sub-event set is: The value range is [0, 1];

Similarity of sub-event sets of two sudden events at the storage yard is the overall similarity of the set of yard sub-event names, ω ₁ , ω ₂ , ω ₃ are the weights assigned to each similarity;

The similarity of the handling tasks and emergency actions of the yard emergency events is calculated to obtain the similarity of the pending tasks and emergency actions of the yard sub-events. Adjusting the values of weights ω ₁ , ω ₂ , ω ₃ can adjust the similarity of emergency actions for storage yard accidents;

Multiple yard event sets are compared in the yard emergency management case library according to similarity sorting to obtain a group of similar yard sub-events, from which relationships similar to yard accident emergency response tasks and actions are extracted to obtain emergency response decisions; and the decisions are checked during the accident handling process to ensure the implementation of emergency response decisions.

The beneficial effects produced by the present invention are:

1) The present invention configures a variety of sensors based on multi-sensor collaboration, designs target recognition and detection algorithms, and realizes efficient and all-round detection of the yard environment to eliminate blind spots.

2) The present invention realizes classification analysis of storage yard accidents, mining of association rules between accident influencing factors and prediction of abnormal events by analyzing accident videos and related data;

3) The present invention proposes a multi-level safety management and control strategy; improves safety prevention through dangerous area division and route planning; uses early warning systems to identify unsafe conditions and unsafe behaviors, and establishes accident templates to achieve rapid emergency response and processing. Through the management and control of the entire process, a multi-level safety management and control strategy for the prevention, early warning and processing of storage yard accidents is implemented.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

FIG1 is a schematic structural diagram of an embodiment of the present invention;

FIG2 is an architecture diagram of the edge-cloud collaboration module according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

As shown in Figure 1, a multi-level safety control device for a smart storage yard without blind spots includes:

The full-factor blind-spot perception module of the storage yard based on cloud-edge collaboration includes a multi-source sensor collaborative perception module and a multi-factor blind-spot perception edge-cloud collaboration module of the storage yard;

The multi-source sensor collaborative perception module is used to perform environmental detection and target recognition using the multi-factor blind-spot-free perception edge-cloud collaborative module based on the multi-source sensor collaboration, thus achieving multi-factor blind-spot-free perception of the storage yard;

The details are as follows:

1) Through the deployment of multiple sensors, comprehensive detection of various elements in the yard is carried out to achieve full perception of the yard; specifically, it includes:

1.1) Active cameras, infrared cameras, video cameras, laser radars and other equipment are integrated to monitor the yard environment;

1.2) Install a facial recognition system and electronic fence at the entrance and exit to manage and record the entry and exit of personnel;

1.3) Deploy visual sensors in every corner of the entire storage yard to achieve real-time monitoring of the storage yard scene;

Add LiDAR-assisted sensing in the stockpile area. Three LiDARs are deployed in the stockpile area and placed on the three walls of the stockpile area to achieve real-time modeling of the stockpile to monitor the number of stacked items, stacking height, type and status of goods.

1.4) An infrared camera is placed in the corner above the stockpile area, with the field of view facing the stockpile area, to achieve real-time detection of the heat distribution of objects in the stockpile, which helps to detect abnormal heat conditions;

1.5) Place a PTC active camera in the middle of the work area to automatically identify and track objects, people or vehicles that appear in the picture, and capture abnormal situations.

1.6) Use the overhead crane to sense and locate objects around the operating vehicle and the vehicle itself, providing a perception basis for vehicle collision avoidance and route planning. The real-time detection system of the overhead crane is to deploy visual sensors on the four corners of the overhead crane, install a laser radar in the center of the overhead crane, and install a wheel speed meter above the overhead crane connected to the grab bucket to detect the speed of the hook to prevent dangerous situations such as the hook hitting the top.

The above steps can deploy a variety of sensors to conduct comprehensive detection of various elements in the yard and realize full-factor perception of the yard.

2) Calibrate and synchronize various visual sensor devices deployed in the yard;

By moving the calibration vehicle and placing a checkerboard on the roof, sensors with the same field of view are jointly calibrated; when jointly calibrating sensors with the same field of view, the overlapping field of view is enhanced;

If the overlapping field of view of the sensors is small, it will lead to false detection and missed detection of the chessboard, and the camera and radar will not be able to associate their poses, resulting in low external parameters. Therefore, auxiliary cameras are introduced to increase the number of cameras to enhance the overlapping field of view;

2.1) According to the principle of visual camera imaging, calculate the corner coordinates of the camera coordinate system of the mobile calibration vehicle and obtain the direction vector of the chessboard edge For the laser radar on the mobile calibration vehicle, within a scanning cycle T, the candidate point set of the calibration plate plane on the calibration vehicle is obtained Then, for the candidate points Obtain local smooth features, expressed as:

If ss>T ₀ , the laser radar scanning beam suddenly changes here, and the point cloud Add smooth feature point set Construct the edge point set of the calibration plate; T ₀ is the set threshold;

2.2) Assume that the plane normal vectors in the yard camera coordinate system and the laser radar coordinate system are and The edge point feature vectors in the camera coordinate system and the lidar coordinate system are and The objective function H is established using the geometric feature constraints of the calibration plate:

in, is the rotation matrix from the visual camera coordinate system to the lidar coordinate system; is the translation matrix from the visual camera coordinate system to the lidar coordinate system; Represents surface feature constraints, represents line feature constraints, Represents corner feature constraint;

Based on the joint solution of the three geometric feature constraints, the closed-form solution of the rotation matrix is obtained, and the point-line-surface constraint equation is brought into play to minimize the objective function H, and the translation vector is obtained, thereby completing the external parameter calibration between different sensors.

2.3) By jointly calibrating the external parameters of sensors with the same field of view, the joint calibration of multi-source heterogeneous sensors in a large area of the yard can be achieved.

3) Fuse information from different sensors to perform target detection;

3.1) First, encode the data collected by the visual sensor in the yard. The encoding method is: perform local sequential color information (RGB) smear encoding on the point cloud within the cone of vision formed by the 2D detection box, and convert the original point cloud information (x, y, z, r) into the code (x, y, z, r, S, R, G, B), where x, y, z of the point cloud are spatial position information, r is the intensity value information of the point cloud, S is the recommended channel, and R, G, B are the color channels corresponding to the point cloud;

3.2) After uniform transformation of the point cloud, project it onto the 2D image:

Wherein, matrix _C1 is the camera intrinsic parameter matrix, _C2 is the extrinsic parameter homogeneous transformation matrix, which is obtained through the previous multi-sensor joint calibration; s is the recommended channel, and _XL is the information of the point cloud in the laser radar coordinate system;

The mapping relationship between the point cloud P (x _L , y _L , z _L ) in the laser radar coordinate system and the position p (u, v) in the plane image coordinate system is obtained by projection.

3.3) Object detection;

Convert the point cloud into a sparse pseudo image in the form of a vertical “column”, and then use a 2D convolutional network to detect the pseudo image and predict the 3D detection box;

The input point cloud is processed using a 2D convolutional network to obtain a 3D bounding box for multi-factor target recognition. The 3D bounding box is defined by parameters (x, y, z, w, l, h, θ), where x, y, z are the coordinates of the center of the bounding box, w, l, h are the width, length, and height of the bounding box, respectively, and θ is the target orientation.

In the convolutional network, the regression loss between the target and the anchor box is:

Among them, the subscript gt represents the value in the real box, and the subscript a represents the value in the predicted box. The position, size and direction of the identified yard target are given by the regression loss, and then the category of the target is given by the classification loss;

The multi-source sensor collaborative perception system platform can manage and configure various types of equipment in the yard, monitor the various environmental components in the yard in real time, and complete tasks such as identifying and supervising all elements in the yard.

The yard multi-factor non-blind-spot perception edge-cloud collaborative module includes a yard perception device, a yard edge node and a yard cloud server; as shown in Figure 2;

(1) Yard sensing device: performs the task of collecting images, point clouds and other data in the yard and uploads them to the edge node N ₀ connected to it. Cameras and lidar collect perception information of detection targets such as personnel and vehicles. Active cameras and video cameras can also collect information such as stacks and items, and infrared cameras can collect information such as the heat distribution of objects in the yard.

(2) Yard edge nodes: Each edge node is connected through a network and is independent of each other when performing computing tasks. The yard camera edge node removes similar frames from the collected video images, and the laser radar edge node completes preliminary tasks such as ground removal and feature extraction. Then the edge node _N0 uploads the extracted features to the cloud server. The cloud server uses these features to train the model. The yard edge nodes _N1 ~ _Nm receive the model sent by the cloud server. Then the yard edge nodes _N1 ~ _Nm detect and identify the yard elements. After obtaining the recognition results, they are transmitted to the yard cloud server for fusion. Due to the characteristics of the special elements of the yard and the urgency of abnormal situations such as fire and large temperature changes, the infrared camera edge node completes tasks such as target detection, abnormality recognition and early warning decision-making at the edge, realizing the process from abnormality recognition to rapid decision-making, saving time and maximizing the protection of the safety of yard personnel and vehicles.

(3) The yard cloud server: receives the features uploaded by the edge node N ₀ , uses the features to train the model, and sends the trained model to each edge node in the yard. Then, the recognition results uploaded by the yard edge nodes N ₁ to N _m are integrated for decision making. The computational tasks of the recognition process are completed by multiple yard edge nodes, which fully utilizes the computing power of the yard edge nodes while reducing the computing pressure of the cloud server.

The historical accident information analysis module of the storage yard based on data mining is used to detect and analyze the video content to identify and predict abnormal events;

It includes: a processing module for accident-oriented video data and a classification and analysis module for storage yard accidents based on social network analysis;

The accident-oriented video data processing module is used to identify abnormal events based on the video data in the dry bulk material yard; the details are as follows:

1) Identify abnormal blocks of video data in the yard;

The accident video data in the dry bulk material yard is segmented into short-time video segments. The original event video sequence of the yard is preprocessed. The video image frames of the yard are segmented into multiple two-dimensional image units using a sliding window. The adjacent two-dimensional image units of consecutive T frames are stacked to form a three-dimensional space-time cube as the yard accident sampling block and the corresponding feature information is extracted. The abnormal blocks of the video data in the yard are identified through the PCANet network:

Among them, I is a unit vector with the same dimension and size as the yard accident sampling block x _t,s ; is the processed storage yard accident sampling block; max _1≤t≤T (G _t ) is the maximum gradient value of the current gradient storage yard accident feature cube in the time and space dimensions; min _1≤tT (G _t ) is the minimum gradient value in the time and space dimensions;

2) Detect and identify abnormal events in the yard based on the identified abnormal blocks;

2.1) Solve using PCA algorithm The first L ₁ largest features of the covariance matrix to the corresponding feature vector are used as PCA filters, where L ₁ corresponds to the number of required filters; for each yard accident gradient unit G _t , the first layer outputs L ₁ convolution feature maps, and the second layer uses convolution filters to generate L ₂ feature maps for each feature map, and then calculates the standard deviation of the histogram feature as the apparent abnormality score of the yard block feature:

Among them, s _app (i,j) is the abnormal score of the yard performance, and v*i,j+(δ) represents the height value corresponding to the δth interval of the histogram feature:

2.2) Sum the optical flow values I _p of all pixels in the yard block to obtain N _f is the number of pixels in the yard block;

2.3) The motion anomaly score and the appearance anomaly score are fused into _scon = _αsmot + β(1- _sapp ), and an anomaly threshold is set. The fusion of the yard anomaly score is compared with the threshold to achieve detection and discrimination of abnormal events in the yard;

According to the difference in score types, the category of abnormal accidents in the yard can be determined, such as stockpile collapse, abnormal movement patterns, operating activities, etc., so as to identify and recognize the abnormal types of abnormal video segments of the yard.

The storage yard accident classification and analysis module based on social network analysis is used to classify historical storage yard accidents and analyze the causes of historical storage yard accidents;

The accident factors in the storage yard are represented as nodes. The causal relationship between these risk network nodes constitutes a dry bulk material storage yard safety accident semantic network, and the storage yard safety accident semantic network is established.

Each causal factor _Xi of the yard accident includes: human factors, equipment failure, environmental factors, yard layout, and operation management; as shown in Table 1;

Table 1 Causes of storage yard accidents

The above yard causal factors are defined as network nodes of the semantic network of dry bulk material yard safety accidents. The relationship between yard accident causal factors is defined as edges. When two representative causal factors appear together in a yard accident, there is a certain correlation. The more common frequencies there are, the closer the relationship is. By collecting data and counting the number of dry bulk material yard accidents, it is assumed that there are K accidents in total, and (X _i , X _j ) represents that factor i and factor j appear together once.

The yard safety accident semantic network is evaluated based on the betweenness centrality of the yard safety accident semantic network, the proximity centrality of the yard safety accident semantic network and the characteristic vector centrality of the yard safety accident semantic network, which are calculated as follows:

(1) The intermediate centrality of the semantic network of storage yard safety accidents

The ratio of the number of shortest paths passing through the yard-causing node i to the total number of paths is the betweenness centrality of the yard-causing node i. Assuming that _Pjk is the number of shortcuts between the yard-causing node j and the yard-causing node k, and _Pjk (i) is the number of shortcuts between the two yard-causing nodes that include the yard-causing node i, then:

(2) Closeness centrality of the semantic network of storage yard safety accidents

The sum of the shortcut distances of other yard-causing nodes connected to yard-causing node i is the closeness centrality of yard-causing node i. Define d(i,j) as the shortest path distance between i and j, then:

(3) Centrality of the eigenvector of the yard accident network

The number of nodes connected around a yard cause node affects the status and importance of the yard cause node, and is also affected by the importance of these connected yard cause nodes, which can be expressed as:

Where c is a proportional constant, a _ij = 1 if and only if i is connected to j, otherwise it is 0.

By building a network diagram, the more yard cause nodes there are, the greater the role of this factor in the entire semantic network diagram of the yard safety accident, and the stronger the ability to influence other yard cause nodes, the frequent item sets and strong association rules between accident causes are mined from the data. By calculating the intermediate centrality of the yard accident network, the proximity centrality of the yard accident network, and the characteristic vector centrality of the yard accident network, a yard accident network with a higher centrality means that it is on multiple shortest paths of other yard cause nodes, a yard accident network with a higher proximity centrality means that it is closer to all other yard cause nodes, and a yard accident network with a higher characteristic vector centrality means that the importance of the yard cause nodes connected to it is high, so as to classify and analyze the causes of yard accidents, pay attention to potential dangerous causes, and take corresponding control measures to reduce risks.

The yard safety management and control module based on multi-level strategies includes a multi-factor real-time planning and control module based on production safety constraints, an advanced warning module integrating multiple production safety status identification, and a rapid response and emergency handling module for sudden accidents in the yard;

Based on the multi-factor real-time planning and control module of production safety constraints, the yard-related technical personnel divide the dangerous areas for vehicle and personnel movement, and establish a 2D grid map with dangerous area identification in combination with the perception system, and use the multi-objective path planning algorithm based on the improved A* algorithm to carry out multi-objective path planning in the yard;

The multi-objective path planning of the yard is expressed as: G = (N, c), G represents the entire yard search space, N represents the yard target node set, and several target point sets f(S,G _i ) represents the estimated value of the starting point S relative to the target point G _i (i＝1,2,3,...,n) calculated by the heuristic evaluation function. When a yard accident occurs, it is necessary to weigh the priorities of different goals, so the weight coefficient k _i is introduced to adjust the search order;

1) Create a global Open and Close list for the multi-objective path planning of the yard, create the starting search node S of the yard goal, use the public list Goals to store the yard planning goal nodes, and establish an Open and Close list for each yard planning goal point, recorded as G _op (i) and G _cl (i);

2) Put the starting point S of the yard target into G _op (i), expand the nodes adjacent to point S into G _op (i), calculate the estimated value of the starting point corresponding to the target point in each G _op (i), that is, calculate f(S, _Gi ) respectively according to the heuristic evaluation function, and at the same time, sort the calculated estimated values in ascending order in each G _op (i), take the first value in the list and put it into the global Open list, then sort the data in the global Open list according to the weight coefficient k _i , take the yard target node with the smallest estimated value as the starting point of the next step, put the original starting point into the global Close list, and clear each G _op (i) and G _cl (i) list;

3) After reaching a target point, loop through 2) to guide to a target point, set it as G _a (1≤a≤n), then delete G _a from the Goals list, and delete the G _op (i) and G _cl (i) lists at the same time. The remaining yard target points G _op (i) and G _cl (i) lists continue to participate in the next step;

4) Termination condition: Determine whether the Open list is empty or the Goals list is empty, that is, all the target paths to be planned in the yard have been allocated. The multi-objective path planning algorithm based on the improved A* algorithm enables the system to efficiently search for the target node while ensuring the optimal solution and priority planning order, thereby ensuring that the working vehicles and personnel can quickly and safely reach the corresponding work location.

The advanced warning module integrates multiple production safety status identification, which is used to monitor possible dangerous events in real time during the production process. Once a dangerous state is identified, it will promptly alarm and take corresponding measures;

The collaborative detection module monitors the height and speed of the lifting hook. When the height exceeds the set safety height, the system sends out an alarm signal; when the height exceeds the danger alarm height, the system stops the overhead crane urgently. For the early warning of engineering vehicle collision behavior, when the operating vehicle sensor detects that the distance to the obstacle is less than the safety distance, the alarm will sound. The safety distance alarm is divided into reminder alarm and danger alarm. When the distance between the vehicle and other objects or people is less than the reminder alarm distance, the system will send out an alarm signal to remind the driver to stop. If the parking operation is not performed in time due to equipment failure or driver error, resulting in the distance being less than the danger alarm distance, the system will control the equipment to stop urgently. For early warning of personnel in the working area, if a person is found to be not wearing a safety helmet or a reflective work vest in the working area, the system alarm will be triggered, the camera will continue to actively track the target, the monitoring center will pop up a screen to display the current picture, and the system voice will continue to alarm. For the early warning of material pile collapse, the real-time detected video sequence is preprocessed to extract four consecutive video frames, and then the area target of the material pile is detected using the adaptive mixed Gaussian background modeling method. If the target is detected and the target range is greater than a certain value, it means that there is an abnormal movement in the material pile, and the system will issue an alarm.

The rapid response and emergency handling module for sudden accidents in the yard divides the accident types according to the accident template and performs corresponding emergency handling according to the accident type to effectively deal with different types of accidents, and manage the safety of the yard in an all-round and multi-level manner, thereby improving the safety and management efficiency of the yard.

Obtain basic data from previous accident cases and post-accident conditions at the storage yard, and structure the data to build a structured storage yard emergency response template generation model;

Determine the current accident type, match it to the processing template for that accident type, and obtain the emergency processing decision;

The yard full-factor perception system uses various sensors (such as cameras, motion sensors, sound sensors, etc.) to monitor the production process of the dry bulk material yard in real time, and identifies unsafe stacking methods, abnormal sounds, movements, etc. in the yard through image recognition, sound recognition and other technologies, and issues early warnings. The data mining module associates the accident type (such as object impact accidents, vehicle injury accidents, mechanical injury accidents) with the perception data and establishes an association model between the accident type and the perception data. In the dry bulk material yard, the system can associate data such as the dumping of piles captured by the camera and the movement of vehicles captured by the sensor with the accident type. Once the perception module captures the relevant data of accidents such as the dumping of piles and vehicle collisions, the emergency processing module immediately starts the accident processing process, and the system quickly determines the current accident type and matches the processing template of the accident type, such as emergency evacuation and parking instructions. In order to realize the intelligent and precise emergency processing of the yard, the web crawler technology is used to obtain basic data from previous accident cases and post-incident conditions of the yard, and the data is structured to construct a structured yard emergency processing template generation model. The specific process is as follows:

The scenarios of the yard emergency are divided into s _k *(x ₁ ,x ₂ ,…,x _n ),(y ₁ ,y ₂ ,…,y _n )+, where x _n is the environmental attribute of the yard sub-event, including the temperature, pressure, relative fitness and wind speed of the yard emergency; and y n is the state attribute of the yard sub-event. A yard emergency is composed of multiple yard sub-events, so the similarity calculation is performed from two levels: the yard sub-event and the yard sub-event set. The similarity of the yard emergency level is Similarity of the number of sub-events in the yard sub-event set The value range is [0, 1]. Similarity of sub-event sets of two sudden events in the storage yard is the overall similarity of the name set of the yard sub-events, and ω ₁ , ω ₂ , and ω ₃ are the weights assigned to each similarity. Similarly, the similarity of the handling tasks and emergency actions of the yard emergency events is calculated to obtain the similarity of the pending tasks and emergency actions of the yard sub-events Adjusting the values of ω ₁ , ω ₂ , and ω ₃ can adjust the similarity of the emergency actions for storage yard accidents. Multiple storage yard events are compared in sequence in the storage yard emergency management case library to obtain a group of similar storage yard sub-events, and the relationships similar to the storage yard accident emergency response tasks and actions are extracted from them, such as rescuing and evacuating personnel, finding the source of fire, clearing blockages, and restoring ventilation.

The emergency handling module verifies decisions during the accident handling process, for example, it will confirm the handling measures for the yard accident, such as whether the evacuation route is unobstructed and whether the vehicle has been parked in a safe area. After the accident occurs, the emergency handling module will save the data of the yard accident, including the accident type, handling template, perception data and other information, so as to analyze the accident later and provide a basis for safety management and prevention.

The full-factor blind-spot-free perception system detects the yard environment through a variety of detection equipment (such as cameras, radars, ultrasonic sensors, etc.), performs fusion processing at the edge node to identify and track the detection target and upload it to the cloud. After receiving the fusion perception data, the information extraction system uses video automatic processing and information extraction technology for analysis and mining. By analyzing historical accident videos and related data, the law of accidents and potential risk factors are discovered. Through analysis, this module can extract information such as potential safety hazards and abnormal behaviors in the yard, and pass the analysis results to the safety management system. After receiving the transmitted analysis results, the safety management system formulates multi-level safety management strategies based on these results. The safety management system will implement corresponding control measures based on the analysis results, such as adjusting the driving routes of yard operation vehicles or personnel, issuing early warning signals, and starting the emergency parking system. The modules have clear division of labor and work together to achieve all-round and multi-level control of yard safety management. It can improve the safety and efficiency of traditional yards, reduce the risk of accidents, and ensure the safety of operators and equipment.

It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims (10)

1. A multi-level safety control device for a smart storage yard without blind spots, characterized by comprising: The full-factor blind-spot perception module of the storage yard based on cloud-edge collaboration includes a multi-source sensor collaborative perception module and a multi-factor blind-spot perception edge-cloud collaboration module of the storage yard; The yard multi-factor non-blind-zone perception edge-cloud collaboration module includes a yard perception device, a yard edge node and a yard cloud server; The multi-source sensor collaborative perception module is used to perform environmental detection and target recognition using a multi-factor blind-spot-free perception edge-cloud collaborative module for a storage yard based on multi-source sensor collaboration, thereby achieving multi-factor blind-spot-free perception of the storage yard; A data mining-based storage yard historical accident information analysis module, including an accident-oriented video data processing module and a storage yard accident classification analysis module based on social network analysis; The accident-oriented video data processing module is used to identify abnormal events based on the video data in the dry bulk material yard; The storage yard accident classification and analysis module based on social network analysis is used to classify historical storage yard accidents and analyze the causes of historical storage yard accidents; The yard safety management and control module based on multi-level strategies includes a multi-factor real-time planning and control module based on production safety constraints, an advanced warning module integrating multiple production safety status identification, and a rapid response and emergency handling module for sudden accidents in the yard; The multi-factor real-time planning and control module based on production safety constraints is used to divide the dangerous areas for vehicle and personnel movement according to the relevant technical personnel of the yard, establish a 2D grid map with dangerous area identification in combination with the perception system, and use the multi-objective path planning algorithm based on the improved A* algorithm to perform multi-objective path planning in the yard; The advanced warning module integrating multiple production safety status identification is used to monitor the identification of abnormal events in real time. If it is judged to be a dangerous state, it will promptly alarm and take corresponding measures; The rapid response and emergency handling module for sudden accidents in the storage yard divides the accident types according to the accident template, and performs corresponding emergency handling according to the results of the storage yard accident classification analysis module according to the accident type, so as to effectively deal with different types of accidents. 2. The multi-level safety management and control device for a smart storage yard without blind spots according to claim 1 is characterized in that the perception module coordinated by the multi-source sensors performs environmental detection and target recognition; The steps used are as follows: 1) By deploying a variety of sensors, comprehensive detection of various elements in the yard is carried out to achieve full-factor perception of the yard; 2) Calibrate and synchronize various visual sensor devices deployed in the yard; 3) Fuse the information from different sensors to complete target detection. 3. The smart storage yard multi-level safety control device without blind spots according to claim 2 is characterized in that in step 1), a variety of sensors are deployed to comprehensively detect a variety of elements in the storage yard to achieve full-element perception of the storage yard; including: 1.1) Active cameras, infrared cameras, video cameras, and laser radar equipment are integrated to collect data on the yard environment; 1.2) Install a facial recognition system and electronic fence at the entrance and exit to manage and record the entry and exit of personnel; 1.3) Deploy visual sensors in every corner of the entire yard to achieve real-time monitoring of the yard scene; add LiDAR-assisted perception in the stockpile area to achieve real-time modeling of the stockpile, monitor the number of stacked items, stacking height, type and status of goods; 1.4) An infrared camera is placed in the corner above the stockpile area, with the field of view facing the stockpile area, to achieve real-time detection of the heat distribution of objects in the stockpile yard; 1.5) Place a PTC active camera in the middle of the work area to identify objects, people or vehicles that appear in the picture and track them; 1.6) The overhead crane is used for real-time detection to sense and locate objects around the operating vehicle and the vehicle itself, providing a perception basis for vehicle collision avoidance and route planning. A wheel speed meter is installed above the overhead crane connected to the grab bucket to detect the speed of the hook. 4. The smart storage yard multi-level safety control device without blind spots according to claim 2 is characterized in that in step 2), multiple visual sensor devices are deployed in the storage yard for calibration and synchronization; specifically as follows: By moving the calibration vehicle and placing a checkerboard on the roof, sensors with the same field of view are jointly calibrated; when jointly calibrating sensors with the same field of view, the overlapping field of view is enhanced; 2.1) According to the principle of visual camera imaging, calculate the corner coordinates of the camera coordinate system of the mobile calibration vehicle and obtain the direction vector of the chessboard edge For the laser radar on the mobile calibration vehicle, within a scanning cycle T, the candidate point set of the calibration plate plane is obtained Then, for the candidate points Obtain local smooth features, expressed as: If ss>T ₀ , the laser radar scanning beam suddenly changes here, and the point cloud Add smooth feature point set Construct the edge point set of the calibration plate; T ₀ is the set threshold; 2.2) Assume that the plane normal vectors in the yard camera coordinate system and the laser radar coordinate system are and The edge point feature vectors in the camera coordinate system and the lidar coordinate system are and The objective function H is established using the geometric feature constraints of the calibration plate: in, is the rotation matrix from the visual camera coordinate system to the lidar coordinate system; is the translation matrix from the visual camera coordinate system to the lidar coordinate system; Represents surface feature constraints, represents line feature constraints, Represents corner feature constraint; The closed-form solution of the rotation matrix is obtained by jointly solving the three geometric feature constraints, and the translation vector is obtained by substituting it into the constraint equation, thereby completing the external parameter calibration between different sensors. 2.3) By jointly calibrating the external parameters of sensors with the same field of view, the joint calibration of multi-source heterogeneous sensors in a large area of the yard can be achieved. 5. The multi-level safety control device for a smart storage yard without blind spots according to claim 2 is characterized in that in step 3), information from different sensors is integrated to complete target detection; specifically as follows: 3.1) Encode the data collected by the visual sensor in the yard. The encoding method is as follows: perform local sequential color information smear encoding on the point cloud within the cone of vision formed by the 2D detection frame, and convert the original point cloud information (x, y, z, r) into the code (x, y, z, r, S, R, G, B), where x, y, z of the point cloud are spatial position information, r is the intensity value information of the point cloud, S is the recommended channel, and R, G, B are the color channels corresponding to the point cloud; 3.2) After uniform transformation of the point cloud, project it onto the 2D image: Wherein, matrix _C1 is the camera intrinsic parameter matrix, _C2 is the extrinsic parameter homogeneous transformation matrix, which is obtained through the previous multi-sensor joint calibration; s is the recommended channel, _XL is the information of the point cloud in the laser radar coordinate system, and the mapping relationship of the point cloud P( _xL , _yL , _zL ) in the laser radar coordinate system to its position p(u, v) in the plane image coordinate system is obtained by projection; 3.3) Object detection; Convert the point cloud into a sparse pseudo image in the form of a vertical “column”, and then use a 2D convolutional network to detect the pseudo image and predict the 3D detection box; The input point cloud is processed using a 2D convolutional network to obtain a 3D bounding box for multi-factor target recognition. The 3D bounding box is defined by parameters (x, y, z, w, l, h, θ), where x, y, z are the coordinates of the center of the bounding box, w, l, h are the width, length, and height of the bounding box, respectively, and θ is the target orientation. In the convolutional network, the regression loss between the target and the anchor box is: Among them, the subscript gt represents the value in the true box, and the subscript a represents the value in the predicted box. The position, size and direction of the identified yard target are given by the regression loss, and then the category of the target is given by the classification loss. 6. The multi-level safety management and control device for a smart storage yard without blind spots according to claim 1 is characterized in that the processing module of the accident-oriented video data identifies abnormal events based on the video data in the dry bulk material storage yard; the method used is as follows: 1) Identify abnormal blocks of video data in the yard; The accident video data in the dry bulk material yard is segmented into short-time video segments. The original event video sequence of the yard is preprocessed. The video image frames of the yard are segmented into multiple two-dimensional image units using a sliding window. The adjacent two-dimensional image units of consecutive T frames are stacked to form a three-dimensional space-time cube as the yard accident sampling block and the corresponding feature information is extracted. The abnormal blocks of the video data in the yard are identified through the PCANet network: Among them, I is a unit vector with the same dimension and size as the yard accident sampling block x _t,s ; is the processed storage yard accident sampling block; max _1≤t≤T (G _t ) is the maximum gradient value of the current gradient storage yard accident feature cube in the time and space dimensions; min _1≤tT (G _t ) is the minimum gradient value in the time and space dimensions; 2) Detect and identify abnormal events in the yard based on the identified abnormal blocks; 2.1) Solve using PCA algorithm The first L ₁ largest features of the covariance matrix to the corresponding feature vector are used as PCA filters, where L ₁ corresponds to the number of required filters; for each yard accident gradient unit G _t , the first layer outputs L ₁ convolution feature maps, and the second layer uses convolution filters to generate L ₂ feature maps for each feature map, and then calculates the standard deviation of the histogram feature as the apparent abnormality score of the yard block feature: Among them, s _app (i, j) is the abnormal score of the yard performance, and v{i, j}(δ) represents the height value corresponding to the δth interval of the histogram feature: 2.2) Sum the optical flow values I _p of all pixels in the yard block to obtain N _f is the number of pixels in the yard block; 2.3) The motion anomaly score and the appearance anomaly score are fused into _scon = _αsmot + β(1- _sapp ), and an anomaly threshold is set. The fusion of the yard anomaly score is compared with the threshold to achieve detection and discrimination of abnormal events in the yard; According to the difference in the score types, the category of abnormal accidents in the yard can be determined, so as to identify and recognize the abnormal types of abnormal video segments in the yard. 7. The smart storage yard multi-level safety management and control device without blind spots according to claim 1 is characterized in that the storage yard accident classification and analysis module based on social network analysis method classifies historical storage yard accidents and analyzes the causes of historical storage yard accidents; the method used is as follows: The causal factors of the accidents in the storage yard are represented as nodes, and the causal relationship between the network nodes constitutes a semantic network of dry bulk material storage yard safety accidents, and the semantic network of storage yard safety accidents is established; The causal factors of the yard accident, _Xi , include: human factors, equipment failure, environmental factors, yard layout, and operation management; The above yard causal factors are defined as nodes of the semantic network of dry bulk material yard safety accidents. The relationship between yard accident causal factors is defined as edges. When two representative causal factors appear together in a yard accident, there is a certain correlation. The more common frequencies there are, the closer the relationship is. By collecting data and counting the number of dry bulk material yard accidents, it is assumed that there are K accidents in total, and (X _i , X _j ) represents that factor i and factor j appear together once. The yard safety accident semantic network is evaluated based on the betweenness centrality of the yard safety accident semantic network, the proximity centrality of the yard safety accident semantic network and the characteristic vector centrality of the yard safety accident semantic network, which are calculated as follows: (1) The intermediate centrality of the semantic network of storage yard safety accidents The ratio of the number of shortest paths passing through the yard-causing node i to the total number of paths is the betweenness centrality of the yard-causing node i. Assuming that _Pjk is the number of shortcuts between the yard-causing node j and the yard-causing node k, and _Pjk (i) is the number of shortcuts between the two yard-causing nodes that include the yard-causing node i, then: (2) Closeness centrality of the semantic network of storage yard safety accidents The sum of the shortcut distances of other yard-causing nodes connected to the yard-causing node i is the proximity centrality of the yard-causing node i; define d(i,j) as the shortest path distance between i and j, then: (3) Centrality of the semantic network feature vector of storage yard safety accidents The number of nodes connected around a yard cause node affects the status and importance of the yard cause node, and is also affected by the importance of these connected yard cause nodes, which can be expressed as: Where c is a proportional constant, a _ij = 1 if and only if i is connected to j, otherwise it is 0; By establishing a network diagram, the more yard cause nodes there are, the greater the role of this factor in the entire yard safety accident semantic network diagram, and the stronger the ability to influence other yard cause nodes, and the frequent item sets and strong association rules between accident causes are mined from the data; by calculating the intermediate centrality of the yard accident network, the closeness centrality of the yard accident network and the characteristic vector centrality of the yard accident network, a yard accident network with a higher centrality means that it is on multiple shortest paths of other yard cause nodes, a yard accident network with a higher closeness centrality means that it is closer to all other yard cause nodes, and a yard accident network with a higher characteristic vector centrality means that the importance of the yard cause nodes connected to it is high, so as to classify and analyze the causes of yard accidents. 8. The smart storage yard multi-level safety control device without blind spots according to claim 1 is characterized in that the multi-factor real-time planning and control module based on production safety constraints adopts a multi-objective path planning algorithm based on the improved A* algorithm to perform multi-objective path planning in the storage yard; the steps are as follows: The multi-objective path planning of the yard is expressed as: G = (N, c), G represents the entire yard search space, N represents the yard target node set, and several target point sets f(S,G _i ) represents the estimated value of the starting point S relative to the target point G _i (i＝1,2,3,...,n) calculated by the heuristic evaluation function; when a yard accident occurs, the weight coefficient k _i is set by weighing the priorities between different targets to adjust the search order; 1) Create a global Open and Close list for the multi-objective path planning of the yard, create the starting search node S of the yard goal, use the public list Goals to store the yard planning goal nodes, and establish an Open and Close list for each yard planning goal point, recorded as G _op (i) and G _cl (i); 2) Put the starting point S of the yard target into G _op (i), expand the nodes adjacent to point S into G _op (i), calculate the estimated value of the starting point corresponding to the target point in each G _op (i), that is, calculate f(S, _Gi ) respectively according to the heuristic evaluation function, and at the same time, sort the calculated estimated values in ascending order in each G _op (i), take the first value in the list and put it into the global Open list, then sort the data in the global Open list according to the weight coefficient k _i , take the yard target node with the smallest estimated value as the starting point of the next step, put the original starting point into the global Close list, and clear each G _op (i) and G _cl (i) list; 3) After reaching a target point, loop through 2) to guide to a target point, set it as _Ga , then delete _Ga from the Goals list, and delete the _Gop (i) and _Gcl (i) lists at the same time; the remaining yard target points _Gop (i) and _Gcl (i) lists continue to participate in the next step; 4) Termination condition: determine whether the Open list is empty or the Goals list is empty, that is, all the target paths to be planned for the storage yard have been allocated. 9. The multi-level safety management and control device for a smart storage yard without blind spots according to claim 1 is characterized in that the advanced warning module integrating multiple production safety status identification is used to monitor the identification of abnormal events in real time, and if it is judged to be a dangerous state, it will promptly alarm and take corresponding measures; The collaborative detection module is used to monitor the recognition results of abnormal events in real time to identify unsafe stacking methods, abnormal sounds and abnormal movements in the yard. If the sensor threshold in the recognition of abnormal events exceeds the safety threshold, an alarm signal is issued; if the sensor threshold in the recognition of abnormal events exceeds the danger alarm threshold, the corresponding equipment is stopped and controlled. 10. The intelligent storage yard multi-level safety management and control device without blind spots according to claim 1 is characterized in that the storage yard sudden accident rapid response and emergency processing module divides the accident types according to the accident template, and performs corresponding emergency processing according to the results of the storage yard accident classification analysis module according to the accident type, so as to effectively deal with different types of accidents; Obtain basic data from previous accident cases and post-accident conditions at the storage yard, and structure the data to build a structured storage yard emergency response template generation model; Determine the current accident type, match it to the processing template for that accident type, and obtain the emergency processing decision; Construct a structured yard emergency treatment template generation model. The specific process is as follows: The scenarios of the yard emergency are divided into s _k {(x ₁ ,x ₂ ,…,x _n ),(y ₁ ,y ₂ ,…,y _n )}, where x _n is the environmental attribute of the yard sub-event, including the temperature, pressure, relative humidity and wind speed of the yard emergency; is the state attribute of the yard sub-event; A yard emergency event is composed of multiple yard sub-events, and similarity calculation is performed from two levels: yard sub-events and yard sub-event sets; The similarity of the emergency level of the storage yard is: The similarity of the number of sub-events in the yard sub-event set is: The value range is [0, 1]; Similarity of sub-event sets of two sudden events at the storage yard is the overall similarity of the set of yard sub-event names, ω ₁ , ω ₂ , ω ₃ are the weights assigned to each similarity; The similarity of the handling tasks and emergency actions of the yard emergency events is calculated to obtain the similarity of the pending tasks and emergency actions of the yard sub-events. The values of weights ω ₁ , ω ₂ , ω ₃ are adjusted to adjust the similarity of the emergency actions for the storage yard accident; Multiple yard event sets are compared in the yard emergency management case library according to similarity sorting to obtain a group of similar yard sub-events, from which relationships similar to yard accident emergency response tasks and actions are extracted to obtain emergency response decisions; and the decisions are checked during the accident handling process to ensure the implementation of emergency response decisions.