CN116844103A - LOTO informatization intelligent integrated security management method - Google Patents
LOTO informatization intelligent integrated security management method Download PDFInfo
- Publication number
- CN116844103A CN116844103A CN202310650492.8A CN202310650492A CN116844103A CN 116844103 A CN116844103 A CN 116844103A CN 202310650492 A CN202310650492 A CN 202310650492A CN 116844103 A CN116844103 A CN 116844103A
- Authority
- CN
- China
- Prior art keywords
- image
- loto
- model
- training
- yolo
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000000007 visual effect Effects 0.000 claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 27
- 230000008569 process Effects 0.000 claims abstract description 11
- 238000011161 development Methods 0.000 claims abstract description 5
- 230000004927 fusion Effects 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 35
- 239000011159 matrix material Substances 0.000 claims description 10
- 238000012360 testing method Methods 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000012795 verification Methods 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000007547 defect Effects 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000011410 subtraction method Methods 0.000 claims description 6
- 239000003086 colorant Substances 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 5
- 238000011068 loading method Methods 0.000 claims description 5
- 238000010200 validation analysis Methods 0.000 claims description 5
- 230000018109 developmental process Effects 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 239000012634 fragment Substances 0.000 claims description 4
- 101100476202 Caenorhabditis elegans mog-2 gene Proteins 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000013210 evaluation model Methods 0.000 claims description 3
- 238000002156 mixing Methods 0.000 claims description 3
- 238000012805 post-processing Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000013526 transfer learning Methods 0.000 claims description 3
- 230000007704 transition Effects 0.000 claims description 3
- 238000002955 isolation Methods 0.000 description 3
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006378 damage Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 230000003628 erosive effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000002620 method output Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 231100000862 numbness Toxicity 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000003449 preventive effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
- G06V30/19173—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/09—Recognition of logos
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an LOTO informatization intelligent integrated safety management method, which comprises the following steps: 1) Optimizing a yolo human body recognition algorithm under a high-definition picture, dividing the picture into a plurality of areas with the same resolution as the standard picture of the algorithm, and respectively recognizing the areas; 2) Image stitching is carried out by using OpenCV, and the regional pictures are divided, matched and stitched, so that the segmented object is kept complete after fusion of multiple pictures, and repeated recognition is avoided; 3) Detecting motion prospects is increased, and foreground detection is carried out by using OpenCV; 4) Visual identification of the number of locks; 5) Identifying a label, and performing visual identification on the safety warning words by using OCR; 6) Visual identification of equipment states, carrying out algorithm development aiming at field equipment scenes, and identifying the running states of equipment; the invention can reduce the risk caused by subjective factors of personnel in the traditional LOTO operation process and greatly improves the LOTO safety.
Description
[ technical field ]
The invention relates to the technical field of LOTO security, in particular to an LOTO informatization intelligent integrated security management method.
[ background Art ]
LOTO (lock out and tag out) is a prior art management technique for preventing the risk of injury from energy release by isolating and locking certain sources of dangerous energy to prevent personal injury. It comprises the following two points: a locking mechanism is provided on the energy isolation device (e.g., power switch, compressed air switch) prior to the authorization of the associated operator to operate, to ensure that the energy isolation device and controlled equipment are not operated or powered (lock out) prior to removal of the locking mechanism. And a tag (tag out) specially used for safety warning is set to remind staff to prohibit the isolated equipment from being powered.
The existing LOTO management technology relies on authorized personnel to carry out energy supply switching and/or padlock and label on site entering and exiting a doorway, and is based on the sufficiently clear subjective awareness of personnel, and the safety risk is concentrated on that the operators do not operate strictly according to LOTO management rules, such as that the operators do not lock or hang a card on an isolation device before entering an operation area, or the personnel number and the lock number are inconsistent when the personnel trailing. Theoretically, if no error occurs in the link, the safety of LOTO design is guaranteed. However, the staff has long repeated operation flow, high repetition times, no directly perceived benefit incentive, fool or fatigue or numbness, and weak safety awareness, which may cause the occurrence of the above-mentioned illegal phenomena, and may further cause safety accidents.
[ summary of the invention ]
The invention aims to solve the defects and provide the LOTO informatization intelligent integrated safety management method, which can reduce risks caused by subjective factors of personnel in the traditional LOTO operation process and greatly improve the safety of LOTO.
In order to achieve the purpose, the LOTO informatization intelligent integrated safety management method comprises the following steps:
1) Optimizing a yolo human body recognition algorithm under a high-definition picture, dividing the picture into a plurality of areas with the same resolution as the standard picture of the algorithm, and respectively recognizing the areas;
2) Image stitching is carried out by using OpenCV, and the regional pictures are divided, matched and stitched, so that the segmented object is kept complete after fusion of multiple pictures, and repeated recognition is avoided;
3) Detecting the motion foreground, detecting the foreground by using OpenCV, judging whether a person exists by adopting whether the motion foreground exists, and judging whether a non-stationary human body exists;
4) Visual identification of the number of locks, training a custom YOLO model, developing an algorithm aiming at the locks and the hasp plates, and taking real-time videos of the site locks and the hasp plates with enough resolution as input;
5) Identifying the tag, and performing visual identification on a security warning word used by the dangerous and DANGER tag by using OCR;
6) Visual identification of the state of the equipment, carrying out algorithm development aiming at the scene of the field equipment, and identifying the running state of the equipment.
Further, the step 1) specifically includes the following steps:
1.1 Pretreatment: loading a high-resolution image, calculating the size of a smaller segment according to the input size of the YOLO model, dividing the original image into different resolutions, and dividing the high-resolution image into non-overlapping segments of each pixel; if it is desired to ensure that objects near segment boundaries are not missed, overlapping segments are used by defining the overlap percentage;
1.2 Segmentation image): traversing the high-resolution image, extracting smaller fragments according to the calculated size and the overlapping percentage, tracking the coordinates of each fragment relative to the original high-resolution image, and reconstructing the position of the detected object in the original image;
1.3 Object detection: for each extracted segment, performing the required pre-processing of the YOLO model, inputting each pre-processed segment into the YOLO model to detect objects, for each detected object, calculating its position relative to the original high resolution image from the coordinates of the associated segment;
1.4 Post-treatment: the objects detected in all segments are combined, repeated detection due to overlapping areas is processed, and recognition is performed separately.
Further, in step 2), image stitching using OpenCV includes the steps of:
2.1 Loading an image: reading input images to be spliced together;
2.2 Feature detection and description: for each input image, feature points are detected using feature detectors including, but not limited to SIFT, SURF, or ORB; calculating a feature descriptor of each key point in each image, wherein the descriptor is a compact representation of a local image area around the key point, and is helpful for matching the key points between the images;
2.3 Feature matching): comparing feature descriptors of key points in different images to find potential matches, using violent matching or FLANN-based matching methods to find optimal matches between key points of different images, applying matching methods including but not limited to RANSAC, filtering out incorrect outlier matches and preserving correct interior point matches;
2.4 Homography matrix estimation: for matched image pairs, computing a homography matrix describing the relationship between the image planes;
2.5 Image distortion): distorting the input image according to the computed homography matrix to align it on a common plane;
2.6 A panoramic view is composed of: combining the warped images together to create a final stitched panorama; finding the best seam between overlapping images to minimize visible artifacts, blending the images together, ensuring a smooth transition in the overlapping region;
2.7 Clipping and adjusting the panorama: finally, any unwanted black areas or artifacts are cropped from the panorama and the overall exposure or color balance is adjusted to create a visually attractive result.
Further, in step 3), the foreground detection using OpenCV includes the steps of:
3.1 Reading a video frame: reading an input video frame using a video capture object;
3.2 Selecting a background subtraction method: openCV provides background subtraction algorithms including, but not limited to MOG, MOG2, and KNN;
3.3 Initializing a background subtraction: creating an instance of the selected background subtraction method, customizing parameters according to the selected algorithm, the parameters including history length, threshold value, shadow detection and learning rate;
3.4 Application background subtraction): traversing the video frames, and applying an apply method of a background canceller to each frame to generate a foreground mask;
3.5 Post-treatment: because the generated foreground mask may contain noise, shadows, or holes, applying post-processing techniques to clean the mask, separating the individual objects using thresholding and contour detection and filtering out unwanted artifacts;
3.6 Extracting foreground objects): extracting foreground objects from the original video frames by using the cleaned foreground mask;
3.7 Analyzing or tracking objects): once the foreground object is extracted, analysis or tracking is performed according to the specific application of object tracking, motion analysis or activity recognition;
3.8 Display or store the results: the original video frames of the detected foreground object are visualized or the result is stored in a new video file or image sequence.
Further, in step 4), the specific steps of custom YOLO model training are:
4.1 Collecting and preparing a data set: collecting an image dataset containing model-detected objects, including locks and labels, labeling bounding boxes and class labels for each object using a labeling tool;
4.2 Splitting the dataset: splitting the data set into a training set, a verification set and a test set, wherein the verification set is used for evaluating the performance of the model, and the test set is used for evaluating the generalization capability of the model after training;
4.3 Conversion annotation: converting the annotation to a format compatible with YOLO;
4.4 Configuration YOLO): selecting a pre-trained YOLO architecture as a starting point, modifying a configuration file to match the number of categories in the custom data set, and adjusting other parameters according to the hardware and the data set size;
4.5 Pre-processing the data set: applying data enhancement techniques to increase the size and diversity of the data set, the enhancement methods including flipping, rotating, scaling, and changing brightness and contrast;
4.6 Initializing weights): training can be started from the beginning, and pre-training weights similar to tasks can be used as starting points for transfer learning;
4.7 Training model: training a customized YOLO model using the data set, configuration and initialization weights, monitoring the training process, and focusing on loss and average accuracy metrics, stopping training when performance on the validation set reaches a plateau or begins to drop;
4.8 Fine tuning model): if the performance of the model is not satisfactory, fine tuning the model by adjusting hyper-parameters, modifying the architecture, or adding more training data;
4.9 Evaluation model: evaluating the performance of the trained model by using the test set, and calculating an evaluation index;
4.10 A deployment model: once the custom YOLO model reaches satisfactory performance, it is deployed into the target environment;
4.11 Monitoring and updating): in practical applications, the performance of the model is continuously monitored, and new data is collected and annotated to solve the problem of discovery.
Further, in step 4), the lock visual identification includes the steps of:
4.1 Pretreatment: the color selection of the lock body, the lock hook and the hasp board is respectively distinguished in hardware, and the selected color is the color which is not in the image environment; in terms of software, searching and focusing on an object area according to color characteristics on the whole image picture, and then calculating;
4.2 The outer ring of the hole of the hasp board of the lockset is coated with a color which is relatively larger than the color of the hasp board, and when the hasp board is padlock, the hole presents a C shape with the color as above; when the padlock is not adopted, the opening presents an O shape;
4.3 Visual recognition: the number of locks is judged by identifying the number of C-shaped and O-shaped locking plates; if the sum of the number of the C-shaped forms and the number of the O-shaped forms is not the number of the hasp plate holes, alarming a visual algorithm error; if the number of "C" types is less than the number of people in the field, alarm "LOTO violation".
Further, in step 4), during the visual identification of the locks, the periphery of the lock hasp board is marked by lines with obvious colors, and then the number of the locks is determined by identifying the number of the separated segments of the lines with the obvious colors.
Further, in step 4), during the visual identification of the lock, the hasp board is selected to be in an A color with higher saturation and brightness, the lock is selected to be in a black color Fang Suogou, the lock body is selected to be in a non-A color, and the number of convex defects in the A color area of the hasp board is identified and calculated, namely the number of locks.
Further, in step 6), the visual recognition of the device status is performed from two aspects: (1) when the locks are all opened or the number of the field locks is 0, the running state of the equipment is set; (2) the operating state of the device is when at least one lock is present in the field.
Compared with the prior art, the invention has the following advantages:
(1) The invention can reduce the risk caused by subjective factors of personnel in the traditional LOTO running process, strengthen the compliance of LOTO rules, improve the safety of LOTO management, and increase the convenience of LOTO related daily operations and form corresponding program records by means of objective, high-reliability and tired-free software algorithm means;
(2) According to the invention, the camera is installed in the LOTO management area, a machine vision algorithm is applied, personnel and authority thereof, locking quantity and the number of people in the activity area are identified, defects of LOTO rule execution are found, and corresponding linkage measures are triggered, so that the safety is improved;
(3) The invention defaults to keep the original LOTO management rule, which has been proved by global practice for many years, guards the safety bottom line, and realizes the functions of prevention, recording and alarming by the linkage of the machine vision algorithm and the LOTO rule;
(4) The invention can effectively eliminate the premise of subjective consciousness of personnel in the traditional LOTO operation rule, and the safety of LOTO is greatly improved by monitoring and eliminating the subjective consciousness of personnel through a machine vision technology;
(5) The method outputs full and computable histories including photos and events marked by visual algorithms for rule compliance and violation initiative, thereby being beneficial to management of inspection, tracking, education and correction;
(6) Based on full verification and full use experience, the user can consider the comprehensive electronic LOTO, namely, directly binding an energy supply switch with detection logic of whether someone exists or not, and using the detection logic as audit verification before equipment energy supply to ensure safety.
[ description of the drawings ]
FIG. 1 is a schematic view of the invention in yellow coating on the outside of the lock hasp plate opening;
FIG. 2 is a schematic illustration of the present invention marked with yellow lines around the lock hasp plate;
FIG. 3 is a schematic view of the lock Fang Suogou of the present invention in black;
FIG. 4 is a schematic representation of the identification of a tag of the present invention;
fig. 5 is a schematic flow chart of the present invention.
Detailed description of the preferred embodiments
The invention provides an LOTO informatization intelligent integrated safety management method, which comprises the following steps:
1) Optimizing a yolo human body recognition algorithm under a high-definition picture, dividing the picture into a plurality of areas with the same resolution as the standard picture of the algorithm, and respectively recognizing the areas;
2) Image stitching is carried out by using OpenCV, and the regional pictures are divided, matched and stitched, so that the segmented object is kept complete after fusion of multiple pictures, and repeated recognition is avoided;
3) Detecting the motion foreground, detecting the foreground by using OpenCV, judging whether a person exists by adopting whether the motion foreground exists, and judging whether a non-stationary human body exists;
4) Visual identification of the number of locks, training a custom YOLO model, developing an algorithm aiming at the locks and the hasp plates, and taking real-time videos of the site locks and the hasp plates with enough resolution as input;
5) Identifying the tag, and performing visual identification on a security warning word used by the dangerous and DANGER tag by using OCR;
6) Visual identification of the state of the equipment, carrying out algorithm development aiming at the scene of the field equipment, and identifying the running state of the equipment.
The invention is further illustrated below in connection with specific examples:
the invention relates to an integrated scheme of informatization automation and intellectualization of LOTO management, which is hereinafter referred to as ALT (AugmentedLock Out & Tag Out), and aims to reduce risks caused by subjective factors of personnel in the traditional LOTO operation process, enhance the compliance of LOTO rules, improve the safety of LOTO management, increase the convenience of LOTO related daily operations and form corresponding program records through objective, high-reliability and tired-free software algorithm means.
The working principle of the invention is as follows: installing a camera in the LOTO management area, applying a machine vision algorithm, identifying personnel and authority thereof, locking quantity and the number of people in the activity area, finding out the defect of LOTO rule execution, and triggering corresponding linkage measures. The main technical target functions of the method comprise static locking quantity recognition in machine vision, dynamic staff quantity judgment, overlap de-duplication of multiple camera recognition ranges and improvement of recognition accuracy. In order to achieve the above purpose, the specific technical scheme of the invention is as follows:
1) Machine vision identifies person, lock, tag, equipment status
1.1 Optimizing yolo human body recognition algorithm under high-definition picture, and improving recognition accuracy
Because the picture used by the yolo algorithm is a high-resolution image shot by the high-definition camera, the picture can be standardized to about 400 x 400 resolution in the human image recognition process, and the utilization rate of image information is greatly reduced. Therefore, the invention improves and optimizes the algorithm, divides the picture into a plurality of areas with the same resolution as the standard picture of the algorithm, and respectively identifies the areas, thereby effectively improving the identification effect. The method comprises the following specific implementation steps:
1.1.1 Pretreatment: the high resolution image is loaded. According to the input size of the YOLO model, the size of the smaller segment is calculated, and the YOLO model used in the embodiment needs to input 416x416 pixels, so that the high-resolution image can be split into non-overlapping segments of 416x416 pixels, and then different versions of YOLO are selected, so that the original image can be split into different resolutions. Alternatively, overlapping segments may be used if it is desired to ensure that objects near the segment boundaries are not missed, by defining a percentage of overlap (e.g., 10-20% overlap).
1.1.2 Segmentation image): traversing the high resolution image, and extracting smaller segments according to the calculated size and percentage of overlap. This information is important when tracking the coordinates of each segment relative to the original high resolution image and reconstructing the position of the detected object in the original image.
1.1.3 Object detection: for each extracted segment, the pre-processing (e.g., normalization, resizing, etc.) required by the YOLO model is performed. Each pre-processed segment is input into the YOLO model to detect objects. For each detected object, its position relative to the original high resolution image is calculated from the coordinates of the segment to which it belongs.
1.1.4 Post-treatment: the objects detected in all segments are combined and repeated detection due to overlapping areas is handled. Duplicate detection may be handled using non-maximum suppression (NMS) or other techniques. The detections may be visualized on the original image, further analyzed, or other tasks performed as needed, first in a list of detected objects, including their position relative to the original high resolution image.
1.2 In addition, since a plurality of cameras are installed for the same area, there is a phenomenon that one object is captured by the plurality of cameras when taking pictures, so that one object is divided among the plurality of pictures, and the recognition accuracy is lowered. Therefore, the invention needs to divide, match and splice the regional pictures, so that the segmented object basically keeps complete after fusion of multiple pictures, repeated recognition is avoided, and the recognition precision is improved.
The following are the main steps for image stitching using OpenCV:
1.2.1 Loading an image: the input images to be stitched together are read.
1.2.2 Feature detection and description: for each input image, feature points (key points) are detected using a feature detector such as SIFT, SURF, or ORB. Feature descriptors are computed for each keypoint in each image, the descriptors being compact representations of local image regions around the keypoint, facilitating matching of keypoints between images.
1.2.3 Feature matching): feature descriptors of keypoints in different images are compared to find potential matches. A strong match, a FLANN-based match, or other method is used to find the best match between keypoints of different images. Robust matching methods, such as RANSAC, are applied to filter out incorrect matches (outliers) and preserve correct matches (inliers).
1.2.4 Homography matrix estimation: for matched pairs of images, a homography matrix (transformation matrix) describing the relationship between the image planes is computed, typically using the inlier matching obtained in the previous step.
1.2.5 Image distortion): the input images are warped according to the computed homography matrix so that they are aligned in a common plane, which is critical to creating a seamless panorama.
1.2.6 A panoramic view is composed of: the warped images are combined together to create a final stitched panorama. The process comprises the following steps: an optimal seam is found between overlapping images to minimize visible artifacts. The images are blended together using techniques such as multi-band blending, ensuring a smooth transition in the overlap region.
1.2.7 Clipping and adjusting the panorama: finally, any unwanted black areas or artifacts are cropped from the panorama and the overall exposure or color balance is adjusted to create a visually attractive result.
2) Detection to increase motion foreground
The foreground detection is much higher than the stability of human body identification, so that whether a person exists or not is judged by adopting whether a motion prospect exists or not, and a safer scheme is adopted when judging whether a non-stationary human body exists or not. Foreground detection (also known as background subtraction) is a technique used in computer vision and image processing to separate moving objects in a video sequence from a static background. The following are the steps for foreground detection using OpenCV:
2.1 Reading a video frame: the input video frames are read using the video capture object, and each frame is processed to detect the foreground object.
2.2 Selecting a background subtraction method: openCV provides various background subtraction algorithms, such as MOG (gaussian mixture), MOG2, and KNN (K-nearest neighbors), with appropriate methods selected based on application requirements and constraints.
2.3 Initializing a background subtraction: creating an instance of the selected background subtraction method, parameters such as history length, threshold, shadow detection, and learning rate can be customized according to the selected algorithm.
2.4 Application background subtraction): traversing the video frames, applying the apply () method of the background subtraction to each frame, this operation will generate a foreground mask, which is a binary image highlighting moving objects.
2.5 Post-treatment: the generated foreground mask may contain noise, shadows, or holes; applying post-processing techniques such as morphological operations (erosion, dilation, open or close) to clean the mask and make it more accurate; thresholding and contour detection may also be used to separate the individual objects and filter out unwanted artifacts.
2.6 Extracting foreground objects): extracting foreground objects from the original video frame using the cleaned foreground mask may be accomplished by performing bitwise and operations between the video frame and the foreground mask.
2.7 Analyzing or tracking objects): once the foreground object is extracted, the analysis or tracking may be performed according to a particular application (e.g., object tracking, motion analysis, or activity recognition).
2.8 Display or store the results: the original video frames of the detected foreground object are visualized or the result is stored in a new video file or image sequence.
3) Visual identification of the number of locks
The method specifically aims at the lock and the hasp board of a customer to develop an algorithm, and takes a real-time video of the field lock and the hasp board with enough resolution as an input.
3.1 Scheme one: the custom YOLO model is trained, and the following specific steps of the process are as follows:
3.1.1 Collecting and preparing a data set: collecting an image dataset containing objects for which model detection is desired, including locks and tags; the data set should be diverse and represent various scenes, lighting conditions, and object directions; each object is labeled with a bounding box and category label using a labeling tool such as Labelbox, VGG Image Annotator (VIA) or labelImg.
3.1.2 Splitting the dataset: splitting the data set into a training set, a verification set and a test set; typical split ratios were 70% for training, 20% for validation, 10% for testing; during training, the validation set is used to evaluate the performance of the model, while the test set is used to evaluate the generalization ability of the model after training.
3.1.3 Conversion annotation: the annotation is converted to a YOLO compatible format, typically comprising one text file per image, containing the class, x, y, width and height of the bounding box, all normalized to the [0,1] range.
3.1.4 Configuration YOLO): selecting a pre-trained YOLO architecture (e.g., YOLOv3, YOLOv4, YOLOv 5) as a starting point, modifying the configuration file to match the number of categories in the custom dataset, and adjusting other parameters, such as learning rate, batch size, and sub-region, based on hardware and dataset size.
3.1.5 Pre-processing the data set: applying data enhancement techniques to increase the size and diversity of the data set, which helps to improve the performance of the model; common enhancement methods include flipping, rotating, scaling, and changing brightness and contrast.
3.1.6 Initializing weights): training may be started from scratch, or pre-training weights similar to tasks may be used as starting points (transfer learning); migration learning is recommended because time can be saved and often better performance can be achieved.
3.1.7 Training model: training a custom YOLO model using the data set, configuration, and initialization weights; monitoring the training process and focusing on loss and average Accuracy (AP) metrics; training is stopped when performance on the validation set reaches a plateau or begins to drop.
3.1.8 Fine tuning model): if the performance of the model is not satisfactory, the model may be fine-tuned by adjusting hyper-parameters, modifying the architecture, or adding more training data.
3.1.9 Evaluation model: evaluating the performance of the trained model using the test set, calculating an evaluation index, such as accuracy, recall, F1 score, and average accuracy (mAP); the model may be further optimized to improve these metrics as desired.
3.1.10 A deployment model: once the custom YOLO model reaches satisfactory performance, it can be deployed into a target environment, such as a mobile device, embedded system, or cloud server, ensuring that the model is optimized to meet the performance and resource requirements of the target platform.
3.1.11 Monitoring and updating): in practical application, the performance of the model is continuously monitored, and new data are collected and marked to solve the found problem; the model may be updated periodically to maintain its performance and accuracy, if desired.
The above is the whole process of training the custom YOLO model, and in practice, the best performance can be obtained through multiple iterations and adjustments until the effect meets the requirement.
3.2 Scheme II
3.2.1 Pretreatment: on the hardware, the invention carries out related design and helps to realize stable identification effect. I.e. the color choices of the lock body, the shackle and the hasp plate, need to be distinguished from each other. Further, the selected color is a color that is not present in the image environment. In terms of software, an object region is searched and focused on by characteristics such as color and the like on the whole image screen, and then calculation is performed.
3.2.2 Embodiment(s): (1) the outer ring of the hole of the lock hasp plate is coated with yellow (other colors which are larger than the color of the hasp plate can be selected). When the hasp board padlock is adopted, as shown in the attached figure 1, the opening presents a yellow C shape; when not padlock, the opening presents an O shape. (2) Visual identification: the number of locks is judged by identifying the number of C-shaped and O-shaped locking plates (for example, the number of locks is 4 when 4C-shaped and 2O-shaped locking plates are arranged in the 6-hole locking plate). If the sum of the number of the 'C' types and the number of the 'O' types is not the number of the snap plate holes, alarming the 'visual algorithm error'. If the number of "C" types is less than the number of people in the field, alarm "LOTO violation".
3.3 Scheme III
3.3.1 Pretreatment: the periphery of the lock hasp board is marked by yellow lines.
3.3.2 Visual recognition: by identifying the number of segments separated by the yellow line, as shown in (1) and (2) of fig. 2, the number of locks is determined (if the number of the segments to be separated is 2, the number of locks is 2).
3.4 Scheme IV
The snap plate is selected from A color (yellow or red color with higher saturation and brightness), the lock is selected from black Fang Suogou, the lock body is selected from non-A color, and the number of convex defects in the snap plate A color area is identified and calculated, as shown in figure 3, namely the lock number.
4) Identification of the tag: OCR is used for carrying out visual recognition on security warning words used by labels such as DANGER, DANGER and the like. Technically, a hundred-degree paddleddle module implementation is called.
5) Visual identification of device status: the method is characterized in that algorithm development is specifically carried out aiming at field equipment scenes, and identification is mainly carried out from two aspects: (1) when the locks are all opened or the number of the field locks is 0, the running state of the equipment is set; (2) the operating state of the device is when at least one lock is present in the field.
The invention may also include aspects such as: a) The application relational database system at the back end; b) An operation interface of the BS structure at the front end; c) The account numbers, login, default labels and labels of related personnel; d) Recording related to manual flow: logging in, approving, entering and exiting, opening and closing the door; e) Device flow events: the device feeds back events that have been energy acceptable, events that have not been energy acceptable. And optionally an automated interlock: a) Communication with related equipment to acquire the equipment state; b) Communication and control of the electromagnetic door lock and the relay in the management area are realized; c) Alarms that violate LOTO rules, such as a message notification, audible and visual alarms, preventive energization.
The present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principles of the invention are intended to be equivalent substitutes and are included in the scope of the invention.
Claims (9)
1. The LOTO informatization intelligent integrated safety management method is characterized by comprising the following steps of:
1) Optimizing a yolo human body recognition algorithm under a high-definition picture, dividing the picture into a plurality of areas with the same resolution as the standard picture of the algorithm, and respectively recognizing the areas;
2) Image stitching is carried out by using OpenCV, and the regional pictures are divided, matched and stitched, so that the segmented object is kept complete after fusion of multiple pictures, and repeated recognition is avoided;
3) Detecting the motion foreground, detecting the foreground by using OpenCV, judging whether a person exists by adopting whether the motion foreground exists, and judging whether a non-stationary human body exists;
4) Visual identification of the number of locks, training a custom YOLO model, developing an algorithm aiming at the locks and the hasp plates, and taking real-time videos of the site locks and the hasp plates with enough resolution as input;
5) Identifying the tag, and performing visual identification on a security warning word used by the dangerous and DANGER tag by using OCR;
6) Visual identification of the state of the equipment, carrying out algorithm development aiming at the scene of the field equipment, and identifying the running state of the equipment.
2. The LOTO informationized intelligent integrated security management method of claim 1, wherein step 1) comprises the steps of:
1.1 Pretreatment: loading a high-resolution image, calculating the size of a smaller segment according to the input size of the YOLO model, dividing the original image into different resolutions, and dividing the high-resolution image into non-overlapping segments of each pixel; if it is desired to ensure that objects near segment boundaries are not missed, overlapping segments are used by defining the overlap percentage;
1.2 Segmentation image): traversing the high-resolution image, extracting smaller fragments according to the calculated size and the overlapping percentage, tracking the coordinates of each fragment relative to the original high-resolution image, and reconstructing the position of the detected object in the original image;
1.3 Object detection: for each extracted segment, performing the required pre-processing of the YOLO model, inputting each pre-processed segment into the YOLO model to detect objects, for each detected object, calculating its position relative to the original high resolution image from the coordinates of the associated segment;
1.4 Post-treatment: the objects detected in all segments are combined, repeated detection due to overlapping areas is processed, and recognition is performed separately.
3. The LOTO informationized intelligent integrated security management method of claim 1, wherein in step 2), using OpenCV for image stitching comprises the steps of:
2.1 Loading an image: reading input images to be spliced together;
2.2 Feature detection and description: for each input image, feature points are detected using feature detectors including, but not limited to SIFT, SURF, or ORB; calculating a feature descriptor of each key point in each image, wherein the descriptor is a compact representation of a local image area around the key point, and is helpful for matching the key points between the images;
2.3 Feature matching): comparing feature descriptors of key points in different images to find potential matches, using violent matching or FLANN-based matching methods to find optimal matches between key points of different images, applying matching methods including but not limited to RANSAC, filtering out incorrect outlier matches and preserving correct interior point matches;
2.4 Homography matrix estimation: for matched image pairs, computing a homography matrix describing the relationship between the image planes;
2.5 Image distortion): distorting the input image according to the computed homography matrix to align it on a common plane;
2.6 A panoramic view is composed of: combining the warped images together to create a final stitched panorama; finding the best seam between overlapping images to minimize visible artifacts, blending the images together, ensuring a smooth transition in the overlapping region;
2.7 Clipping and adjusting the panorama: finally, any unwanted black areas or artifacts are cropped from the panorama and the overall exposure or color balance is adjusted to create a visually attractive result.
4. The LOTO informationized intelligent integrated security management method of claim 1, wherein in step 3), using OpenCV for foreground detection comprises the steps of:
3.1 Reading a video frame: reading an input video frame using a video capture object;
3.2 Selecting a background subtraction method: openCV provides background subtraction algorithms including, but not limited to MOG, MOG2, and KNN;
3.3 Initializing a background subtraction: creating an instance of the selected background subtraction method, customizing parameters according to the selected algorithm, the parameters including history length, threshold value, shadow detection and learning rate;
3.4 Application background subtraction): traversing the video frames, and applying an apply method of a background canceller to each frame to generate a foreground mask;
3.5 Post-treatment: because the generated foreground mask may contain noise, shadows, or holes, applying post-processing techniques to clean the mask, separating the individual objects using thresholding and contour detection and filtering out unwanted artifacts;
3.6 Extracting foreground objects): extracting foreground objects from the original video frames by using the cleaned foreground mask;
3.7 Analyzing or tracking objects): once the foreground object is extracted, analysis or tracking is performed according to the specific application of object tracking, motion analysis or activity recognition;
3.8 Display or store the results: the original video frames of the detected foreground object are visualized or the result is stored in a new video file or image sequence.
5. The LOTO informatization intelligent integrated security management method of claim 1, wherein in step 4), the specific steps of custom YOLO model training are:
4.1 Collecting and preparing a data set: collecting an image dataset containing model-detected objects, including locks and labels, labeling bounding boxes and class labels for each object using a labeling tool;
4.2 Splitting the dataset: splitting the data set into a training set, a verification set and a test set, wherein the verification set is used for evaluating the performance of the model, and the test set is used for evaluating the generalization capability of the model after training;
4.3 Conversion annotation: converting the annotation to a format compatible with YOLO;
4.4 Configuration YOLO): selecting a pre-trained YOLO architecture as a starting point, modifying a configuration file to match the number of categories in the custom data set, and adjusting other parameters according to the hardware and the data set size;
4.5 Pre-processing the data set: applying data enhancement techniques to increase the size and diversity of the data set, the enhancement methods including flipping, rotating, scaling, and changing brightness and contrast;
4.6 Initializing weights): training can be started from the beginning, and pre-training weights similar to tasks can be used as starting points for transfer learning;
4.7 Training model: training a customized YOLO model using the data set, configuration and initialization weights, monitoring the training process, and focusing on loss and average accuracy metrics, stopping training when performance on the validation set reaches a plateau or begins to drop;
4.8 Fine tuning model): if the performance of the model is not satisfactory, fine tuning the model by adjusting hyper-parameters, modifying the architecture, or adding more training data;
4.9 Evaluation model: evaluating the performance of the trained model by using the test set, and calculating an evaluation index;
4.10 A deployment model: once the custom YOLO model reaches satisfactory performance, it is deployed into the target environment;
4.11 Monitoring and updating): in practical applications, the performance of the model is continuously monitored, and new data is collected and annotated to solve the problem of discovery.
6. The LOTO informationized intelligent integrated security management method of claim 1, wherein in step 4), the lock visual identification comprises the steps of:
4.1 Pretreatment: the color selection of the lock body, the lock hook and the hasp board is respectively distinguished in hardware, and the selected color is the color which is not in the image environment; in terms of software, searching and focusing on an object area according to color characteristics on the whole image picture, and then calculating;
4.2 The outer ring of the hole of the hasp board of the lockset is coated with a color which is relatively larger than the color of the hasp board, and when the hasp board is padlock, the hole presents a C shape with the color as above; when the padlock is not adopted, the opening presents an O shape;
4.3 Visual recognition: the number of locks is judged by identifying the number of C-shaped and O-shaped locking plates; if the sum of the number of the C-shaped forms and the number of the O-shaped forms is not the number of the hasp plate holes, alarming a visual algorithm error; if the number of "C" types is less than the number of people in the field, alarm "LOTO violation".
7. The LOTO informatization intelligent integrated security management method of claim 1, wherein: in the step 4), during the visual identification of the lockset, the periphery of the hasp plate of the lockset is marked by lines with obvious colors, and then the quantity of the lockset is determined by identifying the number of separated segments of the lines with the obvious colors.
8. The LOTO informatization intelligent integrated security management method of claim 1, wherein: in the step 4), during visual identification of the lock, the hasp board is selected to be in an A color with higher saturation and brightness, the lock is selected to be in a black Fang Suogou, the lock body is selected to be in a non-A color, and the number of convex defects in the A color area of the hasp board is identified and calculated, namely the number of locks.
9. The LOTO informatization intelligent integrated security management method of claim 1, wherein: in step 6), the visual recognition of the device state is performed from two aspects: (1) when the locks are all opened or the number of the field locks is 0, the running state of the equipment is set; (2) the operating state of the device is when at least one lock is present in the field.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310650492.8A CN116844103A (en) | 2023-06-02 | 2023-06-02 | LOTO informatization intelligent integrated security management method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310650492.8A CN116844103A (en) | 2023-06-02 | 2023-06-02 | LOTO informatization intelligent integrated security management method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116844103A true CN116844103A (en) | 2023-10-03 |
Family
ID=88173410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310650492.8A Pending CN116844103A (en) | 2023-06-02 | 2023-06-02 | LOTO informatization intelligent integrated security management method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116844103A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117522351A (en) * | 2024-01-05 | 2024-02-06 | 武汉朗道工业设备有限公司 | Locking and listing method and system based on data analysis |
| CN118411385A (en) * | 2024-07-01 | 2024-07-30 | 武汉风行在线技术有限公司 | Multi-source data fusion video motion object detection and separation method |
-
2023
- 2023-06-02 CN CN202310650492.8A patent/CN116844103A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117522351A (en) * | 2024-01-05 | 2024-02-06 | 武汉朗道工业设备有限公司 | Locking and listing method and system based on data analysis |
| CN117522351B (en) * | 2024-01-05 | 2024-04-05 | 武汉朗道工业设备有限公司 | Locking and listing method and system based on data analysis |
| CN118411385A (en) * | 2024-07-01 | 2024-07-30 | 武汉风行在线技术有限公司 | Multi-source data fusion video motion object detection and separation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116844103A (en) | LOTO informatization intelligent integrated security management method | |
| US9911055B2 (en) | Method and system for detection and classification of license plates | |
| Yang et al. | A framework for improved video text detection and recognition | |
| US20120093362A1 (en) | Device and method for detecting specific object in sequence of images and video camera device | |
| CN112287875B (en) | Abnormal license plate recognition method, device, equipment and readable storage medium | |
| Shafi et al. | License plate identification and recognition in a non-standard environment using neural pattern matching | |
| Yang et al. | Anomaly detection in moving crowds through spatiotemporal autoencoding and additional attention | |
| Boekhoudt et al. | Hr-crime: Human-related anomaly detection in surveillance videos | |
| Manikandan et al. | A neural network aided attuned scheme for gun detection in video surveillance images | |
| Liu et al. | Video content analysis for compliance audit in finance and security industry | |
| Prabhu et al. | Recognition of Indian license plate number from live stream videos | |
| Nodari et al. | A Multi-Neural Network Approach to Image Detection and Segmentation of Gas Meter Counter. | |
| Agrawal et al. | ABGS Segmenter: pixel wise adaptive background subtraction and intensity ratio based shadow removal approach for moving object detection | |
| Choodowicz et al. | Hybrid algorithm for the detection and recognition of railway signs | |
| US11532158B2 (en) | Methods and systems for customized image and video analysis | |
| El-Said | Shadow aware license plate recognition system | |
| Mantini et al. | Camera Tampering Detection using Generative Reference Model and Deep Learned Features. | |
| Islam et al. | Automatic Vehicle Bangla License Plate Detection and Recognition | |
| Nivetha et al. | Video-object detection using background subtraction in spartan 3 fpga kit | |
| Ranjitha et al. | A review on text detection from multi-oriented text images in different approaches | |
| Singh et al. | Cloud-Based License Plate Recognition for Smart City Using Deep Learning | |
| Bennur et al. | Face Mask Detection and Face Recognition of Unmasked People in Organizations | |
| KR20200136583A (en) | System for searching numbers of vehicle and pedestrian based on artificial intelligence | |
| Moran et al. | Automatic Detection of Knives in Complex Scenes | |
| Parikh et al. | Number Plate Detection and Recognition Using OpenCV |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |