CN112528860A - Safety tool management method and system based on image recognition - Google Patents
Safety tool management method and system based on image recognition Download PDFInfo
- Publication number
- CN112528860A CN112528860A CN202011460022.8A CN202011460022A CN112528860A CN 112528860 A CN112528860 A CN 112528860A CN 202011460022 A CN202011460022 A CN 202011460022A CN 112528860 A CN112528860 A CN 112528860A
- Authority
- CN
- China
- Prior art keywords
- safety
- tool
- recognition
- returning
- tools
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 35
- 238000000034 method Methods 0.000 claims abstract description 22
- 238000001514 detection method Methods 0.000 claims description 42
- 238000012549 training Methods 0.000 claims description 18
- 238000013136 deep learning model Methods 0.000 claims description 13
- 230000000737 periodic effect Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 claims description 2
- 238000007689 inspection Methods 0.000 abstract description 3
- 238000000605 extraction Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 230000001815 facial effect Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000032683 aging Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000000877 morphologic effect Effects 0.000 description 2
- 239000003973 paint Substances 0.000 description 2
- 238000011056 performance test Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Abstract
The invention discloses a safety tool management method and a safety tool management system based on image recognition, which comprise the following steps: the user performs identity recognition and logs in the system; selecting a taking operation command/returning operation command; acquiring images of safety tools needing to be taken or returned; according to the collected image, identifying the safety tool; after the recognition is finished, outputting a recognition result, and sending an instruction whether to confirm taking/returning: generating a taking record/returning record after receiving an instruction for confirming taking/returning; if the fetch/return instruction is cancelled or the recognition fails, the steps are returned. The user realizes intelligent and information management of the safety tool through the control system client; by the method, the safety condition of the tool after use can be accurately judged, misjudgment caused by subjective factors and external environment influence in manual inspection is avoided, the possibility of existence of unsafe tools is reduced, and the life safety of workers is guaranteed.
Description
Technical Field
The invention belongs to the technical field of image recognition and tool management in machine vision, and particularly relates to a safety tool management method and system based on image recognition.
Background
Safety is the basis of all power production work, and no safety exists, and all the work can not be mentioned. Personal safety is more of a core part of safe production. The correct management and maintenance of safety tools is one of the most important means for ensuring personal safety. If the management of the tools is not standard, property loss and casualties can be caused.
At present, the main methods for managing safety tools and appliances in power production enterprises are characterized in that manual management is more, and management is performed by setting special storage management personnel or adopting a worker-independent management mode. The manual management is time-consuming and labor-consuming, the situations of omission and error registration are easy to occur, and the stability of taking, taking and returning the tools is difficult to ensure; moreover, after the safety tool is used, the safety tool is inevitably damaged or broken, and if any user does not feed back the use condition in time or a manager does not find an unsafe tool in time, potential safety hazards can be caused to the next user.
With the continuous development of computer information technology, the application of image recognition technology in various fields is becoming more extensive and popular, and even the technology gradually begins to infiltrate into the daily life of people, so that the image recognition technology should be fully utilized, the labor force is liberated, the work efficiency is provided, and the practical problems are solved.
Disclosure of Invention
Accordingly, the present invention is directed to a method and a system for managing safety tools based on image recognition, which are used to solve the problems of non-standard safety tool management and low efficiency of manual management in power generation enterprises.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
a safety tool management method based on image recognition comprises the following steps:
(1) the user performs identity recognition and logs in the system;
(2) selecting a taking operation command or a returning operation command;
(3) acquiring images of safety tools needing to be taken or returned;
(4) according to the collected image, identifying the safety tool;
(5) after the recognition is finished, outputting a recognition result, and sending an instruction whether to confirm taking or returning: when receiving an instruction for confirming taking or returning, generating a taking record or a returning record; and (4) if the instruction is cancelled or returned or the identification fails, returning to the step (2).
Related personnel or users firstly send login instructions through the client and log in the system after passing identity identification and verification. The client is generally a computer disposed at a set position, and generally has a display with a display function to realize human-computer interaction. After receiving the personnel login request, the client identifies the identity of the personnel, for example, the identity can be identified through facial features or fingerprint features, and then the identity is compared with the pre-stored features. The facial features can be extracted from the facial image acquired by the camera. The fingerprint features may be captured by an existing fingerprint reader for an image of the features, followed by feature extraction and identification. These operations may also be performed in an additional server.
The invention can simultaneously combine and use in-place sensors (such as a displacement sensor, an infrared sensor and the like), and further judge whether the safety tool returns to the designated position or not by combining signals of the sensors after receiving the command of confirming the return by the user. And if the return to the specified position is not successful, displaying an instruction of unsuccessful return, and prompting the user to carry out the return operation again. If successful return to the specified location. Giving a prompt of successful returning, generating a returning record at the same time, and uploading the returning record to a server for storage.
In the invention, an industrial camera is adopted to realize the image acquisition of safety tools and instruments, the image acquisition of human faces and the like. In order to ensure the image acquisition quality, the safety tool to be taken or returned can be arranged at a set position, so that the image acquisition quality and efficiency are ensured. For example, a camera is installed at a specific position in the safety tool storage room, and a face image and an image of the safety tool are captured.
The client transmits the basic information of the worker and the safety tool to the cloud server, and deploys a face library of the worker, a feature library of the tool and a safety detection library of the tool to the server (preferably, a cloud server can be adopted). Therefore, the client can be called conveniently at any time, and the recognition or calculation efficiency can be improved.
When identity recognition or safety tool recognition is performed, feature extraction is generally directly performed on an acquired face image or a safety tool image, and then the extracted features are compared with a face library of workers or a feature library of tools prestored on a server to obtain a recognition result.
When the safety tools are used, the images of the safety tools can be acquired at the same time, and the safety tools can be identified and used at the same time; when returning the safety tools, the image acquisition of the safety tools needs to be performed one by one, and the identification and the return of the safety tools to be returned are performed one by one.
When the safety tools are used, workers select the required tools from the safety tool storage area and place the tools at the appointed position, and the workers log in the system by identifying the face to verify the identity; furthermore, after the camera detects a safety tool, the camera photographs a position area where the tool is placed and uploads the position area to the cloud server, the tool to be taken is identified, the tool needing to be subjected to safety detection is provided with a safety rate result, a user can select according to the safety rate of the tool, after the worker inputs and confirms a taking operation instruction, the generated taking record is transmitted to the cloud server, and if the worker inputs and cancels the operation instruction, the step is carried out again.
When returning the safety tools, putting the tools to be returned to a designated position, returning one by one, identifying the returned tools, outputting an identification result for the tools to be subjected to safety detection, updating the safety rate of the tools, generating a return record, and transmitting the generated return record to a cloud server after a worker inputs a return operation confirmation instruction; if the worker inputs an operation canceling instruction, the step is carried out again;
further, the method for updating the safety rate of the safety tool during the returning operation is as follows:
S＝aSA+bSP+cSF
wherein: sAThe appearance integrity of the safety tool; sPA parameter of the proximity of the periodic detection period of the safety tool; sFThe frequency of use of safety tools; a. b and c are corresponding weight coefficients, and a is more than b and more than c; the safety rate can be S, or S is subjected to simple equivalent conversion to obtain data in other expression forms, such as percentage, a numerical value between 0 and 1, a numerical value between 0 and 100 and the like. a. The ratio of b to c is: (5 to 7): 2 to 4): 1.
For convenience of representation, for example, the appearance integrity of the safety tool, the parameter near the periodic detection limit, and the frequency of use may be represented by numerical values of 0 to 100 points. a + b + c is 1, so that S is obtained as a value of 0 to 100, and then the value can be directly compared with a set threshold value.
And when the safety detection result of the tool is in a disqualified state, or the annual salary of the tool is exceeded, or the obtained safety rate result is lower than a set threshold value, sending a processing request to an administrator. For the safety tool which is not processed in time, a warning for forbidding borrowing is sent to the worker. The threshold value can be set empirically, and the threshold value setting is different for different tools and different safety requirements due to different use occasions.
When the invention is used for face recognition or safety tool recognition, the existing image recognition technology can be adopted, such as the following steps: the method comprises the steps of image acquisition, image preprocessing, image segmentation (or target image interesting region extraction), target feature extraction, comparison with a pre-loaded feature library, comparison result obtaining and the like.
During returning operation, after the safety tools needing safety detection are identified, a safety rate detection link is automatically carried out; the safety rate detection link can adopt a deep learning model for identification.
Preferably, the deep learning model is constructed by adopting the following method:
(1) collecting safety tool images with different wear degrees or different wear types, preprocessing the images, and constructing a training sample set;
(2) classifying the images in the training sample set according to the wear degree, and marking classification labels corresponding to the images;
(3) and training the constructed deep learning model by utilizing the training sample set and the corresponding classification label information to obtain the optimal parameters of the model.
In the present invention, the image preprocessing includes, but is not limited to, one or more of a series of preprocessing operations such as image graying, image denoising, binarization processing, morphological processing, and the like.
In the invention, the deep learning model can adopt a VGG16 deep convolution neural network. In the training stage, a training sample set is directly adopted to pre-train model migration, the spatial characteristics of the tools are extracted through a convolution network, then the full connection layer of the network is modified to enable the full connection layer to meet the classification requirements of corresponding tool labels, and the optimal operation parameters of the model are obtained after the model is trained. And then putting the tool image needing to be judged into the trained model, predicting the corresponding label classification, and further outputting the wear to which the tool image belongs.
Based on the same inventive concept, in another aspect of the present invention, there is provided an image recognition-based security tool management system, including:
a camera: acquiring images of the face of a user or a safety tool needing to be taken or returned, and sending the images to a client;
a client: receiving a user instruction; uploading the acquired image to a server; displaying the identification result of the safety tool;
a server: processing the received image to complete face recognition or recognition of safety tools and instruments, and transmitting the recognition result to the client; generating a taking record or a returning record, and calculating the safety rate of the tools.
Preferably, the server sends the taking record, the returning record and the safety rate result of the safety tool to the client according to the instruction of the client.
In the server, a face library feature library, a tool feature library and a tool safety detection library are deployed, face recognition tool identification and tool safety detection are processed, basic information and historical records of workers (users) and safety tools are stored, and calling instructions of a client side are received;
the client identifies the identity of a user through a face recognition function and enters an operation interface of the client; and finishing man-machine interaction operation. After logging in, the system management personnel can check the use records and the safety rate results of the safety tools. The worker can take and return the safety tool after logging in the system, and can also manually record the use condition of the tool, such as whether the insulating part has cracks, aging and paint peel falling, and whether the fixedly connected part has phenomena of looseness, corrosion, fracture and the like;
according to the safety tool management method and system based on image recognition, users can realize intelligent and information management of safety tools through the control system client, the users do not need abundant working experience, only simple software operation is needed, the detailed recording of the use condition of the safety tools can be completed without errors, and the stability of taking and returning the tools is improved; by using the intelligent safety tool management method, the safety condition of the tool after use can be accurately judged, misjudgment caused by subjective factors and external environment influence in manual inspection is avoided, the possibility of existence of unsafe tools is reduced, and the life safety of workers is guaranteed.
Drawings
Fig. 1 is a flow chart of a security tool management method based on image recognition according to the present invention.
Fig. 2 is a block diagram of a process for detecting the safety of a tool based on a deep learning model according to the present invention.
Detailed Description
In order to more specifically describe the present invention, the following detailed description is provided for the technical solution of the present invention with reference to the accompanying drawings and the specific embodiments.
As shown in fig. 1, a method for managing a security tool based on image recognition includes the following steps:
the safety tool management method of the invention can only comprise the taking operation step, can also comprise the returning operation step in an emergency, and can also comprise the taking operation step and the returning operation step at the same time. In this embodiment, a management method including both a safety tool taking operation step and a return operation step will be described.
A safety tool management method mainly comprises the following steps:
(1) a user logs in the system and carries out identity recognition;
(2) selecting a taking operation command or a returning operation command;
(3) acquiring images of safety tools needing to be taken or returned;
(4) according to the collected image, identifying the safety tool;
(5) after the recognition is finished, outputting a recognition result, and sending an instruction whether to confirm taking or returning: when receiving an instruction for confirming taking or returning, generating a taking record or a returning record; and (4) if the instruction is cancelled or returned or the identification fails, returning to the step (2).
Specifically, when a user needs to take a safety tool, a login instruction is sent through a client, the user logs in a system after passing identity identification verification, firstly, a taking operation command is selected, then, an image of the safety tool to be taken is collected, and the tool is identified; after the recognition is finished, outputting a recognition result, and sending out an instruction whether to confirm the taking or not: generating a taking record after receiving an instruction for confirming taking; if the taking instruction is cancelled or the recognition fails, returning to the step (2);
when a user needs to return the safety tool, after logging in the system, firstly selecting a returning operation command, then acquiring an image of the safety tool to be taken, and identifying the tool; after the identification is finished, outputting an identification result, and sending an instruction whether to confirm the return: generating a returning record after receiving an instruction for confirming returning; and (5) if the return instruction is cancelled or the identification fails, returning to the step (2).
The safety tool management method can be completed by a separately arranged industrial computer with a man-machine interaction function (display). The method can also be finished by combining a remote server and a client, and the client is mainly used for man-machine interaction, image acquisition, result output and the like. Face recognition, tool and instrument recognition and the like can be completed in the server, so that the operation efficiency can be greatly improved. And meanwhile, the requirement on the client is greatly reduced. The present invention is illustrated in the context of a server and client federated system. The server may be a common public server or a server provided by the enterprise itself. The present invention is described with reference to a cloud server as an example.
Firstly, mounting a camera at a specific position in a safety tool storage room in advance, wherein the camera is used for capturing a face image or an image of a safety tool;
when a user (generally, a specific worker or a technician) performs a fetching or returning step, a user login operation needs to be performed at a client first, and when the user logs in, the identity of the user needs to be identified. In this embodiment, a face recognition method is used to identify the user. When the user identity is identified, firstly, a face image of the user is obtained through a camera, then the face image is uploaded to a server, the server extracts the face features of the user, and the extracted face features are compared with a face feature library loaded in advance on a cloud server to complete the identification of the user identity;
if the user needs to take the safety tool, then:
(1-1) selecting a taking operation command by a user on a client;
(1-2) a user places a safety tool to be taken at a designated position, a client outputs an instruction, and a camera collects images of the safety tool to be taken, wherein the safety tool can be multiple or a single safety tool;
(1-3) the client transmits the obtained safety tool image to a server, performs feature extraction on the tool image in the server, compares the extracted features with a tool feature library loaded in advance on a cloud server to complete identification of the safety tool, and outputs an identification result to the client;
(1-4) after receiving the identification result, the client sends out an instruction whether to confirm the taking: when receiving an instruction for confirming taking or returning, generating a taking record, and uploading the taking record to a server for storage; and (5) if the taking instruction is cancelled or the recognition fails, returning to the step (1-2) to collect and recognize the image again and the like.
For the tool needing to perform security detection, a serial number is processed on the tool in advance, the corresponding serial number is identified while the tool is identified, the security rate information corresponding to the serial number is called at the same time, the security rate data is transmitted to the client, and the security rate data and the identification result are displayed at the same time for the user to refer to as shown in fig. 2. If the user needs to return the safety tool, then:
(2-1) the user on the client selects a return operation command;
(2-2) the user places the safety tools to be returned at an appointed position, the client outputs an instruction, the camera collects images of the safety tools to be returned, and the safety tools need to be returned one by one;
(2-3) the client transmits the obtained safety tool image to a server, performs feature extraction on the tool image in the server, compares the extracted features with a tool feature library loaded in advance on the cloud server to complete identification of the safety tool, and outputs an identification result to the client;
(2-4) after receiving the identification result, the client sends out an instruction whether to confirm the return: after receiving the instruction of confirming the return, generating a return record, and uploading the return record to a server for storage; and (5) if the return instruction is cancelled or the recognition fails, returning to the step (2-2) to collect and recognize the image again and the like.
For the kind of tools and instruments which need to be subjected to safety detection after being used, such as safety helmets, insulating gloves, insulating boots and the like, the latest safety rate is given after the tools and instruments are identified, and after the corresponding serial numbers of the tools and instruments which need to be subjected to safety detection are identified during actual detection, the method directly enters a safety detection step:
the appearance integrity of the tool needing safety detection is detected by using the established deep learning model, and meanwhile, the current periodical detection deadline approach parameter and the use frequency of the tool are comprehensively considered, so that a safety rate result is obtained. Specifically, the method for updating the safety rate of the tool comprises the following steps:
S＝aSA+bSP+cSF
wherein: sAThe appearance integrity of the safety tool; sPA parameter of the proximity of the periodic detection period of the safety tool; sFThe frequency of use of safety tools; a. b and c are corresponding weight coefficients, and a is more than b and more than c; the safety rate can be S, or the S is converted into data in other expression forms through simple equivalent conversion; in the present embodiment, a is 0.6, b is 0.3, and c is 0.1;
the appearance integrity, the close parameter of the periodic detection period, the use frequency and the like of the safety tool are all values of 0-100. Wherein the appearance integrity result is directly obtained by a deep learning model constructed by the system at each return. The parameter of the proximity of the periodic detection deadline can be manually input by related personnel, and can also be automatically generated directly through system programming; for example, when an administrator sets an initial value initially, since it meets the national standard, we set it to 100 points, and assume that the inspection period is 360 days, we subtract 0.27 points every 1 day, and thus add to the safety rate calculation. As for the use frequency, the use frequency can be input manually or generated automatically by a system, for example, different copies can be set according to the use times. All the reasons for this are derived from our hypothesis, and the frequent use reduces the safety performance of the tool. The use frequency can be calculated by superposition and updated at each return. When the original results of the parameter close to the periodic detection deadline, the use frequency and the like are not data within 0-100, the parameter close to the periodic detection deadline and the use frequency can be converted into numerical values within 100 through the existing simple conversion.
The obtained safety rate is a value of 0-100, a specific threshold value can be given according to the use requirements, safety requirements and the like of different safety tools, and when the obtained safety rate result is lower than the set threshold value, a processing request is sent to an administrator to ensure the use safety of the safety tools.
The present embodiment is described by taking a helmet as an example, but the present invention is not limited to the helmet, and the insulating glove and the insulating boot can also be used for such detection. Because the appearance of the safety helmet is not enough to ensure the normal use of the safety helmet, which is the most direct judgment mode that can be realized in real life, whether the safety helmet can be used continuously is also related to other performance tests, such as basic performance and special performance tests (safety helmet standards), when the safety rate is calculated, information such as the proximity parameter of the periodic detection period, the use frequency and the like is considered, and the judgment accuracy is further improved.
For example, the ratio of a, b and c is 6:3:1, if the safety helmet is intact and not used, but the periodic detection period date passes, the periodic detection deadline approaching parameter is 0, the values of the three key parameters are (100,0 and 100), and a borrow prohibition warning can be directly given. Similarly, borrow prohibition warnings can also be given directly when the appearance integrity detection result is not required. If the calculated safety ratio is lower than the set safety threshold (different safety tools can be set with different thresholds, such as 80% for high demand, such as safety helmet, 70% for insulating boot, etc.). At this time, the safety rate of the safety helmet is considered to be insufficient, and management personnel are required to perform timely treatment.
The deep learning model is constructed by adopting the following method:
(1) collecting safety tool images with different wear degrees or different wear types, preprocessing the images, and constructing a training sample set;
(2) classifying the images in the training sample set according to the wear degree, and marking classification labels corresponding to the images;
(3) and training the constructed deep learning model by utilizing the training sample set and the corresponding classification label information to obtain the optimal parameters of the model.
Similarly use the safety helmet as an example, during the actual experiment, collect safety helmet picture material and carry out categorised arrangement, the classification here is artifical classification, divide into four types according to the safety helmet apparent degree of wear: 1, complete and no scratch; 2, complete with slight scratches (few scratches); 3. severe scratches (more scratches) are intact; 4 incomplete (incomplete); from these four classes, we give fractional segments, 1-100; 2 to 90; 3 to 70 percent; 4-50; four types of safety helmets, wherein each type of safety helmet comprises 500 pictures and is marked with a classification label; forming a training sample set and corresponding label data;
secondly, performing model training by using the existing VGG16 deep convolutional neural network, learning by using a training data set, extracting the spatial characteristics of the safety helmet through the convolutional network, and modifying the full connection layer of the network to enable the full connection layer to meet the classification requirement of the integrity of the safety helmet; finally, training the model; obtaining the optimal parameters of the model;
and finally, after the model is trained, putting the picture of the safety helmet to be judged into the trained model, and predicting the corresponding type.
In this embodiment, when the face is identified or the safety tool is identified, the identification may be completed by a method shown in the following steps:
a. and (6) image acquisition. Placing the safety tools to be taken or returned to a designated position one by one, and uploading the photographed safety tools to a cloud server by a camera; or acquiring a face image and uploading the face image to a cloud server;
b. and (5) image preprocessing. Converting the obtained image into a gray level image, and performing a series of preprocessing operations such as image denoising, binarization processing, morphological processing and the like;
c. and (5) image segmentation. Segmenting the preprocessed image from the image by using a segmentation method; or the preprocessed image is segmented from the image by a segmentation method;
d. and identifying human faces and safety tools. Extracting the features of the segmented face image and the tool image, and comparing the extracted features with a face feature library and a tool feature library which are trained in advance, so as to identify the user identity and the safety tool;
similarly, for the safety tools with numbers, namely the tools which need to be subjected to safety detection after use, the extracted features are compared with the tool safety detection library trained in advance to give the safety detection results of the tools;
based on the same inventive concept, in another aspect of the present invention, there is provided an image recognition-based security tool management system, including:
(1) the camera is used for shooting a face image of a worker in real time to perform face recognition and shooting a safety tool image in a designated area to perform tool recognition;
(2) the cloud server is used for deploying a face library, a tool feature library and a tool safety detection library, processing face recognition, tool recognition and tool safety detection, storing basic information and historical records of workers and safety tools and receiving a client-side calling instruction;
(3) the client identifies the identity of a user through a face recognition function and enters an operation interface of the client;
after logging in the system, a manager can check the use records and the safety rate results of the safety tools;
after logging in the system, a worker can take and return the safety tool, and can also manually record the use condition of the tool, such as whether the insulating part has cracks, aging and paint peel falling, and whether the fixedly connected part has phenomena of looseness, corrosion, fracture and the like;
the server is provided with an image recognition module, firstly, collected images are preprocessed to segment objects to be recognized, then, the segmented images are subjected to feature extraction, and data comparison is carried out on the segmented images and a face library, a feature library of a tool and a safety detection library which are trained in advance to obtain recognition results; the image recognition module hardware can adopt an industrial control computer, and the core part is computer software for image recognition and writing.
The server is provided with a numerical value calculation module, the safety rate is calculated according to the safety detection result after the tool is used, the periodic test recording result and the taking times, and the data calculation module mainly depends on an industrial control computer and computer software written by a core part.
The system supports historical record query, worker name and working date input, historical records of worker taking and returning and safety rates of various tools and appliances are extracted from a cloud server, and the core part is computer software which is written.
Claims (10)
1. A safety tool management method based on image recognition is characterized by comprising the following steps:
(1) the user performs identity recognition and logs in the system;
(2) selecting a taking operation command/returning operation command;
(3) acquiring images of safety tools needing to be taken or returned;
(4) according to the collected image, identifying the safety tool;
(5) after the recognition is finished, outputting a recognition result, and sending an instruction whether to confirm taking/returning: generating a taking record/returning record after receiving an instruction for confirming taking/returning; and (5) if the taking/returning instruction is cancelled or the recognition fails, returning to the step (2).
2. The method for managing safety tools based on image recognition as claimed in claim 1, wherein when a safety tool is taken out, a plurality of safety tools are simultaneously captured, and the safety tools are simultaneously recognized and taken out; when returning the safety tools, the image acquisition of the safety tools needs to be performed one by one, and the identification and the return of the safety tools to be returned are performed one by one.
3. The method for managing a safety tool based on image recognition according to claim 1, wherein when a safety tool is taken out, a recognition result is directly output to a safety tool which does not require safety detection, and a command for confirming taking out is issued; for a safety tool which needs to be subjected to safety detection, an identification result and a safety rate result are simultaneously output, and a command for confirming taking or not is issued.
4. The method for managing a safety tool based on image recognition according to claim 3, wherein when returning the safety tool, the identification result is directly output to the safety tool which does not require the safety detection, and a command for confirming the return is issued; and outputting the identification result of the appliance needing to be subjected to the safety detection, updating the safety rate of the safety appliance, and sending a command for confirming whether to return.
5. The image recognition-based safety tool management method according to claim 4, wherein the built deep learning model is used for detecting the appearance integrity of the safety tool needing safety detection, and the current periodical detection deadline approach parameter and the use frequency of the safety tool are comprehensively considered to obtain a safety rate result.
6. The method for managing a security tool based on image recognition according to claim 5, wherein the method for updating the security rate of the security tool comprises:
S＝aSA+bSP+c SF
wherein: sAThe appearance integrity of the safety tool; sPA parameter of the proximity of the periodic detection period of the safety tool; sFThe frequency of use of safety tools; a. b and c are corresponding weight coefficients, and a is more than b and more than c; the safety rate can be S, or the S is converted into data in other expression forms through simple equivalent conversion;
and when the obtained safety rate result is lower than the set threshold value, sending a processing request to an administrator.
7. The image recognition-based safety tool management method according to claim 5, wherein the deep learning model is constructed by adopting the following method:
(1) collecting safety tool images with different wear degrees or different wear types, preprocessing the images, and constructing a training sample set;
(2) classifying the images in the training sample set according to the wear degree, and marking classification labels corresponding to the images;
(3) and training the constructed deep learning model by utilizing the training sample set and the corresponding classification label information to obtain the optimal parameters of the model.
8. The image recognition-based security tool management method of claim 7, wherein the deep learning model employs a VGG16 deep convolutional neural network.
9. An image recognition-based security tool management system, comprising:
the camera is used for acquiring images of the face of a user or a safety tool needing to be taken or returned and sending the images to the client;
a client: receiving a user instruction; uploading the received image to a server; displaying the identification result of the safety tool;
a server: processing the received image to complete face recognition or recognition of safety tools and instruments, and transmitting the recognition result to the client; a take or return record is generated.
10. The image recognition-based security tool management system of claim 9, wherein the server sends the access record, the return record and the security rate result of the security tool to the client according to the instruction of the client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011460022.8A CN112528860A (en) | 2020-12-11 | 2020-12-11 | Safety tool management method and system based on image recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011460022.8A CN112528860A (en) | 2020-12-11 | 2020-12-11 | Safety tool management method and system based on image recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112528860A true CN112528860A (en) | 2021-03-19 |
Family
ID=74999175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011460022.8A Pending CN112528860A (en) | 2020-12-11 | 2020-12-11 | Safety tool management method and system based on image recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528860A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168489A (en) * | 2023-02-10 | 2023-05-26 | 深圳市智品格科技有限公司 | Self-service return method, device, equipment and storage medium for transaction objects |
| CN116663940A (en) * | 2023-08-01 | 2023-08-29 | 安徽博诺思信息科技有限公司 | Substation safety tool management system and management method |
| CN116993274A (en) * | 2023-09-28 | 2023-11-03 | 江苏中车数字科技有限公司 | Workshop tool management method and system |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106447856A (en) * | 2016-09-21 | 2017-02-22 | 国网江西省电力公司莲花县供电分公司 | Intelligent and safe tool cabinet system and management method thereof |
| CN107657603A (en) * | 2017-08-21 | 2018-02-02 | 北京精密机电控制设备研究所 | A kind of industrial appearance detecting method based on intelligent vision |
| CN108830263A (en) * | 2018-08-13 | 2018-11-16 | 西安艾锐智能科技有限公司 | intelligent tool management system and tool management cabinet |
| CN109255787A (en) * | 2018-10-15 | 2019-01-22 | 杭州慧知连科技有限公司 | Silk ingot scratch detection system and method based on deep learning and image processing techniques |
| CN110415214A (en) * | 2019-06-26 | 2019-11-05 | 北京迈格威科技有限公司 | Appearance detecting method, device, electronic equipment and the storage medium of camera module |
| CN110705662A (en) * | 2019-09-29 | 2020-01-17 | 武汉里得电力科技股份有限公司 | Image recognition technology-based tool management system and method |
| CN111489113A (en) * | 2020-02-28 | 2020-08-04 | 深圳市康拓普信息技术有限公司 | Safety tool warehouse management method and system |
| CN111610741A (en) * | 2020-06-05 | 2020-09-01 | 上海国泉科技有限公司 | Tool supervision system and method |
| CN111720013A (en) * | 2020-05-27 | 2020-09-29 | 国网安徽省电力有限公司五河县供电公司 | Intelligent door of safety tool storehouse |
-
2020
- 2020-12-11 CN CN202011460022.8A patent/CN112528860A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106447856A (en) * | 2016-09-21 | 2017-02-22 | 国网江西省电力公司莲花县供电分公司 | Intelligent and safe tool cabinet system and management method thereof |
| CN107657603A (en) * | 2017-08-21 | 2018-02-02 | 北京精密机电控制设备研究所 | A kind of industrial appearance detecting method based on intelligent vision |
| CN108830263A (en) * | 2018-08-13 | 2018-11-16 | 西安艾锐智能科技有限公司 | intelligent tool management system and tool management cabinet |
| CN109255787A (en) * | 2018-10-15 | 2019-01-22 | 杭州慧知连科技有限公司 | Silk ingot scratch detection system and method based on deep learning and image processing techniques |
| CN110415214A (en) * | 2019-06-26 | 2019-11-05 | 北京迈格威科技有限公司 | Appearance detecting method, device, electronic equipment and the storage medium of camera module |
| CN110705662A (en) * | 2019-09-29 | 2020-01-17 | 武汉里得电力科技股份有限公司 | Image recognition technology-based tool management system and method |
| CN111489113A (en) * | 2020-02-28 | 2020-08-04 | 深圳市康拓普信息技术有限公司 | Safety tool warehouse management method and system |
| CN111720013A (en) * | 2020-05-27 | 2020-09-29 | 国网安徽省电力有限公司五河县供电公司 | Intelligent door of safety tool storehouse |
| CN111610741A (en) * | 2020-06-05 | 2020-09-01 | 上海国泉科技有限公司 | Tool supervision system and method |
Non-Patent Citations (1)
| Title |
|---|
| 侯峰等: "电力安全工器具智能管理系统的研发及应用", 《宁夏电力》, no. 02, 28 April 2020 (2020-04-28) * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116168489A (en) * | 2023-02-10 | 2023-05-26 | 深圳市智品格科技有限公司 | Self-service return method, device, equipment and storage medium for transaction objects |
| CN116663940A (en) * | 2023-08-01 | 2023-08-29 | 安徽博诺思信息科技有限公司 | Substation safety tool management system and management method |
| CN116663940B (en) * | 2023-08-01 | 2023-10-20 | 安徽博诺思信息科技有限公司 | Substation safety tool management system and management method |
| CN116993274A (en) * | 2023-09-28 | 2023-11-03 | 江苏中车数字科技有限公司 | Workshop tool management method and system |
| CN116993274B (en) * | 2023-09-28 | 2023-12-19 | 江苏中车数字科技有限公司 | Workshop tool management method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112528860A (en) | Safety tool management method and system based on image recognition | |
| CN111460889A (en) | Abnormal behavior identification method, device and equipment based on voice and image characteristics | |
| CN108319926A (en) | A kind of the safety cap wearing detecting system and detection method of building-site | |
| CN111814775B (en) | Target object abnormal behavior identification method, device, terminal and storage medium | |
| CN111191532A (en) | Face recognition method and device based on construction area and computer equipment | |
| CN115330129A (en) | Enterprise safety risk early warning analysis method and system | |
| CN112949457A (en) | Maintenance method, device and system based on augmented reality technology | |
| CN115223249A (en) | Quick analysis and identification method for unsafe behaviors of underground personnel based on machine vision | |
| CN113168528A (en) | Continuous learning image stream processing system | |
| CN111177468A (en) | Laboratory personnel unsafe behavior safety inspection method based on machine vision | |
| CN113807240A (en) | Intelligent transformer substation personnel dressing monitoring method based on uncooperative face recognition | |
| CN117171694A (en) | Distribution scene safety identification system based on AI technology | |
| CN112218044A (en) | Analysis system and analysis method with security inspection artificial intelligence recognition function | |
| KR102230559B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN109841140A (en) | Teaching machine and its operating method, computing device and computer-readable medium | |
| CN206741505U (en) | Identify management equipment and device | |
| KR102342495B1 (en) | Method and Apparatus for Creating Labeling Model with Data Programming | |
| CN114429677A (en) | Coal mine scene operation behavior safety identification and assessment method and system | |
| CN114241400A (en) | Monitoring method and device of power grid system and computer readable storage medium | |
| CN112686156A (en) | Emotion monitoring method and device, computer equipment and readable storage medium | |
| CN116579609B (en) | Illegal operation analysis method based on inspection process | |
| CN116630752B (en) | Construction site target object identification method and device based on AI algorithm | |
| CN110135353B (en) | Method and system for eliminating field fingerprint and palm print of owner and related people | |
| CN114841656B (en) | Military aircraft fault detection method and system based on edge calculation | |
| CN112347824A (en) | Wearing object identification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: No.217 Taizhou Road, gongchenqiao, Hangzhou, Zhejiang, 310015 Applicant after: Zhejiang Huadian Equipment Testing Research Institute Co.,Ltd. Address before: No.217 Taizhou Road, Gongshu District, Hangzhou City, Zhejiang Province Applicant before: ZHEJIANG HUADIAN EQUIPMENT TESTING INSTITUTE Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210319 |