CN116385962A - Personnel monitoring system in corridor based on machine vision and method thereof - Google Patents
Personnel monitoring system in corridor based on machine vision and method thereof Download PDFInfo
- Publication number
- CN116385962A CN116385962A CN202310292036.0A CN202310292036A CN116385962A CN 116385962 A CN116385962 A CN 116385962A CN 202310292036 A CN202310292036 A CN 202310292036A CN 116385962 A CN116385962 A CN 116385962A
- Authority
- CN
- China
- Prior art keywords
- classification
- personnel
- corridor
- feature map
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 114
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000001514 detection method Methods 0.000 claims abstract description 52
- 238000012545 processing Methods 0.000 claims abstract description 31
- 238000013527 convolutional neural network Methods 0.000 claims description 47
- 238000009826 distribution Methods 0.000 claims description 42
- 239000011159 matrix material Substances 0.000 claims description 35
- 238000010586 diagram Methods 0.000 claims description 29
- 238000012937 correction Methods 0.000 claims description 21
- 239000013598 vector Substances 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 10
- 238000012546 transfer Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 239000003245 coal Substances 0.000 abstract description 25
- 238000004519 manufacturing process Methods 0.000 abstract description 15
- 238000005516 engineering process Methods 0.000 abstract description 7
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 238000005065 mining Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 11
- 238000003860 storage Methods 0.000 description 11
- 238000004590 computer program Methods 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 8
- 238000013135 deep learning Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 5
- 239000000428 dust Substances 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 4
- 239000002245 particle Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000007792 addition Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000004873 anchoring Methods 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000014759 maintenance of location Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 230000035939 shock Effects 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The utility model relates to an intelligent monitoring technology field, it specifically discloses a personnel monitored control system and method in corridor based on machine vision, and it adopts the artificial intelligence monitored control technology based on machine vision, regards as the monitored image of personnel in the corridor as input data to carry out the frame of target region of interest to personnel's target in the monitored image, and further carry out showing target detection coding after carrying out image high definition processing, in order to extract the small-size implicit characteristic information about whether personnel wear the helmet in the monitored image based on target frame and high definition processing, and carry out with this whether the classification judgement of whether personnel wear the helmet, through this kind of mode, can carry out accurate monitoring judgement to personnel wear the helmet in the corridor to guarantee coal conveying system's safety in production.
Description
Technical Field
The application relates to the technical field of intelligent monitoring, and more particularly relates to a personnel monitoring system in a gallery based on machine vision and a method thereof.
Background
The existing coal mining industry has a great potential safety hazard in the process of mining raw coal, so that workers are required to wear safety caps when mining the coal, and protective measures are taken to ensure the safety in the mining process. The safety helmet is a shallow dome cap made of raw materials with high strength and good shock resistance, and can prevent the head from being hurt by an impact object. The safety helmet mainly comprises a helmet shell, a helmet liner, a lower cheek belt and a rear hoop, and a certain space is reserved between the helmet shell and the helmet liner, so that instantaneous impact force can be buffered and dispersed, and the damage to the head can be avoided or reduced. Whether the safety helmet is worn directly influences the life safety of the operator, so whether the safety helmet of the operator is worn or not needs to be monitored.
However, because of the special environment of the coal conveying corridor, some dangerous factors and phenomena against safety regulations, such as the phenomenon that personnel in the corridor do not wear safety helmets, the phenomena of timing inspection of site personnel, industrial televisions and personnel (robots) cannot be found and processed in time, hidden dangers are buried for the safe production of the coal conveying system, and the dangerous degree of site personnel is greatly increased. In addition, in the detection of wearing the safety helmet for personnel in a corridor, most of the detection is carried out by means of manual observation and monitoring by workers, so that the efficiency is low, and the detection accuracy of the workers who do not wear the safety helmet is low.
Accordingly, an optimized in-aisle personnel monitoring system is desired that can accurately monitor whether in-aisle personnel wear safety helmets to ensure safe production of coal conveying systems.
Disclosure of Invention
The present application has been made in order to solve the above technical problems. The embodiment of the application provides a personnel monitoring system in a corridor and a method thereof based on machine vision, which adopt an artificial intelligent monitoring technology based on machine vision, take a monitoring image of personnel in the corridor as input data to frame a target region of interest of personnel targets in the monitoring image, and further carry out remarkable target detection coding after carrying out image high-definition processing so as to extract small-size implicit characteristic information about whether personnel wear safety caps or not in the monitoring image based on the target frame and the high-definition processing, and carry out classification judgment about whether the personnel wear the safety caps or not by adopting the method, so that whether the personnel wear the safety caps or not in the corridor can be accurately monitored and judged, and the safety production of a coal conveying system is ensured.
According to one aspect of the present application, there is provided an in-corridor personnel monitoring system based on machine vision, comprising:
The monitoring module is used for acquiring personnel monitoring images in the corridor acquired by the camera;
the target detection module is used for passing the personnel monitoring image in the corridor through a personnel target detection network to obtain a target region of interest;
the image enhancement module is used for enabling the target region of interest to pass through a high-definition image generator based on an countermeasure generation network so as to obtain a generated target region of interest;
the image feature extraction module is used for obtaining a classification feature map by using a convolutional neural network model containing a significant target detector for the generated target region of interest;
the characteristic distribution correction module is used for carrying out characteristic distribution correction on each characteristic matrix of the classification characteristic map along the channel dimension so as to obtain a corrected classification characteristic map; and
and the monitoring result generation module is used for enabling the corrected classification characteristic diagram to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether personnel wear safety helmets.
According to another aspect of the present application, there is provided a machine vision-based in-corridor personnel monitoring method, comprising:
acquiring personnel monitoring images in a corridor acquired by a camera;
the personnel monitoring images in the corridor pass through a personnel target detection network to obtain a target region of interest;
The target region of interest passes through a high-definition image generator based on an countermeasure generation network to obtain a generated target region of interest;
the generated target region of interest is subjected to convolutional neural network model containing a remarkable target detector to obtain a classification characteristic diagram;
performing feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map; and
and passing the corrected classification characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether a person wears the safety helmet.
According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform the machine vision based intra-corridor personnel monitoring method as described above.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the machine vision based intra-corridor personnel monitoring method as described above.
Compared with the prior art, the in-corridor personnel monitoring system and the method based on the machine vision adopt an artificial intelligent monitoring technology based on the machine vision, a monitoring image of personnel in the corridor is used as input data to frame a target region of interest of the personnel target in the monitoring image, and after high-definition processing of the image, significant target detection coding is further carried out, so that small-size implicit characteristic information about whether personnel wear a safety helmet or not in the monitoring image based on the target frame and the high-definition processing is extracted, and classification judgment about whether the personnel wear the safety helmet or not is carried out by the personnel, and in such a way, whether the personnel wear the safety helmet or not in the corridor can be accurately monitored and judged, so that the safety production of a coal conveying system is ensured.
Drawings
The foregoing and other objects, features and advantages of the present application will become more apparent from the following more particular description of embodiments of the present application, as illustrated in the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.
Fig. 1 is an application scenario diagram of a machine vision based personnel monitoring system in a corridor and a method thereof according to an embodiment of the present application.
Fig. 2 is a block diagram schematic diagram of a machine vision based personnel in-corridor monitoring system and method thereof, in accordance with an embodiment of the present application.
Fig. 3 is a block diagram of a monitoring result generation module in a machine vision based in-corridor personnel monitoring system according to an embodiment of the present application.
Fig. 4 is a flow chart of a method of machine vision based in-corridor personnel monitoring, in accordance with an embodiment of the present application.
Fig. 5 is a schematic diagram of a system architecture of a machine vision based in-corridor personnel monitoring method, according to an embodiment of the present application.
Fig. 6 is a flowchart of a method for monitoring personnel in a gallery based on machine vision according to an embodiment of the application, wherein the corrected classification feature map is passed through a classifier to obtain a classification result.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application and not all of the embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Scene overview
As described above, in the current coal mining industry, because of a large potential safety hazard in the process of mining raw coal, workers are required to wear safety helmets when mining coal, and protective measures are taken to ensure safety in the mining process. The safety helmet is a shallow dome cap made of raw materials with high strength and good shock resistance, and can prevent the head from being hurt by an impact object. The safety helmet mainly comprises a helmet shell, a helmet liner, a lower cheek belt and a rear hoop, and a certain space is reserved between the helmet shell and the helmet liner, so that instantaneous impact force can be buffered and dispersed, and the damage to the head can be avoided or reduced. Whether the safety helmet is worn directly influences the life safety of the operator, so whether the safety helmet of the operator is worn or not needs to be monitored.
However, because of the special environment of the coal conveying corridor, some dangerous factors and phenomena against safety regulations, such as the phenomenon that personnel in the corridor do not wear safety helmets, the phenomena of timing inspection of site personnel, industrial televisions and personnel (robots) cannot be found and processed in time, hidden dangers are buried for the safe production of the coal conveying system, and the dangerous degree of site personnel is greatly increased. In addition, in the detection of wearing the safety helmet for personnel in a corridor, most of the detection is carried out by means of manual observation and monitoring by workers, so that the efficiency is low, and the detection accuracy of the workers who do not wear the safety helmet is low. Accordingly, an optimized in-aisle personnel monitoring system is desired that can accurately monitor whether in-aisle personnel wear safety helmets to ensure safe production of coal conveying systems.
At present, deep learning and neural networks have been widely used in the fields of computer vision, natural language processing, speech signal processing, and the like. In addition, deep learning and neural networks have also shown levels approaching and even exceeding humans in the fields of image classification, object detection, semantic segmentation, text translation, and the like.
In recent years, the development of deep learning and neural networks provides new solutions and schemes for monitoring the wearing of safety helmets by staff in galleries.
Accordingly, whether the personnel wear the safety helmet in the corridor can be considered to be identified and detected by means of the camera to acquire monitoring images of the personnel, however, when the personnel wear the safety helmet in the corridor to be monitored actually, the acquired images become fuzzy due to the fact that small air particles such as dust exist in the corridor, the safety helmet worn by the personnel is a small-size object, and the safety helmet is easy to be confused with other characteristic representations, so that the identification precision is not high. With the development of deep learning and neural networks, a special video image recognition system can be developed based on the deep learning and artificial intelligence algorithm in actual detection, so that whether personnel wear safety helmets in a corridor can be recognized.
Specifically, in the technical scheme of the application, an artificial intelligent monitoring technology based on machine vision is adopted, a monitored image of personnel in a gallery is used as input data to frame a target region of interest for personnel targets in the monitored image, and after image high-definition processing is carried out, significant target detection encoding is further carried out, so that small-size hidden characteristic information about whether personnel wear safety helmets or not in the monitored image based on the target frame and the high-definition processing is extracted, and classification judgment about whether the personnel wear the safety helmets or not is carried out by the personnel. Therefore, whether the worker wears the safety helmet in the corridor can be accurately monitored and judged, so that the safety production of the coal conveying system is ensured.
More specifically, in the technical scheme of the application, first, personnel monitoring images in a corridor are acquired through a camera. It should be understood that, considering that the hidden features about the staff in the monitoring image should be focused when detecting whether the staff wears the safety helmet in the corridor, if the rest useless interference feature information can be filtered out when the monitoring image of the staff in the corridor is subjected to feature mining, it is obvious that the accuracy of detecting whether the staff wears the safety helmet can be improved. Based on the above, in the technical solution of the present application, the personnel monitoring image in the corridor is further passed through a personnel target detection network to obtain a target region of interest, where the personnel target detection network is an anchor window-based target detection network. Specifically, the target anchoring layer of the personnel target detection network is used for sliding with an anchor frame B, and the personnel monitoring image in the corridor is processed, so that a personnel target region of interest in the monitoring image is framed, and the target region of interest is obtained. Accordingly, in one specific example of the present application, the anchor window based target detection network is Fast R-CNN, fast R-CNN or RetinaNet.
Then, considering that in the target region of interest in the personnel monitoring image in the corridor, the definition of the target region of interest is affected by a large amount of small environmental particles such as dust in the place of the coal conveying corridor, and the implicit characteristics of the target region of interest become fuzzy due to the interference of external environmental factors, so that the recognition accuracy of whether personnel wear safety helmets in the corridor is reduced. Therefore, in the technical solution of the present application, image sharpness enhancement is performed by a high-definition image generator based on an countermeasure generation network before feature extraction. Specifically, the target region of interest is input to the high-definition image generator based on the countermeasure generation network to generate the target region of interest by deconvolution encoding by a generator of the countermeasure generation network. In particular, here, the countermeasure generation network includes a discriminator for generating an image with enhanced image sharpness and a generator for calculating a difference between the image with enhanced data and a real image and updating network parameters of the generator by a direction propagation algorithm of gradient descent to obtain the generator with enhanced image sharpness.
Further, the generation target region of interest is encoded using a convolutional neural network having excellent performance in the field of image processing. The safety helmet representation worn by the staff is considered to belong to a small-size object and is easy to be confused with other feature representations, so that the identification precision is not high, and the resolution capability of the traditional convolutional neural network on the small-scale features in the image is not strong. Specifically, a salient target feature detection module is integrated in a network structure of the convolutional neural network, so that feature mining is conducted on the generation target region of interest by using a convolutional neural network model containing a salient target detector, and implicit feature distribution information about whether staff in a corridor wear safety helmet in the generation target region of interest is extracted, so that a classification feature map is obtained.
That is, since in the standard convolutional neural network, each layer of the convolutional neural network performs convolutional encoding once using a two-dimensional convolutional collation input data. Accordingly, in the technical scheme of the application, in order to improve the extraction capability of each layer of the convolutional neural network to the local salient features, each layer of the convolutional neural network is modified to perform secondary convolution, and the convolution kernel size of the first convolution is larger than that of the second convolution. It should be appreciated that convolution with a convolution kernel of larger size has a larger receptive field, but the extracted image feature pattern is coarse, and details with resolution in the region of interest of the generated target are easily ignored, and accordingly, secondary convolution with a convolution kernel of smaller size can better model local information. In a specific example of the present application, the first convolution kernel may be set to a 3×3 convolution kernel, and the second convolution kernel is a 1×1 convolution kernel.
And then, the corrected classification characteristic diagram passes through a classifier to obtain a classification result used for indicating whether the personnel wear the safety helmet. Therefore, whether the worker wears the safety helmet in the corridor can be monitored and judged, so that the safety production of the coal conveying system is ensured.
In particular, in the technical solution of the present application, when the target region of interest is obtained by using the high-definition image generator based on the countermeasure generation network, although the high-definition image generator based on the countermeasure generation network can simulate the image semantic distribution of the natural image as far as possible, the generated target region of interest still may have local image semantic distribution deviating from the natural image semantic distribution in some local areas, which makes that when the classification feature map is obtained by using the convolutional neural network model including the significant target detector, there may be a problem that the overall feature distribution monotonicity is poor due to the deviation of the local image semantic distribution in the feature matrix of the classification feature map, thereby affecting the overall distribution monotonicity of the classification feature map, resulting in poor convergence effect of classification of the classification feature map by the classifier, and affecting the training speed of the classifier and the accuracy of the classification result.
Thereby, each feature matrix of the classification feature map is subjected to a smooth maximum function approximation modulation, expressed as:
m i，j is the eigenvalue of each eigenvalue M of the classified eigenvector graph, wherein the eigenvalue is converted into a diagonal matrix by linear transformation, II 2 Is the two norms of the vector, and
representing multiplying each value of the matrix by a predetermined value.
Here, by approximately defining the symbolized distance function with a smooth maximum function along the row and column dimensions of the respective feature matrices, a relatively good union of convex optimizations of the high-dimensional manifold characterized by the respective feature matrices in the high-dimensional feature space can be achieved, and by modulating the structured feature distribution of the respective feature matrices with the relatively good union, a natural distribution transfer of the spatial feature variation from the internal structure of the feature distribution to the feature space can be obtained, and a convex monotonicity retention of the feature expression of the high-dimensional manifold of the respective feature matrices is enhanced, thereby enhancing the distribution monotonicity of the entire classification feature map, further improving the convergence effect of classification of the classification feature map by the classifier, and improving the training speed of the classifier and the accuracy of the classification result. Therefore, whether the worker wears the safety helmet in the corridor can be accurately monitored and judged, so that the safety production of the coal conveying system is ensured.
Fig. 1 is an application scenario diagram of a machine vision based personnel monitoring system in a corridor and a method thereof according to an embodiment of the present application. As shown in fig. 1, in this application scenario, a monitoring image of a person (e.g., P illustrated in fig. 1) in a corridor is acquired by a camera (e.g., C illustrated in fig. 1). The acquired in-aisle personnel monitoring images are then input to a server (e.g., S illustrated in fig. 1) deployed with an in-aisle personnel monitoring algorithm, where the server is capable of processing the in-aisle personnel monitoring images using the in-aisle personnel monitoring algorithm for use in representing classification results of whether personnel are wearing a helmet.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.
Exemplary System
FIG. 2 is a block diagram schematic of a machine vision based in-corridor personnel monitoring system, in accordance with an embodiment of the present application. As shown in fig. 2, the machine vision-based in-corridor personnel monitoring system 100, according to an embodiment of the present application, includes: the monitoring module 110 is used for acquiring personnel monitoring images in the corridor acquired by the camera; the target detection module 120 is configured to pass the personnel monitoring image in the corridor through a personnel target detection network to obtain a target region of interest; an image enhancement module 130 for passing the target region of interest through a high definition image generator based on an countermeasure generation network to obtain a generated target region of interest; an image feature extraction module 140, configured to obtain a classification feature map by using a convolutional neural network model including a salient object detector for the generated target region of interest; the feature distribution correction module 150 is configured to perform feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map; and a monitoring result generating module 160, configured to pass the corrected classification feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether the person wears the helmet.
In this embodiment of the present application, the monitoring module 110 is configured to obtain a monitored image of personnel in the corridor acquired by the camera. As described above, in the technical solution of the present application, it is considered that the identification and detection of whether the personnel wear the safety helmet in the corridor can be performed by means of the camera to collect the monitoring image of the personnel, however, when the personnel wear the safety helmet in practice, the collected image becomes blurred due to the existence of small air particles such as dust in the corridor, and the safety helmet worn by the personnel is represented as a small-sized object, and is easily confused with other feature representations, so that the identification accuracy is not high. With the development of deep learning and neural networks, a special video image recognition system can be developed based on the deep learning and artificial intelligence algorithm in actual detection, so that whether personnel wear safety helmets in a corridor can be recognized.
Specifically, in the technical scheme of the application, an artificial intelligent monitoring technology based on machine vision is adopted, a monitored image of personnel in a gallery is used as input data to frame a target region of interest for personnel targets in the monitored image, and after image high-definition processing is carried out, significant target detection encoding is further carried out, so that small-size hidden characteristic information about whether personnel wear safety helmets or not in the monitored image based on the target frame and the high-definition processing is extracted, and classification judgment about whether the personnel wear the safety helmets or not is carried out by the personnel. Therefore, whether the worker wears the safety helmet in the corridor can be accurately monitored and judged, so that the safety production of the coal conveying system is ensured.
In this embodiment of the present application, the target detection module 120 is configured to pass the personnel monitoring image in the corridor through a personnel target detection network to obtain a target region of interest. It will be appreciated that it is contemplated that only relevant parts of the monitoring image concerning the personnel need be focused, and other irrelevant parts need not be of interest, in the detection of whether the personnel are wearing a helmet in the corridor. Meanwhile, if feature mining is performed on other irrelevant parts, the calculation amount is increased, and the detection speed and the accuracy are reduced. Therefore, in the technical scheme of the application, the personnel monitoring image in the gallery further passes through the personnel target detection network to obtain the target region of interest, wherein the target region of interest only comprises hidden features about the personnel in the monitoring image, and other useless interference feature information is filtered out, so that the accuracy of whether the personnel wear safety helmet detection is improved.
In a specific embodiment of the present application, the personnel object detection network is an anchor window based object detection network. It should be understood that the deep learning-based target detection method classifies a network into two main categories, namely an Anchor-based (Anchor-based) and an Anchor-free (Anchor-free) based on whether or not an Anchor window is used in the network. The algorithm based on the anchor window is more mature and the detection precision of the universal target is higher, so in the embodiment, the target detection network based on the anchor window is adopted. Specifically, the target anchoring layer of the personnel target detection network is used for sliding by an anchor frame B, candidate areas with targets possibly existing are found from personnel monitoring images in the corridor, probability screening and correction are carried out on the candidate areas, and the staff target interested areas in the monitoring images are framed, so that the target interested areas are obtained.
In a specific embodiment of the present application, the anchor window-based target detection network is Fast R-CNN, fast R-CNN or RetinaNet.
In the embodiment of the present application, the image enhancement module 130 is configured to make the target region of interest pass through a high-definition image generator based on a countermeasure generation network to obtain a generated target region of interest. It should be understood that, considering that in the target region of interest in the personnel monitoring image in the corridor, since a large amount of small environmental particles such as dust exist in the place of the coal conveying corridor and affect the definition of the target region of interest, the implicit characteristics of the target region of interest become blurred due to the interference of external environmental factors, so that the accuracy of identifying whether personnel wear safety helmets in the corridor is reduced. Therefore, in the technical solution of the present application, image sharpness enhancement is performed by a high-definition image generator based on an countermeasure generation network before feature extraction.
In a specific embodiment of the present application, the countermeasure generation network includes a discriminator and a generator, wherein the image enhancement module is further configured to input the target region of interest into the high definition image generator based on the countermeasure generation network to generate the target region of interest by deconvolution encoding by the generator of the countermeasure generation network. It will be appreciated that the generator is for generating an image with enhanced image sharpness, and the discriminator is for calculating the difference between the data-enhanced image and the real image and updating the network parameters of the generator by a gradient descent direction propagation algorithm to obtain a generator with enhanced image sharpness.
In the embodiment of the present application, the image feature extraction module 140 is configured to obtain the classification feature map by using a convolutional neural network model including a significant object detector for the generated target region of interest. It should be appreciated that the generation target region of interest is encoded using a convolutional neural network having excellent performance in the field of image processing. However, the safety helmet representation worn by the staff is considered to belong to a small-size object and is easy to be confused with other feature representations, so that the identification precision is not high, and the resolution capability of the traditional convolutional neural network on the small-scale features in the image is not strong. Specifically, a salient target feature detection module is integrated in a network structure of the convolutional neural network, so that feature mining is conducted on the generation target region of interest by using a convolutional neural network model containing a salient target detector, and implicit feature distribution information about whether staff in a corridor wear safety helmet in the generation target region of interest is extracted, so that a classification feature map is obtained.
In a specific embodiment of the present application, the image feature extraction module is further configured to: each layer of the convolutional neural network model containing the salient object detector is used for respectively carrying out input data in forward transfer of the layer: performing convolution processing on the input data by using a first convolution kernel to obtain a first convolution feature map; performing convolution processing on the first convolution feature map by using a second convolution kernel to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than that of the second convolution kernel; pooling the second convolution feature map to obtain a pooled feature map; activating the pooled feature map to obtain an activated feature map; the output of the last layer of the first convolutional neural network model is the classification characteristic diagram, and the input of the first layer of the first convolutional neural network model is the generation target region of interest.
That is, since in the standard convolutional neural network, each layer of the convolutional neural network performs convolutional encoding once using a two-dimensional convolutional collation input data. Accordingly, in the technical scheme of the application, in order to improve the extraction capability of each layer of the convolutional neural network to the local salient features, each layer of the convolutional neural network is modified to perform secondary convolution, and the convolution kernel size of the first convolution is larger than that of the second convolution. It should be appreciated that convolution with a convolution kernel of larger size has a larger receptive field, but the extracted image feature pattern is coarse, and details with resolution in the region of interest of the generated target are easily ignored, and accordingly, secondary convolution with a convolution kernel of smaller size can better model local information. In a specific example of the present application, the first convolution kernel may be set to a 3×3 convolution kernel, and the second convolution kernel is a 1×1 convolution kernel.
In this embodiment of the present application, the feature distribution correction module 150 is configured to perform feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map. In particular, in the technical solution of the present application, when the target region of interest is obtained by using the high-definition image generator based on the countermeasure generation network, although the high-definition image generator based on the countermeasure generation network can simulate the image semantic distribution of the natural image as far as possible, the generated target region of interest still may have local image semantic distribution deviating from the natural image semantic distribution in some local areas, which makes that when the classification feature map is obtained by using the convolutional neural network model including the significant target detector, there may be a problem that the overall feature distribution monotonicity is poor due to the deviation of the local image semantic distribution in the feature matrix of the classification feature map, thereby affecting the overall distribution monotonicity of the classification feature map, resulting in poor convergence effect of classification of the classification feature map by the classifier, and affecting the training speed of the classifier and the accuracy of the classification result. Thereby, each feature matrix of the classification feature map is subjected to a smooth maximum function approximation modulation.
In a specific embodiment of the present application, the feature distribution correction module is further configured to: carrying out feature distribution correction on each feature matrix of the classification feature map along the channel dimension by using the following formula to obtain a corrected classification feature map;
wherein, the formula is:
wherein M represents each feature matrix of the classification feature map along the channel dimension, M i，j Is the eigenvalue of each individual position of each eigenvector matrix along the channel dimension of the classification eigenvector graph, II 2 Is the two norms of the vector, and
representing multiplication of each value of the matrix by a predetermined value,/->
Representing addition by position, M' represents each feature matrix along the channel dimension of the corrected classification feature map.
Here, by approximately defining the symbolized distance function with a smooth maximum function along the row and column dimensions of the respective feature matrices, a relatively good union of convex optimizations of the high-dimensional manifold characterized by the respective feature matrices in the high-dimensional feature space can be achieved, and by modulating the structured feature distribution of the respective feature matrices with the relatively good union, a natural distribution transfer of the spatial feature variation from the internal structure of the feature distribution to the feature space can be obtained, and a convex monotonicity retention of the feature expression of the high-dimensional manifold of the respective feature matrices is enhanced, thereby enhancing the distribution monotonicity of the entire classification feature map, further improving the convergence effect of classification of the classification feature map by the classifier, and improving the training speed of the classifier and the accuracy of the classification result. Therefore, whether the worker wears the safety helmet in the corridor can be accurately monitored and judged, so that the safety production of the coal conveying system is ensured.
In this embodiment of the present application, the monitoring result generating module 160 is configured to pass the corrected classification feature map through a classifier to obtain a classification result, where the classification result is used to indicate whether the person wears a helmet. That is, hidden features between the worker and the helmet are mined, so that classification processing is performed in the classifier to obtain a classification result for indicating whether the worker wears the helmet. Therefore, whether the worker wears the safety helmet in the corridor can be monitored and judged, so that the safety production of the coal conveying system is ensured.
Fig. 3 is a block diagram of a monitoring result generation module in a machine vision based in-corridor personnel monitoring system according to an embodiment of the present application. As shown in fig. 3, in a specific embodiment of the present application, the monitoring result generating module 160 includes: a full-connection encoding unit 161, configured to perform full-connection encoding on the corrected classification feature map by using a full-connection layer of the classifier to obtain a classification feature vector; a soft maximum value calculation unit 162, configured to input the classification feature vector into a Softmax classification function of the classifier to obtain probability values of the classification feature vector belonging to each classification label, where the classification labels include that the person wears a helmet and that the person does not wear a helmet; and a result determining unit 163 for determining a classification label corresponding to the largest one of the probability values as the classification result.
That is, the full-connection layer of the classifier is used for carrying out full-connection coding on the corrected classification characteristic diagram so as to fully utilize the information of each position in the corrected classification characteristic diagram to obtain classification characteristic vectors; then, a Softmax function value of the classification feature vector, i.e. a probability value that the classification feature vector belongs to each classification label, which in the embodiment of the application comprises that the person wears a helmet (first label) and that the person does not wear a helmet (second label), is calculated. And finally, determining the classification label corresponding to the largest probability value as the classification result, namely, if the first probability of the first label is larger than the second probability of the second label, the classification result is that the personnel wear the safety helmet, and otherwise, the classification result is opposite.
Still further, can also be provided with the alarm, when the classification result is that the personnel does not wear the safety helmet, can report to the police to remind the staff to wear the safety helmet, thereby guarantee the safety in production of coal conveying system.
In summary, according to the machine vision-based intra-corridor personnel monitoring system provided by the embodiment of the application, an artificial intelligent monitoring technology based on machine vision is adopted, a monitored image of personnel in a corridor is used as input data to frame a target region of interest for a personnel target in the monitored image, and after image high definition processing, significant target detection encoding is further carried out, so that small-size implicit characteristic information about whether personnel wear a safety helmet or not in the monitored image based on target frame and high definition processing is extracted, and classification judgment about whether the personnel wear the safety helmet or not is carried out by the personnel, and in such a way, whether the personnel wear the safety helmet or not in the corridor can be accurately monitored and judged, so that safety production of a coal conveying system is ensured.
As described above, the in-corridor personnel monitoring system 100 based on machine vision according to the embodiment of the present application may be implemented in various terminal devices, for example, a server or the like deployed with an in-corridor personnel monitoring algorithm based on machine vision. In one example, the personnel monitoring system 100 may be integrated into the terminal device as a software module and/or hardware module according to the machine vision based in-aisle. For example, the machine vision based in-corridor personnel monitoring system 100 may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the machine vision based in-aisle personnel monitoring system 100 can also be one of the plurality of hardware modules of the terminal equipment.
Alternatively, in another example, the machine vision based in-corridor personnel monitoring system 100 and the terminal device may also be separate devices, and the machine vision based in-corridor personnel monitoring system 100 may connect to the terminal device through a wired and/or wireless network and transmit the interactive information in a agreed data format.
Exemplary method
Fig. 4 is a flow chart of a method of machine vision based in-corridor personnel monitoring, in accordance with an embodiment of the present application. As shown in fig. 4, the method for monitoring personnel in a gallery based on machine vision according to an embodiment of the application includes: s110, acquiring personnel monitoring images in a corridor acquired by a camera; s120, passing the personnel monitoring image in the corridor through a personnel target detection network to obtain a target region of interest; s130, enabling the target region of interest to pass through a high-definition image generator based on an countermeasure generation network to obtain a generated target region of interest; s140, the generated target region of interest is subjected to convolutional neural network model containing a significant target detector to obtain a classification characteristic diagram; s150, carrying out feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map; and S160, passing the corrected classification characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether a person wears the safety helmet.
Fig. 5 is a schematic diagram of a system architecture of a machine vision based in-corridor personnel monitoring method, according to an embodiment of the present application. As shown in fig. 5, in the system architecture of the in-corridor personnel monitoring method based on machine vision according to the embodiment of the present application, first, an in-corridor personnel monitoring image acquired by a camera is acquired. And then, the personnel monitoring image in the corridor passes through a personnel target detection network to obtain a target region of interest, and the target region of interest passes through a high-definition image generator based on a countermeasure generation network to obtain a generated target region of interest. And then, the generated target region of interest is subjected to characteristic distribution correction on each characteristic matrix of the classified characteristic map along the channel dimension by using a convolutional neural network model containing a remarkable target detector to obtain the classified characteristic map. And finally, the corrected classification characteristic diagram is passed through a classifier to obtain a classification result, wherein the classification result is used for indicating whether a person wears the safety helmet.
In a specific embodiment of the present application, the personnel object detection network is an anchor window based object detection network.
In a specific embodiment of the present application, the anchor window-based target detection network is Fast R-CNN, fast R-CNN or Ret i naNet.
In a specific embodiment of the present application, the countermeasure generation network includes a discriminator and a generator, wherein the generating the target region of interest by the high-definition image generator based on the countermeasure generation network includes: the target region of interest is input to the countermeasure generation network based high definition image generator to generate the target region of interest by deconvolution encoding by a generator of the countermeasure generation network.
In a specific embodiment of the present application, the generating the target region of interest by using a convolutional neural network model including a salient target detector to obtain a classification feature map includes: each layer of the convolutional neural network model containing the salient object detector is used for respectively carrying out input data in forward transfer of the layer: performing convolution processing on the input data by using a first convolution kernel to obtain a first convolution feature map; performing convolution processing on the first convolution feature map by using a second convolution kernel to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than that of the second convolution kernel; pooling the second convolution feature map to obtain a pooled feature map; performing activation processing on the pooled feature map to obtain an activated feature map; the output of the last layer of the first convolutional neural network model is the classification characteristic diagram, and the input of the first layer of the first convolutional neural network model is the generation target region of interest.
In a specific embodiment of the present application, the performing feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map includes: carrying out feature distribution correction on each feature matrix of the classification feature map along the channel dimension by using the following formula to obtain a corrected classification feature map;
wherein, the formula is:
wherein M represents each feature matrix of the classification feature graph along the channel dimension, M i，j Is the eigenvalue of each individual position of each eigenvector matrix along the channel dimension of the classification eigenvector graph, II 2 Is the two norms of the vector, and
representing multiplication of each value of the matrix by a predetermined value,/->
Representing addition by position, M' represents each feature matrix along the channel dimension of the corrected classification feature map.
Fig. 6 is a flowchart of a method for monitoring personnel in a gallery based on machine vision according to an embodiment of the application, wherein the corrected classification feature map is passed through a classifier to obtain a classification result. As shown in fig. 6, in a specific embodiment of the present application, the step of passing the corrected classification feature map through a classifier to obtain a classification result includes: s210, performing full-connection coding on the corrected classification characteristic map by using a full-connection layer of the classifier to obtain a classification characteristic vector; s220, inputting the classification feature vector into a Softmax classification function of the classifier to obtain a probability value of the classification feature vector belonging to each classification label, wherein the classification labels comprise that the personnel wear safety helmets and that the personnel do not wear safety helmets; and S230, determining the classification label corresponding to the maximum probability value as the classification result.
Here, it will be understood by those skilled in the art that the specific operations of the respective steps in the above-described machine vision-based in-corridor personnel monitoring method have been described in detail in the above description of the machine vision-based in-corridor personnel monitoring system with reference to fig. 1 to 3, and thus, repetitive descriptions thereof will be omitted.
Exemplary electronic device
Next, an electronic device according to an embodiment of the present application is described with reference to fig. 7.
Fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.
As shown in fig. 7, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).
The input means 13 may comprise, for example, a keyboard, a mouse, etc.
The output device 14 may output various information including the classification result and the like to the outside. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.
Exemplary computer program product and computer readable storage Medium
In addition to the methods and apparatus described above, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the steps of the machine vision based intra-corridor personnel monitoring method described in the above "exemplary methods" section of the present specification, according to various embodiments of the present application.
The computer program product may write program code for performing the operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions, which when executed by a processor, cause the processor to perform the steps of the machine vision based intra-corridor personnel monitoring method according to the various embodiments of the present application described in the above "exemplary method" section of the present specification.
The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not limiting, and these advantages, benefits, effects, etc. are not to be considered as necessarily possessed by the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not intended to be limited to the details disclosed herein as such.
The block diagrams of the devices, apparatuses, devices, systems referred to in this application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.
It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent to the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.
Claims (10)
1. An in-corridor personnel monitoring system based on machine vision, comprising:
the monitoring module is used for acquiring personnel monitoring images in the corridor acquired by the camera;
the target detection module is used for passing the personnel monitoring image in the corridor through a personnel target detection network to obtain a target region of interest;
the image enhancement module is used for enabling the target region of interest to pass through a high-definition image generator based on an countermeasure generation network so as to obtain a generated target region of interest;
the image feature extraction module is used for obtaining a classification feature map by using a convolutional neural network model containing a significant target detector for the generated target region of interest;
the characteristic distribution correction module is used for carrying out characteristic distribution correction on each characteristic matrix of the classification characteristic map along the channel dimension so as to obtain a corrected classification characteristic map; and
And the monitoring result generation module is used for enabling the corrected classification characteristic diagram to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether personnel wear safety helmets.
2. The machine vision-based in-corridor personnel monitoring system of claim 1, wherein the personnel objective detection network is an anchor window-based objective detection network.
3. The machine vision based in-corridor personnel monitoring system of claim 2, wherein the anchor window based target detection network is Fast R-CNN, or RetinaNet.
4. The machine vision based in-corridor personnel monitoring system of claim 3, wherein the countermeasure generation network includes a discriminator and a generator, wherein the image enhancement module is further configured to input the target region of interest into the countermeasure generation network based high definition image generator to generate the target region of interest by deconvolution encoding by the countermeasure generation network generator.
5. The machine vision based in-corridor personnel monitoring system of claim 4, wherein the image feature extraction module is further configured to: each layer of the convolutional neural network model containing the salient object detector is used for respectively carrying out input data in forward transfer of the layer:
Performing convolution processing on the input data by using a first convolution kernel to obtain a first convolution feature map;
performing convolution processing on the first convolution feature map by using a second convolution kernel to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than that of the second convolution kernel;
pooling the second convolution feature map to obtain a pooled feature map; and
performing activation processing on the pooled feature map to obtain an activated feature map;
the output of the last layer of the first convolutional neural network model is the classification characteristic diagram, and the input of the first layer of the first convolutional neural network model is the generation target region of interest.
6. The machine vision based in-corridor personnel monitoring system of claim 5, wherein the feature distribution correction module is further configured to: carrying out feature distribution correction on each feature matrix of the classification feature map along the channel dimension by using the following formula to obtain a corrected classification feature map;
wherein, the formula is:
wherein M represents each feature matrix of the classification feature map along the channel dimension, M i，j Is the eigenvalue of each individual position of each eigenvector matrix along the channel dimension of the classification eigenvector graph, II 2 Is the two norms of the vector, and
representing multiplication of each value of the matrix by a predetermined value,/->
Representing addition by position, M' represents each feature matrix along the channel dimension of the corrected classification feature map.
7. The machine vision based in-corridor personnel monitoring system of claim 6, wherein the monitoring result generation module comprises:
the full-connection coding unit is used for carrying out full-connection coding on the corrected classification characteristic map by using a full-connection layer of the classifier so as to obtain a classification characteristic vector;
the soft maximum value calculation unit is used for inputting the classification feature vector into a Softmax classification function of the classifier to obtain a probability value of the classification feature vector belonging to each classification label, wherein the classification labels comprise that the personnel wear safety helmets and that the personnel do not wear safety helmets; and
and the result determining unit is used for determining the classification label corresponding to the maximum probability value as the classification result.
8. The method for monitoring personnel in the corridor based on machine vision is characterized by comprising the following steps:
acquiring personnel monitoring images in a corridor acquired by a camera;
the personnel monitoring images in the corridor pass through a personnel target detection network to obtain a target region of interest;
The target region of interest passes through a high-definition image generator based on an countermeasure generation network to obtain a generated target region of interest;
the generated target region of interest is subjected to convolutional neural network model containing a remarkable target detector to obtain a classification characteristic diagram;
performing feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map; and
and passing the corrected classification characteristic diagram through a classifier to obtain a classification result, wherein the classification result is used for indicating whether a person wears the safety helmet.
9. The machine vision based intra-corridor personnel monitoring method of claim 8, wherein said generating the target area of interest by using a convolutional neural network model including a salient target detector to obtain a classification feature map comprises: each layer of the convolutional neural network model containing the salient object detector is used for respectively carrying out input data in forward transfer of the layer:
performing convolution processing on the input data by using a first convolution kernel to obtain a first convolution feature map;
performing convolution processing on the first convolution feature map by using a second convolution kernel to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than that of the second convolution kernel; and
Pooling the second convolution feature map to obtain a pooled feature map; and
performing activation processing on the pooled feature map to obtain an activated feature map;
the output of the last layer of the first convolutional neural network model is the classification characteristic diagram, and the input of the first layer of the first convolutional neural network model is the generation target region of interest.
10. The machine vision based intra-corridor personnel monitoring method of claim 9, wherein the performing feature distribution correction on each feature matrix of the classification feature map along the channel dimension to obtain a corrected classification feature map comprises: carrying out feature distribution correction on each feature matrix of the classification feature map along the channel dimension by using the following formula to obtain a corrected classification feature map;
wherein, the formula is:
wherein M represents each feature matrix of the classification feature map along the channel dimension, M i，j Is the eigenvalue of each individual position of each eigenvector matrix along the channel dimension of the classification eigenvector graph, II 2 Is the two norms of the vector, and
representing multiplication of each value of the matrix by a predetermined value,/->
Representing addition by position, M' representing each of the corrected classification feature maps along the channel dimension And (3) feature matrices.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2023101845080 | 2023-02-28 | ||
| CN202310184508 | 2023-02-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116385962A true CN116385962A (en) | 2023-07-04 |
Family
ID=86968639
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310292036.0A Pending CN116385962A (en) | 2023-02-28 | 2023-03-22 | Personnel monitoring system in corridor based on machine vision and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116385962A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116824517A (en) * | 2023-08-31 | 2023-09-29 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
-
2023
- 2023-03-22 CN CN202310292036.0A patent/CN116385962A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116824517A (en) * | 2023-08-31 | 2023-09-29 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
| CN116824517B (en) * | 2023-08-31 | 2023-11-17 | 安徽博诺思信息科技有限公司 | Substation operation and maintenance safety control system based on visualization |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Huang et al. | Detection algorithm of safety helmet wearing based on deep learning | |
| CN116385962A (en) | Personnel monitoring system in corridor based on machine vision and method thereof | |
| CN116311005A (en) | Apparatus, method and storage medium for moving image processing | |
| CN115294528A (en) | Pedestrian safety monitoring method and device | |
| CN112489087A (en) | Method for detecting shaking of suspension type operation platform for high-rise building construction | |
| CN116152863A (en) | Personnel information identification method and device, electronic equipment and storage medium | |
| CN113988180A (en) | Model fingerprint-based generated image tracing method | |
| CN111931573A (en) | Helmet detection and early warning method based on YOLO evolution deep learning model | |
| CN117115743A (en) | Mining safety production monitoring system and method thereof | |
| CN115620178A (en) | Real-time detection method for abnormal and dangerous behaviors of power grid of unmanned aerial vehicle | |
| Yang et al. | Safety helmet wearing detection based on an improved YOLOv3 scheme | |
| CN114529821A (en) | Offshore wind power safety monitoring and early warning method based on machine vision | |
| CN116310922A (en) | Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium | |
| CN117423157A (en) | Mine abnormal video action understanding method combining migration learning and regional invasion | |
| CN112183532A (en) | Safety helmet identification method based on weak supervision collaborative learning algorithm and storage medium | |
| CN117351409A (en) | Intelligent concrete dam face operation risk identification method | |
| CN116994049A (en) | Full-automatic flat knitting machine and method thereof | |
| CN116216234A (en) | Intelligent monitoring system and method for coal conveying system based on video | |
| Peng et al. | [Retracted] Helmet Wearing Recognition of Construction Workers Using Convolutional Neural Network | |
| CN111414825B (en) | Method for detecting wearing of safety helmet | |
| CN114663972A (en) | Target marking method and device based on motion difference | |
| CN113887634A (en) | Improved two-step detection-based electric safety belt detection and early warning method | |
| CN112668535A (en) | YOLOv3 model-based coal mine monkey vehicle violation detection and early warning method | |
| Wu et al. | Design and implementation of safety helmet detection system based on computer vision | |
| Zhang et al. | Single-Stage Related Object Detection for Intelligent Industrial Surveillance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |