CN115240277A - Security check behavior monitoring method and device, electronic equipment and storage medium - Google Patents

Security check behavior monitoring method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN115240277A
CN115240277A CN202210956039.5A CN202210956039A CN115240277A CN 115240277 A CN115240277 A CN 115240277A CN 202210956039 A CN202210956039 A CN 202210956039A CN 115240277 A CN115240277 A CN 115240277A
Authority
CN
China
Prior art keywords
label
security
classification model
security inspection
security check
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210956039.5A
Other languages
Chinese (zh)
Inventor
陈运华
余祖杰
李鹏飞
曾平平
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lansi System Integration Co ltd
Original Assignee
Lansi System Integration Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lansi System Integration Co ltd filed Critical Lansi System Integration Co ltd
Priority to CN202210956039.5A priority Critical patent/CN115240277A/en
Publication of CN115240277A publication Critical patent/CN115240277A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06398—Performance of employee with respect to a job function
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

The invention discloses a method and a device for monitoring security check behaviors, electronic equipment and a storage medium, and belongs to the technical field of security and protection. Wherein, the method comprises the following steps: acquiring security inspection images at a plurality of visual angles; inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; and analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement. The invention solves the technical problem that manual supervision of the security inspection behaviors is difficult to realize constant supervision in the related technology, and improves the monitoring efficiency of the security inspection behaviors.

Description

Security check behavior monitoring method and device, electronic equipment and storage medium
Technical Field
The invention relates to the technical field of security and protection, in particular to a method and a device for monitoring security check behaviors, electronic equipment and a storage medium.
Background
For high security workplaces, detection of contraband is required, and for high security products, monitoring of the outflow of forbidden products or designs is also required. With the development of security inspection products, most of prohibited articles can be solved by security inspection instruments and devices, but none or a few security inspection devices can cover the detection of all materials and articles, so that the hand touch type security inspection of security inspectors still becomes necessary and accounts for a certain proportion. The security personnel of security installations touches the key inspection position of the person to be inspected through the hand clapping, and the conditions of product leakage and carrying of forbidden articles are avoided. The operation of the security inspector has the standard, and how to judge whether the actions of the security inspector are complete is an important link for monitoring the security inspection work. The quality control of the work of the security inspector has great risk, under the scene, the work quality supervision of the security inspector in the related technology is only an online and offline inspection spot check mode, the security inspector does not fall to the ground through an intelligent analysis method, a large amount of manpower is required to be invested for a park and a platform with more security inspectors to ensure the monitoring quality, and the supervision at any time cannot be achieved.
In view of the above problems in the related art, no effective solution has been found at present.
Disclosure of Invention
The invention provides a method and a device for monitoring a security check behavior, electronic equipment and a storage medium, and aims to solve the technical problems that the manual monitoring of the security check behavior in the related art is difficult to realize the monitoring all the time and the monitoring efficiency is low.
According to an aspect of the embodiments of the present application, there is provided a method for identifying a security check behavior, including: acquiring security inspection images at a plurality of visual angles; inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; and analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement.
Further, analyzing the current security check behavior according to the target personnel tag information, and monitoring whether the current security check behavior meets a preset security check requirement comprises: outputting a security inspection part label matched with the label information of the target personnel by adopting the multi-label classification model; acquiring security inspection action images at a plurality of visual angles; inputting the security inspection action images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring action labels and corresponding confidence degrees of the security inspection part labels output by the multi-label classification model, wherein the confidence degrees corresponding to the action labels are used for representing the probability of security inspection of a security inspector on an inspected part of an inspected person according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part labels; and if the confidence corresponding to the action tag is greater than a preset threshold, determining that the security check behavior meets the preset security check requirement.
Further, inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model, and obtaining the label information of the target person output by the multi-label classification model includes: inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring a first confidence coefficient of a person appearance label and a second confidence coefficient corresponding to a person identity attribute label output by the multi-label classification model; respectively determining the appearance attribute and the identity attribute of the person to be detected according to the first confidence coefficient of the person appearance tag and the second confidence coefficient corresponding to the person identity attribute tag; and adding the target personnel label information on the security inspection image according to the identity attribute and the appearance attribute.
Further, the multi-label classification model comprises a feature extraction module, a feature fusion module and a multi-label prediction module, the security inspection images at the multiple viewing angles are input into the pre-trained multi-label classification model, and the target personnel label information output by the multi-label classification model comprises: inputting the security inspection images at the multiple visual angles into the feature extraction module, and extracting feature information corresponding to the personnel label information at each visual angle; inputting the feature information into the feature fusion module, and fusing the corresponding feature information under a plurality of visual angles; and inputting the fused characteristic information into the multi-label prediction module to obtain the label information of the target personnel.
Further, before the security inspection images at the multiple viewing angles are input into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model, the method further comprises: obtaining a security inspection image sample; setting sample labels of the security inspection image samples, wherein the sample labels comprise positive sample labels, negative sample labels and ignore labels, the positive sample labels are used for identifying that the corresponding image samples comprise corresponding personnel attributes or security inspection actions, the negative sample labels are used for identifying that the corresponding image samples do not comprise the corresponding personnel attributes or security inspection actions, and the ignore labels are used for indicating that the security inspection image samples are not used when the designated personnel attributes or security inspection actions are trained; and training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model.
Further, training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model comprises: training an initial model by using the security inspection image sample and the following Loss function Loss to obtain the multi-label classification model:
Figure BDA0003791401290000031
wherein, y i The value of the label value corresponding to the ith label is 0 or 1, P is the probability value predicted by the multi-label classification model aiming at each label, and the value range of the probability value is [0,1]]，
Figure BDA0003791401290000032
Is the sum of the cross entropies corresponding to all tags from i =1 to i = n, 1[ y ], [ i ≠-1]Is shown when y i When not equal to-1, the output is 1 or the output is 0, m is the number of tags having a tag value of 0 or 1, and n is the number of all tags.
Further, before the security inspection images at the multiple viewing angles are input into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model, the method further comprises the following steps: acquiring an original video frame sequence under each visual angle; respectively acquiring current video frames in an original video frame sequence under each visual angle at the same moment; and cutting the current video frame according to a preset security check demarcation area to obtain a corresponding security check area map under each visual angle, and taking the security check area map as the security check image.
According to another aspect of the embodiments of the present application, there is also provided a security check behavior monitoring apparatus, including: the acquisition module is used for acquiring security inspection images at a plurality of visual angles; the label determination module is used for inputting the security inspection images at the multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; and the analysis module is used for analyzing the current security check behavior according to the target personnel label information and monitoring whether the current security check behavior meets the preset security check requirement.
Further, the analysis module comprises a first analysis module, and the first analysis module is used for outputting a security inspection part tag matched with the tag information of the target personnel by adopting the multi-tag classification model; acquiring security inspection action images at a plurality of visual angles; inputting the security inspection action images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring action labels and corresponding confidence degrees of the security inspection part labels output by the multi-label classification model, wherein the confidence degrees corresponding to the action labels are used for representing the probability of security inspection of a security inspector on an inspected part of an inspected person according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part labels; and if the confidence corresponding to the action tag is greater than a preset threshold, determining that the security check behavior meets a preset security check requirement.
Further, the label determination module comprises a first determination module, and the first determination module is configured to input the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model, and obtain a first confidence level of the appearance labels of the people and a second confidence level corresponding to the identity attribute labels of the people, which are output by the multi-label classification model; respectively determining the appearance attribute and the identity attribute of the person to be detected according to the first confidence coefficient of the person appearance tag and the second confidence coefficient corresponding to the person identity attribute tag; and adding the target personnel label information on the security inspection image according to the identity attribute and the appearance attribute.
Further, the tag determination module comprises a second determination module, and the second determination module is configured to input the security inspection images at the multiple viewing angles into the feature extraction module, and extract feature information corresponding to the personnel tag information at each viewing angle; inputting the feature information into the feature fusion module, and fusing the corresponding feature information under a plurality of visual angles; and inputting the fused characteristic information into the multi-label prediction module to obtain the label information of the target personnel.
Further, the monitoring device for the security check behavior comprises a model training module, wherein the model training module is used for acquiring a security check image sample; setting sample labels of the security check image samples, wherein the sample labels comprise positive sample labels, negative sample labels and ignore labels, the positive sample labels are used for identifying that the corresponding image samples comprise corresponding personnel attributes or security check actions, the negative sample labels are used for identifying that the corresponding image samples do not comprise corresponding personnel attributes or security check actions, and the ignore labels are used for indicating that the security check image samples are not used when the designated personnel attributes or security check actions are trained; and training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model.
Further, the model training module is further configured to train an initial model by using the security inspection image sample and the following Loss function Loss to obtain the multi-label classification model:
Figure BDA0003791401290000041
wherein, y i The value of the label value corresponding to the ith label is 0 or 1, P is the probability value predicted by the multi-label classification model aiming at each label, and the value range of the probability value is [0,1]]，
Figure BDA0003791401290000042
1[ y ] is the sum of the corresponding cross entropies of all labels from i =1 to i = n i ≠-1]Is shown when y i When not equal to-1, the output is 1 or the output is 0, m is the number of tags having a tag value of 0 or 1, and n is the number of all tags.
Further, the monitoring device for the security check behavior comprises a data preprocessing module, wherein the data preprocessing module is used for acquiring an original video frame sequence under each visual angle; respectively acquiring current video frames in an original video frame sequence under each visual angle at the same moment; and cutting the current video frame according to a preset security check demarcation area to obtain a corresponding security check area map under each visual angle, and taking the security check area map as the security check image.
According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program that executes the above steps when the program is executed.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; wherein: a memory for storing a computer program; a processor for executing the steps of the method by running the program stored in the memory.
Embodiments of the present application also provide a computer program product containing instructions, which when run on a computer, cause the computer to perform the steps of the above method.
According to the invention, security inspection images at multiple visual angles are acquired; inputting security inspection images at multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; the method comprises the steps of analyzing the current security check behavior according to the label information of a target person, monitoring whether the current security check behavior meets the preset security check requirement, analyzing the multi-view security check image through a multi-label classification model to obtain the person label information of a person to be checked, and then automatically monitoring whether the current security check behavior of the security checker meets the security check standard according to the security check requirement corresponding to the person label information, so that the technical problem that the manual supervision security check behavior of the related technology is difficult to supervise constantly is solved, and the monitoring efficiency of the security check behavior is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a block diagram of a hardware configuration of a computer according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of monitoring security activities according to an embodiment of the invention;
FIG. 3 is a schematic diagram of a security check behavior monitoring system according to an embodiment of the present invention;
FIG. 4 is a schematic field layout of an embodiment of the present invention;
FIG. 5 is a flow diagram of a multi-label classification model according to an embodiment of the invention;
FIG. 6 is a schematic diagram of a feature extraction module based on a Residual Block structure according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a feature fusion module based on the Attention Block structure according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a prediction module based on a Residual Block structure according to an embodiment of the present invention;
fig. 9 is a block diagram of a security check behavior monitoring apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application. It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the accompanying drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
The method provided in the first embodiment of the present application may be executed in a mobile phone, a computer, a tablet, or a similar computing device. Taking an example of the present invention running on a computer, fig. 1 is a block diagram of a hardware structure of a computer according to an embodiment of the present invention. As shown in fig. 1, the computer may include one or more (only one shown in fig. 1) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally, a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those of ordinary skill in the art that the configuration shown in FIG. 1 is illustrative only and is not intended to limit the configuration of the computer described above. For example, a computer may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a video motion and static rate identification method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the above method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In this embodiment, a method for monitoring a security inspection behavior is provided, and fig. 2 is a flowchart of the method for monitoring a security inspection behavior according to the embodiment of the present invention, as shown in fig. 2, the flowchart includes the following steps:
s10, acquiring security inspection images at multiple visual angles;
the security check gate sentry area usually has more personnel to get in and out, the security check action of the security check personnel is fast, and the number of check parts is large, so that the real-time requirement for supervising the security check action is high. Simultaneously, the distance between security inspector and the person of examining is nearer when carrying out the security check action, takes place local sheltering from easily, and the problem that the vision sheltered from is hardly adjusted suitable angle and position through single camera and is solved, sets up a plurality of cameras in this embodiment, acquires the security check image of security inspector's security check action under a plurality of visual angles simultaneously through a plurality of cameras. Referring to fig. 4, taking a single gate sentry as an example, cameras are installed on two sides of the single gate sentry, and a left-side camera and a right-side camera are used to monitor a security inspection area, so as to make the shooting clear, the following hardware installation parameters and camera system parameters may be referred to: the camera head is set to have a height H in the range of 3.5-4.5 meters, an angle theta formed by the camera head and the security check pad is about 45 degrees, a reference vertical distance d2 from the security check pad is 1.4 meters, a reference parallel distance d1 from the security check pad is 0.9 meter, wherein the diameter of the security check pad is 0.5 meter, the security check pad can be set to be blue or other colors convenient for identification, and the height of the security check pad is 10-15 cm; the resolution of the camera is greater than 1280x720, the frame rate is greater than 20 frames/second, the camera is a network camera, and other parameters may be set, which is not limited herein.
Step S20, inputting the security inspection images at the multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection;
inputting security inspection images at multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; the personnel label information can be information predetermined according to specific security check requirements, specifically, the personnel label information comprises a personnel appearance label and a personnel identity attribute label, wherein the personnel appearance label comprises a direct appearance label which can be distinguished by characteristics such as personnel orientation, dressing and the like, such as 'presence/absence', 'long sleeve/short sleeve', 'long hair/short hair/hat wearing', 'front/back' and the like; the personnel identity attribute labels comprise security inspectors, suppliers, clients, police assistants, employees and the like, can be obtained by distinguishing dressing colors and/or styles of inspectors, and can be conveniently processed and recorded by special security inspection specifications aiming at personnel with different appearance attributes and identity attributes in a follow-up procedure. Each person tag information corresponds to a group of security check behaviors required by security check, for example, if a multi-tag classification model identifies that a checked person wears a short sleeve, a security check wrist action is not required, if the checked person is short-hair, security check hair is not required in the security check action, and if the checked person is a special group supplier, a security check instrument detection mode is required. Judge personnel business turn over based on personnel label information to personnel attribute to the person examined carries out the analysis, wherein, judges personnel business turn over and includes: when the confidence level of the "someone" tag output by the multi-tag classification model is greater than a set threshold (for example, 60% of the set threshold, or other thresholds may be set according to actual standard requirements), it is determined that the frame of security inspection image indicates that someone has entered the security inspection area, and meanwhile, if the "front" tag output by the multi-tag classification model is greater than the set threshold, it is determined that the front of the person to be inspected has entered, otherwise, it is determined that the back of the person has entered. Similarly, when the value of the 'person' label given to the security inspection image by the algorithm of the multi-label classification model is smaller than the set threshold value, the person is considered to leave the security inspection area. It should be noted that, the confidence levels of different tags may be set with different thresholds, that is, the confidence level setting thresholds corresponding to different tags may be the same or different.
And S30, analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement.
Analyzing the current security check behavior according to the target personnel tag information, and monitoring whether the current security check behavior meets the preset security check requirement; for example, the target person tag information output by the multi-tag classification model is that the tag information of the target person is a ' long-sleeve ', ' long-hair ', ' supplier ', ' long-sleeve ', ' long-hair ', ' supplier ' and ' tag corresponding to the security inspection requirement is that the security inspection wrist and the hair are detected by the security inspection instrument, whether the current security inspection action performs the actions of the security inspection wrist and the security inspection hair is analyzed, whether the security inspection instrument is used for detection is detected, and if yes, the security inspection action of the security inspector can be determined to meet the preset security inspection requirement.
Through the steps, security inspection images at a plurality of visual angles are obtained; inputting security inspection images at multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection; the method comprises the steps of analyzing current security check behaviors according to target personnel label information, monitoring whether the current security check behaviors meet preset security check requirements, analyzing multi-view security check images through a multi-label classification model to obtain personnel label information of a person to be checked, and then automatically monitoring whether the current security check behaviors of the security checker meet security check specifications according to the security check requirements corresponding to the personnel label information, so that the technical problem that manual supervision of the security check behaviors in related technologies is difficult to achieve supervision all the time is solved, and the monitoring efficiency of the security check behaviors is improved.
In an embodiment of this embodiment, analyzing the current security check behavior according to the target person tag information, and monitoring whether the current security check behavior meets a preset security check requirement includes:
step A, outputting a security inspection part label matched with the label information of the target personnel by adopting the multi-label classification model;
the security inspection part labels matched with the label information of the target personnel are output by adopting a multi-label classification model, and the security inspection part labels matched with the label information of the target personnel exist for different label information of the target personnel, for example, the label information of the target personnel is long-sleeved, the label information of the security inspection part corresponds to body parts including elbows and wrists, the label information of the target personnel is short-sleeved, the label information of the security inspection part does not include body parts including elbows and wrists, the label information of the security inspection part can include body parts including necks, left shoulders, right shoulders, left elbows, right elbows, left wrists, right wrists, and the like, and the body parts can be increased and refined according to security inspection requirements.
B, acquiring security inspection action images at a plurality of visual angles;
step C, inputting the security inspection action images under the multiple visual angles into a multi-label classification model trained in advance,
step D, obtaining action tags and corresponding confidence degrees of the security inspection part tags output by the multi-tag classification model, wherein the confidence degrees corresponding to the action tags are used for representing the probability of security inspection of a security inspector on an inspected part of an inspected person according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part tags;
and E, if the confidence corresponding to the action tag is greater than a preset threshold, determining that the security check behavior meets the preset security check requirement.
Further collecting security inspection action images at multiple viewing angles, inputting the security inspection action images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring action labels and corresponding confidence degrees of security inspection part labels output by the multi-label classification model, wherein the confidence degrees corresponding to the action labels are used for expressing the probability of security inspection of a security inspector on an inspected part of an inspected person according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part labels; the action tags are that if a security inspector touches one or more inspected parts of the inspected person by hands, the touched inspected parts are determined to be finished by security inspection, the corresponding security inspection part tags indicate that the security inspector finishes security inspection according to preset security inspection requirements, if the hand positions of the security inspector do not touch the inspected parts, the touched inspected parts are determined not to be subjected to security inspection, and the corresponding security inspection part tags indicate that the security inspector does not perform security inspection according to the preset security inspection requirements, for example, if the hands of the security inspector are kept in the air or have a preset distance from the inspected parts, the security inspector determines that the security inspector is in an 'stateless' state, and the tags indicate that a person exists but do not have clear security inspection actions. If the confidence degree corresponding to the action tag is greater than a preset threshold value, determining that the security check behavior meets the preset security check requirement, for example, if the confidence degree corresponding to the action tag of the left shoulder of the security check part tag is 90% and the preset threshold value is 60%, determining that a security checker performs security check on the left shoulder according to the preset security check requirement, and determining that the security check behavior meets the preset security check requirement, wherein one or two parts can reach the standard at the same time due to two-hand security check.
In this embodiment, the step of inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model includes:
step S201, inputting the security inspection images at multiple viewing angles into a pre-trained multi-label classification model, and acquiring a first confidence coefficient of a person appearance label and a second confidence coefficient corresponding to a person identity attribute label output by the multi-label classification model;
step S202, respectively determining the appearance attribute and the identity attribute of the person to be detected according to the first confidence coefficient of the person appearance label and the second confidence coefficient corresponding to the person identity attribute label;
and step S203, adding the target personnel label information on the security inspection image according to the identity attribute and the appearance attribute.
The security inspection images at multiple visual angles are input into a pre-trained multi-label classification model, and a first confidence coefficient of a person appearance label and a second confidence coefficient corresponding to a person identity attribute label output by the multi-label classification model can be obtained; during model training, all personnel appearance labels and personnel identity attribute labels are assigned to be one of '1', '0' and '1', and respectively represent 'yes', 'not' and 'neglect'; for the condition that the label assignment is 0 or 1, each person appearance label corresponds to a first confidence score, the value interval corresponding to the first confidence score is [0,1], the larger the first confidence score is, the more the corresponding person appearance label is determined to be 'yes', the multi-label classification model corresponds to a score on all person appearance labels (such as 'short sleeve', 'long hair', 'short hair', 'hat wearing' and the like), only the person appearance labels with the scores larger than a preset threshold value need to be determined from all the person appearance labels, only the person appearance labels with the scores larger than the preset threshold value are output, and the higher the scores corresponding to the labels are, the higher the probability of the corresponding person appearance attributes is, so that the corresponding appearance attributes of the detected person are determined according to the person appearance labels with the first confidence scores larger than the preset threshold value, and the corresponding appearance attributes are added to the security inspection image. Under normal conditions, the multi-label classification model only has one personnel identity attribute label with a higher second confidence score in all personnel identity attribute labels, and the second confidence scores of other personnel identity attribute labels are lower, so that the personnel identity attribute label with the second confidence reaching a preset threshold is determined, the personnel identity attribute of the detected person can be determined according to the personnel identity attribute label with the second confidence reaching the preset threshold, for example, the multi-label classification model outputs that the second confidence corresponding to a supplier is 80%, the second confidence corresponding to other safety inspectors, other customers, cooperation police and employees is respectively 11%, 12%, 13% and 14%, the second confidence corresponding to the supplier is far lower than the second confidence corresponding to the supplier and is 60%, the second confidence corresponding to the supplier is higher than the preset threshold, which indicates that the detected person is the supplier identity attribute, the personnel label information of the supplier is added to the safety inspection image, the visual image of the visual inspection image and the visual image of the detected person are subjected to visual inspection according to the visual classification model, and the visual image of the visual inspection image and the visual image of the visual inspection person are displayed in a real-time.
Further, the security inspection image of each frame and the personnel label information marked on the security inspection image are reserved and stored; the historical video corresponding to each in-and-out examinee is independently stored as a security check record, and a user can check the historical video through a platform; the security check time and the security check part of each off-duty personnel are written into the database; the security inspection behavior data analysis of the current time and the historical time can be carried out at the application end, for example, the number of people who appear on duty, the abnormal number, the abnormal proportion, the proportion of abnormal parts and the like every day (or multiple days) can be checked aiming at different security inspection doorposts, so that the security protection strategy can be improved and promoted in a targeted manner. Referring to fig. 3, the video monitoring and collecting system collects security inspection images at multiple viewing angles, i.e., obtains multiple video frames, and preprocesses the multiple video frames, where specific preprocessing process may refer to the following steps L-N, which are not described herein, so as to perform personnel entry and exit analysis, personnel attribute analysis, and security inspection action analysis through the multiple-viewing-angle multiple-label classification model, and further perform data storage and data analysis, visual display, and abnormal alarm processing.
In an embodiment of this embodiment, the multi-label classification model includes a feature extraction module, a feature fusion module, and a multi-label prediction module, and the step of inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model includes:
step F, inputting the security inspection images at the multiple viewing angles into the feature extraction module, and extracting feature information corresponding to the personnel label information at each viewing angle;
step G, inputting the feature information into the feature fusion module, and fusing the corresponding feature information under a plurality of visual angles;
and H, inputting the fused characteristic information into the multi-label prediction module to obtain the label information of the target personnel.
Referring to fig. 5, a module i (feature extraction layer), that is, a feature extraction module, inputs security inspection images image1 and image2 at left and right viewing angles into the feature extraction module, and extracts feature information corresponding to personnel tag information at each viewing angle, where the feature extraction layer is implemented in a weight sharing manner. And then inputting the feature information of the left and right visual angles into a second module (a feature fusion layer), namely a feature fusion module, namely superposing, combining and fusing the corresponding feature information under a plurality of visual angles into a whole, and inputting the fused feature information into a third module (a prediction layer), namely a multi-label prediction module, for secondary learning to obtain the label information of the target personnel. For example, the convolutional neural network performs feature extraction on the left and right visual angles, the two features are superposed and combined into a whole, and the fusion process can be as follows: and processing the characteristic information corresponding to the security inspection image at each view angle through a splicing mechanism, an Attention mechanism, an AvgPool (average pooling) mechanism and other mechanisms to obtain the fused characteristic information. And then extracting the characteristics of the whole body, and obtaining the final output probability by carrying out operation on the fused characteristics through a plurality of groups of Residual Block convolutional Layers, full Connected Layers (FC) and Sigmoid. In this embodiment, a process of feature extraction in the feature extraction module is denoted as a first stage, a process of feature extraction in the multi-label prediction module is denoted as a second stage, multiple security inspection images are continuously learned through the first stage and the second stage to generate a one-dimensional feature vector, each feature corresponds to a confidence score, the confidence scores are used for describing conditions under multiple viewing angles, and finally, a corresponding confidence score is obtained under a corresponding label through processing, for example, the confidence score on the long-sleeve label is 0.8.
Specifically, the structure diagram of the feature extraction module shown in fig. 6 includes a set of Convolution Block (Convolution Block) and a set of Residual Block (Residual Block), and the Residual Block can be extended to multiple sets to implement more abstract feature extraction. The restriction Block is composed of a plurality of Convolution layers and a group of Maxpool (Maxpool is the maximum value in a region), the Residual Block is divided into an upper part and a lower part, the first part comprises a group of 1x1 Convolution layers on the left side and a plurality of groups of Convolution layers on the right side, then the left part and the right part are added to serve as outputs, the second part only comprises a plurality of groups of Convolution layers, and then the inputs and the outputs are added to serve as final outputs.
FIG. 7 is a schematic diagram of a feature fusion module, which includes a set of Attention Block. Wherein the composite material comprises a group of splicing layers, a group of convolution layers and an AvgPool layer. And splicing the multiple groups of features according to the spatial dimension, processing the convolution layer and Sigmoid (mapped to a relation function of [0,1 ]) to obtain fusion weight, multiplying the convolution layer and Sigmoid, and performing AvgPool processing to obtain the fused features, wherein the AvgPool is dynamically adjusted according to the input feature quantity.
Fig. 8 is a schematic structural diagram of a multi-label prediction module, which includes a plurality of groups of Residual Block, an AvgPool module, and an FC + Sigmoid module. The Residual Block is basically similar to the Residual Block of the Feature extraction module, only the number of convolution layers and the number of channels are adjusted, the AvgPool module compresses the Feature Map of the convolution layer to obtain a Feature vector, and finally, the final output probability is obtained through FC + Sigmoid. The whole multi-label classification model structure carries out multi-label classification training under multiple visual angles in an end-to-end mode, and can fully utilize information of multi-visual angle image data to solve the problem of visual occlusion, so that the accuracy and recall rate of behavior analysis of security personnel on a security post are improved, and missing reports and false reports are reduced.
In this embodiment, before the security inspection images at the multiple viewing angles are input into a multi-label classification model trained in advance to obtain the label information of the target person output by the multi-label classification model, the method further includes:
step I, obtaining a security inspection image sample;
step J, setting sample labels of the security check image samples, wherein the sample labels comprise positive sample labels, negative sample labels and ignore labels, the positive sample labels are used for identifying that the corresponding image samples comprise corresponding personnel attributes or security check actions, the negative sample labels are used for identifying that the corresponding image samples do not comprise corresponding personnel attributes or security check actions, and the ignore labels are used for indicating that the security check image samples are not used when the designated personnel attributes or security check actions are trained;
and K, training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model.
Training an initial model through a security inspection image sample to obtain a multi-label classification model, setting sample labels of the security inspection image sample, wherein the sample labels comprise a positive sample label, a negative sample label and a neglecting labelThe positive sample label is used for identifying that the corresponding image sample comprises the corresponding personnel attribute or security check action, understandably, if the label i corresponding to the image sample K is true, for example, if the image sample really contains a person, the 'person' label is 1, namely
Figure BDA0003791401290000141
The negative sample label is used for identifying that the corresponding image sample does not comprise the corresponding personnel attribute or the security check action, if the label i corresponding to the image sample K is not true, for example, no person exists in the image sample, the 'person' label is 0, namely
Figure BDA0003791401290000142
In order to reduce data interference and be more beneficial to learning of a model, special cases such as partially fuzzy sample labels and 'no person' are additionally added
Figure BDA0003791401290000143
An ignore tag i indicating that sample K needs to be ignored. The ignoring label is used for indicating that the security inspection image sample is not used when the designated personnel attribute or the security inspection action is trained, and the personnel attribute corresponding to the ignoring label is in an uncertain intermediate state; for example, if the image sample is uncertain whether a person exists, the sample is ignored and does not participate in training to reduce data interference, and for example, the hair of the person to be examined in the image sample is a long hair wound up in a disc, and whether the person to be examined is a short hair cannot be accurately determined from a front view, so that the security image sample is not used when the attributes of the person with the long hair and the short hair are trained, and the label data is a one-dimensional vector consisting of-1, 0 and 1. And then training the initial model by using a security inspection image sample and a loss function to obtain a multi-label classification model, meanwhile, updating model parameters by adopting a gradient descent method based on Adam, training for 40 periods in total, and setting the batch size (BatchSize) to be 64.
Further, training an initial model by using the security inspection image sample and the loss function, and obtaining the multi-label classification model comprises:
training an initial model by using the security inspection image sample and the following Loss function Loss to obtain the multi-label classification model:
Figure BDA0003791401290000151
Figure BDA0003791401290000152
wherein: 1[ 2 ], [ y ] i ≠-1]Is shown when y i When not equal to-1, the output is 1, otherwise the output is 0, i.e. samples labeled-1 are ignored.
This example uses Binary Cross Entropy (BCE) as a loss function. That is, assuming that the prediction probability of the security inspection image to be classified is P, the label is Y = { Y = { (Y) } 1 ，y 2 ，...，y n Yi is the label value corresponding to the ith label, the label value is 0 or 1, p is the probability value predicted by the multi-label classification model for each label, the probability value may be the confidence level in this embodiment, and the probability value is [0,1]]I.e. any value between 0 and 1, [ -y [ - ] i log(P)-(1-y i )log(1-P)]Is a binary cross entropy, i.e. the cross entropy of positive samples plus the cross entropy of negative samples,
Figure BDA0003791401290000153
for each label from i = label No. 1 to i = label No. n, each label corresponds to a value of cross entropy, the values of cross entropy corresponding to all labels are accumulated, wherein: 1[ 2 ], [ y ] i ≠1]Is shown when y i With ≠ -1, the output is 1 or the output is 0, i.e. samples labeled-1 are ignored. m is the number of tags with tag values not-1 (i.e., tag values of 0 or 1), i.e., ignoring tags with tag values of-1, does not participate in the summation, n is the number of all tags, and m is the number of tags with tag values not-1 among all tags. The embodiment provides a multi-label classification model method based on multiple visual angles, which is used for obtaining security inspection images of security inspector behavior analysis of security inspection posts to be classified under multiple visual angles, and obtaining multi-label classification of the images through feature extraction, feature fusion, result prediction and the likeAnd (4) obtaining the result. The visual occlusion of the image data at multiple viewing angles can be effectively reduced, and the accuracy and recall rate of behavior analysis of security personnel at a security post can be improved, so that the missing report and the false report are reduced.
Further, in an embodiment of this embodiment, before inputting the security inspection images at the multiple viewing angles into a multi-label classification model trained in advance to obtain the label information of the target person output by the multi-label classification model, the method further includes:
step L, acquiring an original video frame sequence under each visual angle;
step M, respectively acquiring current video frames in the original video frame sequences under all visual angles at the same moment;
and step N, according to a preset security check demarcation area, cutting the current video frame to obtain a corresponding security check area image under each visual angle, and taking the security check area image as the security check image.
The method comprises the steps of obtaining an original video frame sequence under each visual angle, obtaining a current video frame in the original video frame sequence under each visual angle at the same moment, defining areas according to preset security check, cutting the current video frame to obtain a corresponding security check area image under each visual angle, and inputting the security check area image as a security check image into a multi-label classification model for reasoning. After the video monitoring equipment is implemented on site, a security inspector and a human to be inspected perform simulation actions, the whole security inspection action is simulated, a security inspection area is manually framed in image configuration software, and a preset security inspection demarcating area is the security inspection area framed manually. The security demarcated area can include security doormats, security inspectors, and examinees such that all actions are in the frame and the examinees are in the middle of the field of view as much as possible and the security action range is greater than 80% (other values are also possible). Obtaining a stream from a network camera through a program, acquiring an original video frame sequence under each visual angle, taking a left visual angle and a right visual angle as an example for explanation, and if a monitoring video cannot be read normally, performing monitoring exception processing to wait for recovery and then performing subsequent image analysis; a program maintains a plurality of video frame buffer areas under a plurality of visual angles, the buffer areas store original video frame sequences and respectively acquire current video frames in the original video frame sequences under the visual angles at the same moment; the program respectively takes out the latest video frame under each visual angle at the same moment from a plurality of buffer area queues each time; after the video frame is obtained, a security inspection area graph is cut out through a program opencv according to a security inspection area defined after the field camera is erected; and after the security inspection areas are intercepted, processing each security inspection area image into images with the same size, forming a group of multi-view image data, and sending the multi-view image data into the multi-label classification model for reasoning.
Further, after monitoring whether the current security check behavior meets the preset security check requirement, if the current security check behavior does not meet the preset security check requirement, an abnormal prompt is output, and the gate entering and exiting machine is controlled to be closed. And when the checked person leaves the security check area, the security check action is finished, and final judgment is made on the security check action. If the abnormal flow is judged to be missed or not detected, a series of abnormal processing is immediately carried out. The processing mode comprises the following steps: an abnormal window is popped up on the web system in time, and personnel in a monitoring room can see abnormal prompt information; directly controlling the gate machine to enter and exit from the gate of the on-site entrance and exit station in a linkage way through an interface, actively closing the gate machine and sending out sound prompt; and (4) pushing a short message of the mobile phone, and directly sending abnormal information (time, gate post and which kind of abnormality) to the mobile phone of the on-duty person in the week after an abnormality alarm. The embodiment can effectively monitor whether the security check operation of the security inspector is standard or not, and can utilize limited hardware resources to perform security check behavior analysis with high real-time rate and high accuracy. The security inspection process of each in-out employee can be digitalized and monitored and checked in real time. The covering of single-gate sentry, double-gate sentry, 3-gate sentry and more than 3-gate sentry can be finished, the false detection rate is lower than 5% through analysis and statistics, the abnormal capturing efficiency of the video is extremely high, and the risk of omission of security inspection actions of a security inspector is effectively reduced; secondly, the labor cost is saved mainly in the aspect of saving the cost.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
Example 2
In this embodiment, a monitoring apparatus for a security check behavior is further provided, which is used to implement the foregoing embodiments and preferred embodiments, and the description of the foregoing embodiments is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware or a combination of software and hardware is also possible and contemplated.
Fig. 9 is a block diagram of a monitoring apparatus for security check activities according to an embodiment of the present invention, and as shown in fig. 9, the apparatus includes: an acquisition module 60, a label determination module 61, an analysis module 62, wherein,
an acquiring module 60, configured to acquire security inspection images at multiple viewing angles;
the label determining module 61 is configured to input the security inspection images at the multiple viewing angles into a multi-label classification model trained in advance to obtain target personnel label information output by the multi-label classification model, where the multi-label classification model includes multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection;
and the analysis module 62 is configured to analyze the current security check behavior according to the target person tag information, and monitor whether the current security check behavior meets a preset security check requirement.
Optionally, the analysis module includes a first analysis module, and the first analysis module is configured to output a security inspection location tag matched with the target person tag information by using the multi-tag classification model; acquiring security inspection action images at a plurality of visual angles; inputting the security inspection action images at the multiple visual angles into a pre-trained multi-label classification model, and acquiring action labels and corresponding confidence degrees of the security inspection part labels output by the multi-label classification model, wherein the confidence degrees corresponding to the action labels are used for representing the probability of security inspection of the inspected part of the inspected person by a security inspector according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part labels; and if the confidence corresponding to the action tag is greater than a preset threshold, determining that the security check behavior meets a preset security check requirement.
Optionally, the tag determination module includes a first determination module, and the first determination module is configured to input the security inspection images at the multiple viewing angles into a pre-trained multi-tag classification model, and obtain a first confidence level of the person appearance tag and a second confidence level corresponding to the person identity attribute tag, which are output by the multi-tag classification model; respectively determining the appearance attribute and the identity attribute of the person to be detected according to the first confidence coefficient of the person appearance tag and the second confidence coefficient corresponding to the person identity attribute tag; and adding the target personnel label information on the security inspection image according to the identity attribute and the appearance attribute.
Optionally, the tag determination module includes a second determination module, and the second determination module is configured to input the security inspection images at the multiple viewing angles into the feature extraction module, and extract feature information corresponding to the personnel tag information at each viewing angle; inputting the feature information into the feature fusion module, and fusing the corresponding feature information under a plurality of visual angles; and inputting the fused characteristic information into the multi-label prediction module to obtain the label information of the target personnel.
Optionally, the monitoring device for the security check behavior includes a model training module, where the model training module is used to obtain a security check image sample; setting sample labels of the security check image samples, wherein the sample labels comprise positive sample labels, negative sample labels and ignore labels, the positive sample labels are used for identifying that the corresponding image samples comprise corresponding personnel attributes or security check actions, the negative sample labels are used for identifying that the corresponding image samples do not comprise corresponding personnel attributes or security check actions, and the ignore labels are used for indicating that the security check image samples are not used when the designated personnel attributes or security check actions are trained; and training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model.
Optionally, the model training module is further configured to train an initial model by using the security inspection image sample and the following Loss function Loss, to obtain the multi-label classification model:
Figure BDA0003791401290000181
wherein, y i The value of the label value corresponding to the ith label is 0 or 1, P is the probability value predicted by the multi-label classification model aiming at each label, and the value range of the probability value is [0,1]]，
Figure BDA0003791401290000182
Is the sum of the cross entropies corresponding to all tags from i =1 to i = n, 1[ y ], [ i ≠-1]Is shown when y i When not equal to-1, the output is 1 or the output is 0, m is the number of tags having a tag value of 0 or 1, and n is the number of all tags.
Optionally, the monitoring apparatus for the security check behavior includes a data preprocessing module, where the data preprocessing module is configured to obtain an original video frame sequence at each view angle; respectively acquiring current video frames in an original video frame sequence under each visual angle at the same moment; and cutting the current video frame according to a preset security check demarcation area to obtain a corresponding security check area map under each visual angle, and taking the security check area map as the security check image.
It should be noted that, the above modules may be implemented by software or hardware, and for the latter, the following may be implemented, but not limited to: the modules are all positioned in the same processor; alternatively, the modules are respectively located in different processors in any combination.
Example 3
Embodiments of the present invention also provide a storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the storage medium may be configured to store a computer program for executing the steps of:
s1, obtaining security inspection images at multiple visual angles;
s2, inputting the security inspection images at the multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection;
and S3, analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement.
Optionally, in this embodiment, the storage medium may include but is not limited to: various media capable of storing computer programs, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Embodiments of the present invention also provide an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, obtaining security inspection images at a plurality of visual angles;
s2, inputting the security inspection images at the multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection;
and S3, analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
The above-mentioned serial numbers of the embodiments of the present application are merely for description, and do not represent the advantages and disadvantages of the embodiments.
In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (10)

1. A method for monitoring security activities, the method comprising:
acquiring security inspection images at a plurality of visual angles;
inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a group of security inspection behaviors required by security inspection;
and analyzing the current security check behavior according to the target personnel label information, and monitoring whether the current security check behavior meets the preset security check requirement.
2. The method of claim 1, wherein analyzing a current security check behavior according to the target personnel tag information, and monitoring whether the current security check behavior meets a preset security check requirement comprises:
outputting a security inspection part label matched with the label information of the target personnel by adopting the multi-label classification model;
acquiring security inspection action images at a plurality of visual angles;
inputting the security inspection action images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring action labels and corresponding confidence degrees of the security inspection part labels output by the multi-label classification model, wherein the confidence degrees corresponding to the action labels are used for representing the probability of security inspection of a security inspector on an inspected part of an inspected person according to a preset security inspection requirement, and the inspected part corresponds to the security inspection part labels;
and if the confidence corresponding to the action tag is greater than a preset threshold, determining that the security check behavior meets a preset security check requirement.
3. The method of claim 1, wherein the step of inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model comprises:
inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model, and acquiring a first confidence coefficient of a person appearance label and a second confidence coefficient corresponding to a person identity attribute label output by the multi-label classification model;
respectively determining the appearance attribute and the identity attribute of the person to be detected according to the first confidence coefficient of the person appearance tag and the second confidence coefficient corresponding to the person identity attribute tag;
and adding the target personnel label information on the security inspection image according to the identity attribute and the appearance attribute.
4. The method according to claim 1, wherein the multi-label classification model comprises a feature extraction module, a feature fusion module and a multi-label prediction module, and the step of inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model comprises the steps of:
inputting the security inspection images at the multiple viewing angles into the feature extraction module, and extracting feature information corresponding to the personnel label information at each viewing angle;
inputting the feature information into the feature fusion module, and fusing the corresponding feature information under a plurality of visual angles;
and inputting the fused characteristic information into the multi-label prediction module to obtain the label information of the target personnel.
5. The method of claim 1, wherein before the security inspection images at the plurality of viewing angles are input into a pre-trained multi-label classification model to obtain the label information of the target person output by the multi-label classification model, the method further comprises:
obtaining a security inspection image sample;
setting sample labels of the security check image samples, wherein the sample labels comprise positive sample labels, negative sample labels and ignore labels, the positive sample labels are used for identifying that the corresponding image samples comprise corresponding personnel attributes or security check actions, the negative sample labels are used for identifying that the corresponding image samples do not comprise corresponding personnel attributes or security check actions, and the ignore labels are used for indicating that the security check image samples are not used when the designated personnel attributes or security check actions are trained;
and training an initial model by using the security inspection image sample and the loss function to obtain the multi-label classification model.
6. The method of claim 5, wherein training an initial model using the security check image samples and a loss function to obtain the multi-label classification model comprises:
training an initial model by using the security inspection image sample and the following Loss function Loss to obtain the multi-label classification model:
Figure FDA0003791401280000031
Figure FDA0003791401280000032
wherein, y i The value of the label value corresponding to the ith label is 0 or 1, P is the probability value predicted by the multi-label classification model aiming at each label, and the value range of the probability value is [0,1]]，
Figure FDA0003791401280000033
Is the sum of the cross entropies corresponding to all tags from i =1 to i = n, 1[ y ], [ i ≠-1]Is shown when y i When not equal to-1, the output is 1 or the output is 0, m is the number of tags having a tag value of 0 or 1, and n is the number of all tags.
7. The method of claim 1, wherein before inputting the security inspection images at the multiple viewing angles into a pre-trained multi-label classification model and obtaining the label information of the target person output by the multi-label classification model, the method further comprises:
acquiring an original video frame sequence under each visual angle;
respectively acquiring current video frames in an original video frame sequence under each visual angle at the same moment;
and according to a preset security check demarcation area, cutting the current video frame to obtain a corresponding security check area map under each visual angle, and taking the security check area map as the security check image.
8. A device for monitoring security activities, comprising:
the acquisition module is used for acquiring security inspection images at multiple visual angles;
the label determining module is used for inputting the security inspection images at the multiple visual angles into a pre-trained multi-label classification model to obtain target personnel label information output by the multi-label classification model, wherein the multi-label classification model comprises multiple personnel label information, and each personnel label information corresponds to a set of security inspection behaviors required by security inspection;
and the analysis module is used for analyzing the current security check behavior according to the target personnel label information and monitoring whether the current security check behavior meets the preset security check requirement.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus; wherein:
a memory for storing a computer program;
a processor for performing the method steps of any of claims 1 to 7 by executing a program stored on a memory.
10. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program is operative to perform the method steps of any of the preceding claims 1 to 7.
CN202210956039.5A 2022-08-10 2022-08-10 Security check behavior monitoring method and device, electronic equipment and storage medium Pending CN115240277A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210956039.5A CN115240277A (en) 2022-08-10 2022-08-10 Security check behavior monitoring method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210956039.5A CN115240277A (en) 2022-08-10 2022-08-10 Security check behavior monitoring method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115240277A true CN115240277A (en) 2022-10-25

Family

ID=83679819

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210956039.5A Pending CN115240277A (en) 2022-08-10 2022-08-10 Security check behavior monitoring method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115240277A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116189101A (en) * 2023-04-28 2023-05-30 公安部第一研究所 Method and system for identifying, judging and guiding visual operation specification of security inspector

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116189101A (en) * 2023-04-28 2023-05-30 公安部第一研究所 Method and system for identifying, judging and guiding visual operation specification of security inspector
CN116189101B (en) * 2023-04-28 2023-08-04 公安部第一研究所 Method and system for identifying, judging and guiding visual operation specification of security inspector

Similar Documents

Publication Publication Date Title
US10812761B2 (en) Complex hardware-based system for video surveillance tracking
CN110674772B (en) Intelligent safety control auxiliary system and method for electric power operation site
JP7014440B2 (en) Video surveillance system, video processing method and video processing program
CN110428522A (en) A kind of intelligent safety and defence system of wisdom new city
CN109922310A (en) The monitoring method of target object, apparatus and system
CN109309808A (en) A kind of monitoring system and method based on recognition of face
CN106355367A (en) Warehouse monitoring management device
KR102149832B1 (en) Automated Violence Detecting System based on Deep Learning
CN112396658A (en) Indoor personnel positioning method and positioning system based on video
CN111191507A (en) Safety early warning analysis method and system for smart community
CN114863489B (en) Virtual reality-based movable intelligent auxiliary inspection method and system for construction site
CN115240277A (en) Security check behavior monitoring method and device, electronic equipment and storage medium
CN111191498A (en) Behavior recognition method and related product
CN113807240A (en) Intelligent transformer substation personnel dressing monitoring method based on uncooperative face recognition
CN115083229B (en) Intelligent recognition and warning system of flight training equipment based on AI visual recognition
CN111860187A (en) High-precision worn mask identification method and system
Hameete et al. Intelligent Multi-Camera Video Surveillance.
CN115190277B (en) Safety monitoring method, device and equipment for construction area and storage medium
CN115829324A (en) Personnel safety risk silent monitoring method
CN114429677A (en) Coal mine scene operation behavior safety identification and assessment method and system
CN111126771B (en) Safety inspector identification picture quality supervision and guarantee system and method based on regional attention prediction
CN112347889A (en) Substation operation behavior identification method and device
CN112528825A (en) Station passenger recruitment service method based on image recognition
CN110443197A (en) A kind of visual scene intelligent Understanding method and system
CN207995400U (en) A kind of decision-making system of multiple body binding relationships based on wireless sensor

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination