CN112818802B - Bank counter personnel hand-lifting identification method and system - Google Patents
Bank counter personnel hand-lifting identification method and system Download PDFInfo
- Publication number
- CN112818802B CN112818802B CN202110105141.XA CN202110105141A CN112818802B CN 112818802 B CN112818802 B CN 112818802B CN 202110105141 A CN202110105141 A CN 202110105141A CN 112818802 B CN112818802 B CN 112818802B
- Authority
- CN
- China
- Prior art keywords
- hand
- worker
- lifting
- picture
- working
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a bank counter personnel hand-lifting identification method and a system, wherein the method comprises the following steps: respectively acquiring work videos and pictures of workers at a bank counter by using an image acquisition device and respectively preprocessing the work videos and the pictures; inputting the preprocessed data into a target detection model for behavior analysis recognition and reasoning, and outputting the positions of workers and hands of the workers in a working video and a picture; setting a hand-lifting contact degree threshold value, calculating the position frame of each worker and the hand-lifting contact degree of the worker in the working video and the working picture, and comparing the calculated hand-lifting contact degree with the set hand-lifting contact degree threshold value; and if the calculated hand lifting contact ratio is greater than the set hand lifting contact ratio threshold value, the worker is considered to have a hand lifting behavior, otherwise, the worker is determined not to have the hand lifting behavior. The invention can complete the hand-lifting action recognition on a GPU 1080Ti host at the inference speed of 10+ ms, thereby improving the target event recognition precision and efficiency.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a method and a system for identifying hands of bank counter personnel.
Background
At a cash register of a consuming place such as a supermarket, a hotel, a bank or a shopping mall, a cashier can use the cash register to settle the consumption behavior of a customer. Meanwhile, the merchant can check whether the cashier's cash registering behavior is illegal according to the video monitored or recorded by the monitoring system and the relevant cash registering behavior specification.
For banks with manual service, no matter whether customers need to handle low-authority or high-authority services, the non-differentiated processing mode is adopted, counter service personnel with manual service can handle the banks in the whole process, but the problem that whether the behavior of bank counter personnel meets the behavior specification in the manual service process or not can not be found in time by monitoring of the personnel, the labor cost is increased in the monitoring process, and the behavior specification of the counter personnel cannot be supervised and managed in time.
Meanwhile, the conventional gesture recognition technical means are usually time-consuming, and key point data collection and labeling aiming at internal monitoring of a depression angle are difficult. Therefore, a method and a system capable of automatically recognizing the hand-lifting action of the worker are lacked.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method and a system for identifying the hands held by bank counter personnel. And setting a threshold value of the contact ratio, calculating the contact ratio of each lifted hand in the position frame and the image of each person, comparing the contact ratio with the threshold value, and if the obtained calculated value exceeds the threshold value, considering that the person lifts the hand, otherwise, not considering that the currently judged person lifts the hand.
The purpose of the invention is realized by the following technical scheme:
a bank counter personnel hand-lifting identification method comprises the following steps:
the method comprises the following steps: acquiring image data, respectively acquiring work videos and pictures of staff at a bank counter by using an image acquisition device, and respectively preprocessing the acquired work videos and pictures;
step two, behavior recognition and inference, namely inputting the preprocessed working video and pictures into a target detection model for behavior analysis recognition and inference, and outputting the positions of workers in the working video and pictures and the positions of hands of the workers during hand lifting;
step three: behavior comparison and judgment, namely setting a hand lifting contact degree threshold value, calculating the position frame of each worker in the working video and the picture and the hand lifting contact degree of the hands of the workers, and comparing the calculated hand lifting contact degree with the set hand lifting contact degree threshold value; and if the calculated hand lifting contact ratio is greater than the set hand lifting contact ratio threshold value, the worker is considered to have a hand lifting behavior, otherwise, the worker is determined not to have the hand lifting behavior.
Specifically, the second step specifically includes the following substeps:
s201, acquiring and preprocessing working videos and pictures of bank workers through monitoring equipment, and inputting the preprocessed working videos and pictures into a CNN-backbone network extraction feature map;
s202, a CNN-backbone network extracts feature graphs to accelerate circulation of feature information through a PANET structure, and a similar YOLOV3-Head detection Head is used for generating detection result feature graphs of three different scales;
and S203, performing spatial filtering on the detection result characteristic diagram through the adaptive ASFF module, filtering out conflict information, and converting and splicing the detection result characteristic diagram to output the body position of the worker and the hand position of the worker during hand lifting in the working video and picture.
Specifically, the preprocessing process of the collected working video and picture in the first step specifically includes:
(1) the adaptive picture scaling of the input image specifically comprises the following steps: firstly, the scaling coefficients of the length and the width are calculated according to the length and the width of the input model. And then selecting the smaller scaling coefficient as the overall scaling coefficient, and multiplying the original length and width by the scaling coefficient to obtain the scaled size. The width of the original image is larger than the height, the height is filled with black edges, and because the model is sampled by 32 times, the size of the width of the picture is filled to be 32 times, and the difference with the scaled size is minimum. And after calculating the difference value, performing black edge supplement on the upper part and the lower part of the picture.
(2) And after the picture pixel points are divided by 255 for normalization, exchanging channels to obtain the characteristic value of the input model.
A bank counter personnel hand-lifting identification system comprises an image preprocessing module, a target detection module, a central calculation module, a comparison module and an alarm module;
the image preprocessing module is used for acquiring and acquiring working videos and pictures of staff in a bank counter in real time and preprocessing the acquired working videos and pictures;
the target detection module is used for reasoning and analyzing the preprocessed working video and picture and outputting the body position of the worker and the hand position of the worker when the worker lifts the hand in the working video and picture;
the central computing module is used for computing the coincidence degree of a position frame of each worker in the monitoring image and the hand-lifting action of each worker in the working video and the picture;
the comparison module is used for comparing the calculated contact ratio of the hand lifting action of each worker with a set contact ratio threshold value according to a preset contact ratio threshold value of the hand lifting action of the worker, and finally outputting a comparison result.
The invention has the beneficial effects that:
1. the simple model architecture is beneficial to being transplanted to various edge devices for operation.
2. The training data required by the model is collected and manufactured simply, and the efficiency is saved
3. The model based on target detection has high performance, the speed can reach 10+ ms, and the industrial requirement is met.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of a hand lifting identification process according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a hand raising recognition model according to an embodiment of the present invention.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will now be described with reference to the accompanying drawings.
In this embodiment, as shown in fig. 1, a method for identifying a bank counter person by raising a hand includes the following steps:
the method comprises the following steps: acquiring image data, respectively acquiring work videos and pictures of staff at a bank counter by using an image acquisition device, and respectively preprocessing the acquired work videos and pictures;
step two, behavior recognition and inference, namely inputting the preprocessed working video and pictures into a target detection model for behavior analysis recognition and inference, and outputting the positions of workers in the working video and pictures and the positions of hands of the workers during hand lifting;
step three: performing behavior comparison and judgment, setting a threshold value of the overlap ratio of hands, calculating the overlap ratio of the position frame of each worker and the hands of the workers in the working video and the pictures, and comparing the calculated overlap ratio of hands with the set threshold value of the overlap ratio of hands; and if the calculated hand lifting contact ratio is greater than the set hand lifting contact ratio threshold value, the worker is considered to have a hand lifting behavior, otherwise, the worker is determined not to have the hand lifting behavior.
Specifically, the second step specifically includes the following substeps:
s201, acquiring and preprocessing working videos and pictures of bank workers through monitoring equipment, and inputting the preprocessed working videos and pictures into a CNN-backbone network extraction feature map;
s202, a CNN-backbone network extracts feature graphs to accelerate circulation of feature information through a PANET structure, and a similar YOLOV3-Head detection Head is used for generating detection result feature graphs of three different scales;
and S203, performing spatial filtering on the detection result characteristic diagram through the adaptive ASFF module, filtering out conflict information, and converting and splicing the detection result characteristic diagram to output the body position of the worker and the hand position of the worker during hand lifting in the working video and picture.
Specifically, the preprocessing process of the collected working video and pictures in the first step specifically includes:
(1) the process of the adaptive picture scaling of the input image specifically comprises the following steps: firstly, the scaling coefficients of the length and the width are calculated according to the length and the width of the input model. And then selecting the smaller scaling coefficient as the overall scaling coefficient, and multiplying the original length and width by the scaling coefficient to obtain the scaled size. The width of the original image is larger than the height, the height is filled with black edges, and because the model is sampled by 32 times, the size of the width of the picture is filled to be 32 times, and the difference with the scaled size is minimum. And after calculating the difference value, performing black edge supplement on the upper part and the lower part of the picture.
(2) And after the picture pixel points are divided by 255 for normalization, exchanging channels to obtain the characteristic value of the input model.
A bank counter personnel hand-lifting identification system comprises an image preprocessing module, a target detection module, a central calculation module, a comparison module and an alarm module;
the image preprocessing module is used for acquiring and acquiring working videos and pictures of staff in a bank counter in real time and preprocessing the acquired working videos and pictures;
the target detection module is used for reasoning and analyzing the preprocessed working video and picture and outputting the body position of the worker and the hand position of the worker when the worker lifts the hand in the working video and picture;
the central computing module is used for computing the coincidence degree of a position frame of each worker in the monitoring image and the hand-lifting action of each worker in the working video and the picture;
the comparison module is used for comparing the calculated contact ratio of the hand lifting action of each worker with a set contact ratio threshold value according to a preset contact ratio threshold value of the hand lifting action of the worker, and finally outputting a comparison result.
The target detection model of the invention adopts a model structure of YOLOV4 to output the position frames of people and seals, and the specific structure is shown in FIG. 3. YOLOV4 is to extract feature maps by using CSPDarknet53 as backbone network, then to perform multi-scale sampling by SPP, to speed up the communication between feature information by PANet, and to obtain the required output by using the head structure of YOLOV 3.
In the embodiment of the invention, the hand-lifting identification process is as shown in fig. 2, when the worker identifies the hand-lifting, the input video and the picture are preprocessed, the preprocessed image is input into the target detection model for reasoning, and the positions (if the positions) of the person and the hand-lifting are output. And setting a threshold value of the contact ratio, calculating the contact ratio of each lifted hand in the position frame and the image of each person, comparing the contact ratios with the threshold value, and if the obtained calculated value exceeds the threshold value, considering that the person lifts the hand, otherwise, not considering that the currently judged person lifts the hand.
The hand-lifting recognition model image used by the invention is shown in fig. 3, the model input is a preprocessed image, the preprocessed image is firstly transmitted into a CNN-backbone network to extract a feature map, the feature map accelerates the circulation of feature information through a PANET structure, a similar YOLOV3-Head detection Head is used for generating three detection result feature maps with different scales, and a method for inhibiting inconsistency by learning spatial filtering conflict information through an adaptive ASFF module improves the scale invariance of features. And finally, transforming and splicing the predictions by a method to obtain the content which needs to be output.
The foregoing shows and describes the general principles, principal features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (4)
1. A bank counter personnel hand-lifting identification method is characterized by comprising the following steps:
the method comprises the following steps: acquiring image data, respectively acquiring a working video and a picture of a worker at a bank counter by using an image acquisition device, and respectively preprocessing the acquired working video and the picture;
step two, behavior recognition and inference, namely inputting the preprocessed working video and pictures into a target detection model for behavior analysis recognition and inference, and outputting the positions of workers in the working video and pictures and the positions of hands of the workers during hand lifting;
step three: behavior comparison and judgment, namely setting a hand lifting contact degree threshold value, calculating the position frame of each worker in the working video and the picture and the hand lifting contact degree of the hands of the workers, and comparing the calculated hand lifting contact degree with the set hand lifting contact degree threshold value; and if the calculated hand lifting contact ratio is greater than the set hand lifting contact ratio threshold value, the worker is considered to have a hand lifting behavior, otherwise, the worker is determined not to have the hand lifting behavior.
2. The bank counter personnel hand-lifting identification method according to claim 1, wherein the second step specifically comprises the following substeps:
s201, acquiring and preprocessing working videos and pictures of bank workers through monitoring equipment, and inputting the preprocessed working videos and pictures into a CNN-backbone network extraction feature map;
s202, a CNN-backbone network extracts feature graphs to accelerate circulation of feature information through a PANET structure, and a similar YOLOV3-Head detection Head is used for generating detection result feature graphs of three different scales;
and S203, performing spatial filtering on the detection result characteristic diagram through the adaptive ASFF module, filtering out conflict information, and converting and splicing the detection result characteristic diagram to output the body position of the worker and the hand position of the worker during hand lifting in the working video and picture.
3. The method for identifying hands-on of bank counter personnel as claimed in claim 1, wherein the step one of preprocessing the collected work video and pictures specifically comprises:
s101, carrying out adaptive image zooming on an input image, and calculating respective zooming coefficients of length and width according to the length and width of an input model; then, selecting the smaller scaling coefficient as the whole scaling coefficient, and multiplying the original length and width by the scaling coefficient to obtain the scaled size; then black edge filling is carried out on the height, the width size of the picture is filled to be a multiple of 32, and the size difference value of the picture and the size after scaling is minimum; after the difference value is calculated, black edge supplement is carried out on the upper part and the lower part of the picture;
s102, after the picture pixel points are divided by 255 for normalization, the channels are exchanged to obtain the characteristic value of the input model.
4. A bank counter personnel hand-lifting identification system is characterized by comprising an image preprocessing module, a target detection module, a central calculation module, a comparison module and an alarm module;
the image preprocessing module is used for acquiring and acquiring working videos and pictures of staff in a bank counter in real time and preprocessing the acquired working videos and pictures;
the target detection module is used for reasoning and analyzing the preprocessed working video and picture and outputting the body position of the worker and the hand position of the worker when the worker lifts the hand in the working video and picture;
the central calculation module is used for calculating the coincidence degree of a position frame of each worker and the hand-lifting action of each worker in the monitoring image in the working video and the picture;
the comparison module is used for comparing the calculated contact ratio of the hand lifting action of each worker with a set contact ratio threshold value according to a preset contact ratio threshold value of the hand lifting action of the worker, and finally outputting a comparison result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105141.XA CN112818802B (en) | 2021-01-26 | 2021-01-26 | Bank counter personnel hand-lifting identification method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110105141.XA CN112818802B (en) | 2021-01-26 | 2021-01-26 | Bank counter personnel hand-lifting identification method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112818802A CN112818802A (en) | 2021-05-18 |
| CN112818802B true CN112818802B (en) | 2022-07-05 |
Family
ID=75859431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110105141.XA Active CN112818802B (en) | 2021-01-26 | 2021-01-26 | Bank counter personnel hand-lifting identification method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112818802B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113449717B (en) * | 2021-05-27 | 2023-07-11 | 众安在线财产保险股份有限公司 | Invoice picture processing method, insurance claim settlement method, apparatus, equipment and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013013487A1 (en) * | 2011-07-26 | 2013-01-31 | 华南理工大学 | Device and method for monitoring driving behaviors of driver based on video detection |
| CN103297751A (en) * | 2013-04-23 | 2013-09-11 | 四川天翼网络服务有限公司 | Wisdom skynet video behavior analyzing system |
| CN110399822A (en) * | 2019-07-17 | 2019-11-01 | 思百达物联网科技(北京)有限公司 | Action identification method of raising one's hand, device and storage medium based on deep learning |
| WO2020192498A1 (en) * | 2019-03-28 | 2020-10-01 | 虹软科技股份有限公司 | Method and apparatus for detecting state of holding steering wheel by hands |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10380429B2 (en) * | 2016-07-11 | 2019-08-13 | Google Llc | Methods and systems for person detection in a video feed |
-
2021
- 2021-01-26 CN CN202110105141.XA patent/CN112818802B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013013487A1 (en) * | 2011-07-26 | 2013-01-31 | 华南理工大学 | Device and method for monitoring driving behaviors of driver based on video detection |
| CN103297751A (en) * | 2013-04-23 | 2013-09-11 | 四川天翼网络服务有限公司 | Wisdom skynet video behavior analyzing system |
| WO2020192498A1 (en) * | 2019-03-28 | 2020-10-01 | 虹软科技股份有限公司 | Method and apparatus for detecting state of holding steering wheel by hands |
| CN110399822A (en) * | 2019-07-17 | 2019-11-01 | 思百达物联网科技(北京)有限公司 | Action identification method of raising one's hand, device and storage medium based on deep learning |
Non-Patent Citations (3)
| Title |
|---|
| Real Time Hand Gesture Recognition Including Hand Segmentation and Tracking;Thomas Coogan 等;《Advances in Visual Computing. ISVC 2006. Lecture Notes in Computer Science》;20061108;第4291卷;第495–504页 * |
| 基于形态嵌入特征的室内日常行为识别方法研究;袁欢;《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》;20191215;C038-254 * |
| 面向教室场景的举手动作识别算法的研究与实现;廖望;《中国优秀硕士学位论文全文数据库 信息科技辑》;20200315;I138-1235 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112818802A (en) | 2021-05-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107808139B (en) | Real-time monitoring threat analysis method and system based on deep learning | |
| CN110472870A (en) | A kind of cashier service regulation detection system based on artificial intelligence | |
| CN114049568B (en) | Target object deformation detection method, device, equipment and medium based on image comparison | |
| CN113033706B (en) | Multi-source two-stage dish identification method based on visual detection and re-identification | |
| Cerman et al. | A mobile recognition system for analog energy meter scanning | |
| CN105117740A (en) | Font identification method and device | |
| CN110909578A (en) | Low-resolution image recognition method and device and storage medium | |
| CN112818802B (en) | Bank counter personnel hand-lifting identification method and system | |
| EP3376438A1 (en) | A system and method for detecting change using ontology based saliency | |
| CN110414431B (en) | Face recognition method and system based on elastic context relation loss function | |
| CN114863189A (en) | Intelligent image identification method based on big data | |
| CN104966109A (en) | Medical laboratory report image classification method and apparatus | |
| CN114612897A (en) | Intelligent fruit and vegetable weighing and ticketing method and device, electronic equipment and storage medium | |
| CN112419257A (en) | Method and device for detecting definition of text recorded video, computer equipment and storage medium | |
| CN112861841B (en) | Training method and device for bill confidence value model, electronic equipment and storage medium | |
| CN113723093B (en) | Personnel management policy recommendation method and device, computer equipment and storage medium | |
| CN102004537B (en) | System power-on and power-off control device and method | |
| CN114612971A (en) | Face detection method, model training method, electronic device, and program product | |
| CN113627426A (en) | Method, server, equipment and storage medium for detecting specific materials of store | |
| CN114359997A (en) | Service guiding method and system | |
| CN113792569A (en) | Object identification method and device, electronic equipment and readable medium | |
| CN111079660A (en) | Cinema online people counting method based on thermal infrared imaging picture | |
| CN211262448U (en) | Intelligent scale | |
| CN114494754B (en) | Workshop sundry detection method, device, terminal and storage medium | |
| Louis et al. | An Image-based Intelligent System for Data Extraction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Third floor, No.10, Jiuxing Avenue, high tech Zone, Chengdu, Sichuan 610041 Patentee after: Sichuan Tianyi Network Co.,Ltd. Address before: Third floor, No.10, Jiuxing Avenue, high tech Zone, Chengdu, Sichuan 610041 Patentee before: SICHUAN TIANYI NETWORK SERVICE Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A method and system for identifying bank counter personnel by raising their hands Effective date of registration: 20230804 Granted publication date: 20220705 Pledgee: Zheshang Bank Co.,Ltd. Chengdu Branch Pledgor: Sichuan Tianyi Network Co.,Ltd. Registration number: Y2023510000192 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |