CN115171222B - Behavior detection method and device, computer equipment and storage medium - Google Patents
Behavior detection method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN115171222B CN115171222B CN202211081587.4A CN202211081587A CN115171222B CN 115171222 B CN115171222 B CN 115171222B CN 202211081587 A CN202211081587 A CN 202211081587A CN 115171222 B CN115171222 B CN 115171222B
- Authority
- CN
- China
- Prior art keywords
- video
- business
- personnel
- service
- handling device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Abstract
The embodiment of the application discloses a behavior detection method, a behavior detection device, computer equipment and a storage medium, wherein in the scheme, if a client operates a business handling device in a current business scene, a video with a shooting range including the business handling device is obtained; carrying out hand recognition according to the video, and determining the target hand position of the business personnel of the business scene in the video; identifying the device position of the business handling device in the video according to the video; calculating the relative positions of the target hand position and the device position in the same video frame; and determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position, thereby improving the detection efficiency of the violation behaviors of the business personnel.
Description
Technical Field
The present application relates to the field of detection technologies, and in particular, to a behavior detection method and apparatus, a computer device, and a storage medium.
Background
With the development of society, banks provide more and more banking businesses for people to handle, for example, sales-type banking businesses such as selling financial products, selling insurance products by bank agent insurance companies, and the like.
In the process of banking business handling, it is required to make sure that the banking business to be handled is handled voluntarily and initiatively by a client, and business personnel are prohibited from handling the banking business rapidly and initiatively assisting the client in handling the banking business, so that certain key links in the process of banking business handling need to be selected by the client autonomously, such as product selection, money selection, signature confirmation handling and the like.
Aiming at the phenomenon, in the prior art, a worker needs to monitor the whole banking business handling process in real time so as to manually judge whether the violation behavior of the worker exists, but a large amount of labor cost is undoubtedly consumed, and the detection efficiency of the violation behavior of the worker is low.
Disclosure of Invention
The embodiment of the application provides a behavior detection method, a behavior detection device, computer equipment and a storage medium which can be used for financial science and technology or other related fields, and the detection efficiency of illegal behaviors of business personnel can be improved.
The embodiment of the application provides a behavior detection method, which comprises the following steps:
if the current business scene is in a stage of operating a business handling device by a client, acquiring a video of which the shooting range comprises the business handling device;
performing hand recognition according to the video, and determining the target hand position of the business personnel of the business scene in the video;
identifying the device position of the business handling device in the video according to the video;
calculating the relative positions of the target hand position and the device position in the same video frame;
and determining whether the business personnel have preset illegal behaviors at the stage of operating the business handling device by the client according to the relative position.
Correspondingly, the embodiment of the present application further provides a behavior detection apparatus, including:
the video acquisition module is used for acquiring a video of which the shooting range comprises the business handling device if the client operates the business handling device in the current business scene;
the position determining module is used for carrying out hand recognition according to the video and determining the target hand position of the business personnel of the business scene in the video;
the position identification module is used for identifying the position of the business handling device in the video according to the video;
the calculation module is used for calculating the relative positions of the target hand position and the device position in the same video frame;
and the behavior determining module is used for determining whether the business personnel has preset illegal behaviors at the stage of operating the business handling device by the client according to the relative position.
In some embodiments, the behavior detection device further includes:
the face recognition module is used for carrying out face recognition on the personnel in the video and determining the personnel relative position between the service personnel and the client;
accordingly, the position determination module is specifically configured to:
carrying out hand recognition according to the video, and determining the relative position of the hands between the business personnel and the client in the video;
determining the continuous frame number with the phenomenon that the relative position of the hand does not match with the relative position of the personnel according to the relative position of the hand between the service personnel and the client in the video;
when the continuous frame number is larger than a preset frame number threshold value, carrying out human body posture estimation on the service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel of the service scene in the video based on the human body posture estimation result of the service personnel.
In some embodiments, the behavior detection device further includes:
the personnel identification module is used for detecting personnel in the video and determining the quantity of the personnel in the service scene;
accordingly, the position determination module is specifically configured to:
performing hand recognition according to the video, and determining the number of hands in the current video frame of the video;
when the number of the hands is not matched with the number of the personnel, carrying out human body posture estimation on the service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel of the service scene in the video based on the human body posture estimation result of the service personnel.
In some embodiments, the behavior determination module is specifically configured to:
when the distance between the relative positions is smaller than a first preset distance, determining that a preset violation behavior exists in the stage of operating the business handling device by the client by the business personnel;
when the distance between the relative positions is larger than a first preset distance and smaller than a second preset distance, acquiring the hand posture corresponding to the service personnel;
and if the hand posture accords with a preset posture, determining that the business personnel has a preset violation behavior in the stage of operating the business handling device by the client.
In some embodiments, the behavior detection device further includes:
the information identification module is used for identifying the sound information in the video to obtain the sound information of the client;
a wish identification module, configured to perform wish identification on the client voice information to obtain a wish category of the client when the client operates the service handling apparatus;
the behavior determining module is further configured to determine that no preset violation behavior exists in the stage that the business personnel operates the business handling device at the client when the intention category belongs to the help-seeking category.
In some embodiments, the behavior detection device further includes:
the behavior determining module is further configured to determine whether the business staff has a preset auxiliary behavior at a stage of the client operating the business handling device according to the relative position when the intention category belongs to a help-seeking category;
and the instruction sending module is used for sending a blocking instruction to the control center when the auxiliary behavior exists so that the control center does not respond to the service confirmation submission information sent by the service handling device according to the blocking instruction.
In some embodiments, the behavior detection device further includes:
and the warning module is used for warning and reminding the service personnel through a preset warning device if the service personnel have preset illegal behaviors in the stage of operating the service handling device by the client.
Accordingly, embodiments of the present application further provide a computer device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the behavior detection method provided in any of the embodiments of the present application.
Correspondingly, the embodiment of the application also provides a storage medium, wherein the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by the processor to execute the behavior detection method.
If the video of the service handling device is in the stage of the client operating the service handling device in the current service scene, the video of the service handling device is obtained in the shooting range, then the hand recognition is carried out according to the video, the target hand position of the service staff of the service scene in the video is determined, the device position of the service handling device in the video is recognized according to the video, then the relative position of the target hand position and the device position in the same video frame is calculated, and whether the preset violation behavior exists in the stage of the client operating the service handling device or not is determined according to the relative position, so that the detection efficiency of the violation behavior of the service staff is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flow chart of a behavior detection method according to an embodiment of the present application.
Fig. 2 is a block diagram of a behavior detection apparatus according to an embodiment of the present disclosure.
Fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides a behavior detection method, a behavior detection device, a storage medium and computer equipment. Specifically, the behavior detection method in the embodiment of the present application may be executed by a computer device, where the computer device may be a server or a terminal. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as cloud service, a cloud database, cloud computing, a cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, big data and artificial intelligence platform. The terminal may be, but is not limited to, a smart phone, a desktop computer, a notebook computer, a tablet computer, etc. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.
For example, the computer device may be a terminal, and the terminal may acquire a video whose shooting range includes a service handling apparatus if the terminal is in a stage where a client operates the service handling apparatus in a current service scene; identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video; identifying the position of the service handling device in the video according to the video; calculating the relative positions of the target hand position and the device position in the same video frame; and determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position.
Based on the above problems, embodiments of the present application provide a behavior detection method, an apparatus, a computer device, and a storage medium, which can improve detection efficiency of an illegal behavior of a business worker.
The following are detailed below. It should be noted that the following description of the embodiments is not intended to limit the preferred order of the embodiments.
The embodiment of the present application provides a behavior detection method, which may be executed by a terminal or a server, and the embodiment of the present application is described by using an example in which the behavior detection method is executed by the terminal.
Referring to fig. 1, fig. 1 is a schematic flow chart of a behavior detection method according to an embodiment of the present disclosure. The specific flow of the behavior detection method can be as follows:
101. and if the current service scene is in a stage of operating the service handling device by the client, acquiring the video of which the shooting range comprises the service handling device.
The stage of the client operating the business handling device is the stage of a key link which needs to be selected by the user independently in the process of banking business handling, such as product selection, money selection, signature confirmation handling and the like.
It can be understood that, in the process of banking transaction, each stage needs to click the confirmation button on the transaction device to send the service confirmation submission information corresponding to the stage to the control center, or to enter the next adjacent stage. Therefore, the terminal receives the stage information of the current stage sent by the service handling device, and judges the stage information of the current stage, namely judges whether the stage information of the current stage accords with the preset stage information of the client operation service handling device, if so, the terminal confirms that the current service scene is in the stage of the client operation service handling device.
In this embodiment, when the terminal detects that the banking business handling stage of the current business scene is the stage in which the client operates the business handling apparatus, the terminal captures a video of the business scene to detect the behavior of the business staff.
The shooting range of the video comprises the business handling device and at least one of business personnel and clients, for example, if the hands of the business personnel do not exist in the range of the business handling device, the business personnel do not have violation behaviors.
102. And identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video.
In this embodiment, the terminal continuously acquires the video of the service scene in real time to perform hand recognition on the video, so as to determine a target hand position of a service person of the service scene in the video, where the target hand position is a position where a hand of the service person is located, where the hand includes, but is not limited to, a hand of a client and a hand of the service person.
In some embodiments, the terminal may use a hand pose estimation algorithm in MediaPipe to obtain the hand position of each person in the video, and further use bytrack algorithm to track the hand to obtain the hand position of each person in the video in each video frame, for example, the hand id of the service person in the first frame of the video is A1, A2, the above A1 corresponds to (x 1, y1, z 1) in the first frame of the video, the above A2 corresponds to (x 2, y2, z 2) in the first frame of the video, the hand id of the service person in the second frame of the video is A1, A2, the above A1 corresponds to (x 3, y3, z 3) in the second frame of the video, and the above A2 corresponds to (x 4, y4, z 4) in the second frame of the video.
In some embodiments, sometimes, due to long-time tracking or other factors, when the human hand is recognized, the human hand recognition result is inaccurate, so that the behavior judgment of the service personnel is wrong, therefore, a priori condition can be preset, when the human hand is recognized, the human hand recognition result is verified based on the preset priori condition, if the human hand recognition result is not matched with the priori condition, the current human hand recognition result corresponding to the current frame of the video is inaccurate, and then the human hand in the current frame of the video needs to be re-recognized in a preset recognition mode, so that an accurate target human hand position is obtained.
In some embodiments, the re-recognizing the human hand in the current frame of the video in the preset recognition manner to obtain an accurate target human hand position may include: the terminal carries out human body posture estimation on service personnel in a current video frame of the video to obtain a human body posture estimation result of the service personnel; the target hand position of the service personnel in the service scene in the video is determined based on the human body posture estimation result of the service personnel, and the tracking precision when the hand is shielded is improved.
In this embodiment, the terminal may obtain the wrist key point position of the service staff from the human body posture estimation result of the service staff; and determining the hand closest to the position of the wrist key point as a target hand corresponding to the service personnel according to the hand positions of all hands appearing in the current video frame, and then determining the target hand position of the target hand corresponding to the service personnel.
The positions of the hands in the current video frame can be corrected based on the positions of the hands in the last video frame adjacent to the current video frame to obtain the corrected positions of the hands, and the corrected positions of the hands are determined as the positions of the hands which are finally compared with the positions of the key points of the wrist in the current video frame.
It can be understood that, because the motion trajectory of the human hand is relatively fast and random, the id of the human hand appearing in the current video frame can be determined by calculating IoU of the current human hand between the detection frame of the current video frame and the detection frame of the human hand of the previous video frame, and when the current video frame and the previous video frame are determined to be the same human hand, the position of the same human hand in the current video frame is corrected based on the position of the human hand in the previous video frame. For example, the human hands id of the service personnel in the first frame of the video are A1 and A2, the human hand id appearing in the current video frame is determined by calculating IoU of the current human hand between the detection frame of the second frame of the video and the detection frames corresponding to all the human hands A1 and A2 in the first frame of the video according to the size of the human hand id, for example, the current human hand corresponding to the id is A1.
Specifically, the correcting the position of each human hand in the current video frame based on the position of the human hand in the previous video frame adjacent to the current video frame may include: and predicting by using a recursive predictive filtering (Kalman Filter) algorithm based on the hand position of the hand in the last video frame adjacent to the current video frame and the position of each hand in the current video frame to obtain a Kalman Filter predicted position.
In some embodiments, the terminal may perform person detection on a person in a current video frame of the video by using a Yolox target detection model to obtain a detection box of each person, and then perform human body posture estimation on the person in the current video frame by using a human body posture estimation algorithm in MediaPipe.
In some embodiments, the prior condition may be set based on the relative position of the person in the video, or may be set based on the number of the persons in the video, and may be specifically set according to a requirement, which is not limited herein.
Optionally, a priori condition may be set as to whether the relative position of the person matches the relative position of the human hand, and to avoid that the relative position of the person and the relative position of the human hand are not matched currently due to some environmental factors, for example, the human hand of the service person goes over the client to take some data, which may result in the relative position of the person and the relative position of the human hand being not matched.
Specifically, after acquiring the video whose shooting range includes the business transaction device, the method may further include: carrying out face recognition on personnel in the video, and determining the relative position of the personnel between the service personnel and the client; the terminal can only perform face recognition on the first frames of the video, and then determine the relative position of the business personnel and the client based on the face recognition results of the first frames, so that the speed of determining the target hand position of the business personnel in the business scene is increased.
Correspondingly, the above identifying the hands according to the video to determine the target hand position of the service personnel in the service scene in the video may include: the terminal identifies hands according to the video, determines the relative positions of the hands between the business personnel and the clients in the video, determines the continuous frames with the unmatched relative positions of the hands and the personnel according to the relative positions of the hands between the business personnel and the clients in the video, judges the continuous frames according to a preset frame number threshold, and estimates the human body posture of the business personnel in the current video frame of the video to obtain the human body posture estimation result of the business personnel when the continuous frames are larger than the preset frame number threshold and indicate that the current human hand identification result is inaccurate. And then determining the target hand position of the service personnel in the service scene in the video based on the human body posture estimation result of the service personnel.
Optionally, the prior condition may be set as whether the number of people matches the number of human hands, that is, if the number of people in the video does not match the number of human hands, the current human hand recognition result is considered to be inaccurate.
Specifically, after acquiring the video whose shooting range includes the business handling apparatus, the method may further include: and detecting the personnel in the video to determine the number of the personnel in the service scene, for example, if only one service personnel and one client are in the service scene, the number of the personnel is 2, and the number of the hands which can appear in the video is less than or equal to 4.
Correspondingly, the above identifying the hands according to the video to determine the target hand position of the service personnel in the service scene in the video may include: the terminal identifies hands according to the video, determines the number of the hands in the current video frame of the video, judges the number of the hands according to the number of the personnel, and estimates the human body posture of the service personnel in the current video frame of the video when the number of the hands is not matched with the number of the personnel, for example, the video only contains 2 persons, but the current hand identification result is 5 hands, which indicates that the current hand identification result is inaccurate, so that the human body posture estimation result of the service personnel is obtained. And then determining the target hand position of the service personnel in the service scene in the video based on the human body posture estimation result of the service personnel.
103. And identifying the position of the business handling device in the video according to the video.
In this embodiment, the terminal needs to track and identify the service handling device in the video all the time to obtain the device position of the service handling device of each frame, so as to determine the violation of the service staff based on the device position. The service handling device includes, but is not limited to, a mobile device, a machine related to bank handling, and the like, and the mobile device may be an electronic product such as a mobile phone, an iPad, a computer, and the like.
In some embodiments, the tracking and identifying the transaction apparatuses in the video may include: firstly, initializing the key point position and key point sequence of the service handling device, and effectively eliminating the interference of other objects, wherein the key point position can be determined according to the precision requirement and the type of the service handling device, for example, iPad can take four corners of the iPad as key points; then, the key point position of the business handling device is tracked in real time.
The terminal can detect the initial position of a key point of the service handling device by using an AruCo Marker-based detection algorithm; the terminal can roughly track the business handling device by using a Lucas-Kanade algorithm, find edge information of the business handling device, such as four edges of an iPad, according to the rough area of the business handling device, and then find key point positions of the business handling device by using local edge information of the business handling device and shape characteristics of the business handling device after photographic transformation, for example, find accurate iPad four-corner coordinates by using local iPad side line information and rectangular characteristics of the iPad after photographic transformation.
Specifically, the tracking the position of the key point of the transaction apparatus in real time may include: according to the key point position and the key point sequence and the shape characteristics of the business handling device, the rough position of the business handling device is obtained by performing optical flow estimation through a Lucas-Kanade algorithm, the local area of the business handling device is divided by utilizing the rough position of the business handling device, all edges in the local area are detected, then the edge line of the business handling device is detected by utilizing a Hough transform algorithm, points on the edge line are combined, mapping is performed through projective transformation to obtain a point set of the business handling device, then the target shape of the business handling device meeting the shape characteristics of the business handling device is obtained by utilizing a minimum error method according to the point set of the business handling device, and the position of the business handling device is obtained based on the target shape of the business handling device.
104. The relative positions of the target hand position and the device position in the same video frame are calculated.
The relative position includes, but is not limited to, a distance between the target hand position and the device position, or a positional relationship between the target hand position and the device position.
105. And determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position.
The illegal action client operates the substitute action of the service handling device.
In this embodiment, the terminal may determine whether a predetermined violation occurs when the service staff operates the service handling device by the client according to the distance between the relative positions, or may determine whether the predetermined violation occurs when the service staff operates the service handling device by the client according to the position relationship between the relative positions, so that the efficiency of detecting the violation occurs to the service staff is improved by position determination.
In some embodiments, the determining whether the business person has a preset violation at the stage of the client operating the business handling apparatus according to the relative position may include:
when the distance between the relative positions is smaller than a first preset distance, the fact that the distance between the current target hand position and the device position is too small is indicated, and it can be determined that a business person has preset illegal behaviors in the stage of operating the business handling device by a client.
Further, when there is a certain distance between the target hand position and the device position, it cannot be said that the business staff has no violation, for example, the business staff may adopt a pointing manner to prompt the customer to select a certain commodity, so the hand posture can be recognized, and it is determined whether the business staff has a preset violation at the stage of the customer operating the business handling device according to the recognition result of the hand posture.
Specifically, the determining whether the business person has a preset violation in the stage of operating the business handling apparatus by the client according to the relative position may include: when the distance between the relative positions is greater than a first preset distance and smaller than a second preset distance, acquiring the hand posture corresponding to the service personnel; and if the gesture of the hand of the service staff accords with the preset gesture, which indicates that the service staff has the violation behavior of directing the client to transact, determining that the service staff has the preset violation behavior at the stage of operating the service transaction device by the client.
In the embodiment of the invention, the first and the second are only used for distinguishing the preset distance, and have no other special meanings.
In some embodiments, if a business person has a preset violation behavior in a stage of the client operating the business handling device, a preset warning device is used to warn the business person.
The warning device can be integrated with the shooting device or integrated with the business handling device, or respectively arranged with the shooting device and the business handling device, and needs to be arranged in a range where the sight of business personnel can reach, so that the business personnel can be timely notified when warning reminding is carried out.
The above-mentioned warning reminding mode includes but is not limited to text reminding, voice reminding, etc.
In some embodiments, after acquiring the video whose shooting range includes the service handling apparatus, the method may further include: identifying the sound information in the video to obtain the sound information of the client; the voice information of the client is subjected to intention identification to obtain the intention type of the client when the client operates the service handling device, so that the current requirements of the client can be judged in time according to the intention type; when the intention type belongs to the help seeking type, the fact that the client possibly needs to assist the business personnel for auxiliary operation is described, and at the moment, it is determined that the business personnel do not have preset violation behaviors in the stage of the client operating the business handling device, namely, even if the business personnel perform corresponding operation on the business handling device, the business personnel are not considered to have the preset violation behaviors in the stage of the client operating the business handling device.
In some embodiments, when a service person is in the face of a help-seeking requirement of a client, although the service person can help the client to perform demonstration, since a current service scene is in a stage of the client operating a service handling device, the service person is prohibited from clicking a confirmation button on the service handling device, and therefore when a wish category belongs to a help-seeking category, whether the service person has a preset auxiliary behavior in the stage of the client operating the service handling device is determined according to a relative position; and when the auxiliary behavior exists, sending a blocking instruction to the control center so that the control center does not respond to the service confirmation submission information sent by the service handling device according to the blocking instruction until the terminal detects that the service personnel completes the auxiliary behavior, namely determining that the service personnel does not have the preset auxiliary behavior at the stage of operating the service handling device by the client according to the relative position, and sending an acceptance instruction to the control center so that the control center responds to the service confirmation submission information sent by the service handling device according to the acceptance instruction.
The judging mechanism for determining whether the business personnel has the preset auxiliary behavior at the stage of the business handling device operated by the client can refer to the judging mechanism for the illegal behavior.
The embodiment of the application discloses a behavior detection method, which comprises the following steps: if the current business scene is in a stage of operating the business handling device by a client, acquiring a video of which the shooting range comprises the business handling device; identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video; identifying the position of the device in the video according to the video; calculating the relative positions of the target hand position and the device position in the same video frame; and determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position, so that the detection efficiency of the violation behaviors of the business personnel can be improved.
In order to better implement the behavior detection method provided by the embodiment of the present application, the embodiment of the present application further provides a behavior detection device based on the behavior detection method. The terms are the same as those in the above behavior detection method, and details of implementation may refer to the description in the method embodiment.
Referring to fig. 2, fig. 2 is a block diagram of a behavior detection apparatus according to an embodiment of the present disclosure, where the apparatus includes:
the video acquisition module 201 is configured to acquire a video of which a shooting range includes a service handling device if the video is in a stage of a client operating the service handling device in a current service scene;
the position determining module 202 is used for performing hand recognition according to the video and determining the target hand position of a business person of a business scene in the video;
the position identification module 203 is used for handling the position of the device in the video according to the video identification service;
a calculating module 204, configured to calculate relative positions of the target hand position and the device position in the same video frame;
and the behavior determining module 205 is configured to determine whether a preset violation behavior exists in the stage of operating the business handling apparatus by the business personnel according to the relative position.
In some embodiments, the behavior detection device further comprises:
the face recognition module is used for carrying out face recognition on personnel in the video and determining the relative position of the personnel between the service personnel and the client;
accordingly, the position determining module 202 is specifically configured to:
identifying hands according to the video, and determining the relative positions of the hands between the business personnel and the client in the video;
determining the continuous frame number of the phenomenon that the relative positions of the hands and the personnel are not matched according to the relative positions of the hands and the personnel between the business personnel and the client in the video;
when the continuous frame number is larger than a preset frame number threshold, performing human body posture estimation on service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel in the service scene in the video based on the human body posture estimation result of the service personnel.
In some embodiments, the behavior detection device further comprises:
the personnel identification module is used for detecting personnel in the video and determining the quantity of the personnel in the service scene;
accordingly, the position determining module 202 is specifically configured to:
identifying hands according to the video, and determining the number of the hands in the current video frame of the video;
when the number of hands is not matched with the number of people, carrying out human body posture estimation on service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel in the service scene in the video based on the human body posture estimation result of the service personnel.
In some embodiments, the behavior determination module 205 is specifically configured to:
when the distance between the relative positions is smaller than a first preset distance, determining that a business person has a preset violation behavior at the stage of operating the business handling device by the client;
when the distance between the relative positions is greater than a first preset distance and smaller than a second preset distance, acquiring the hand posture corresponding to the service personnel;
and if the posture of the hand of the user accords with the preset posture, determining that the business personnel has preset violation behaviors in the stage of operating the business handling device by the client.
In some embodiments, the behavior detection device further comprises:
the information identification module is used for identifying the sound information in the video to obtain the sound information of the client;
the system comprises a willingness identification module, a willingness identification module and a service processing module, wherein the willingness identification module is used for carrying out willingness identification on voice information of a client to obtain the willingness type of the client when the client is in a stage of operating the service processing device by the client;
the action determining module 205 is further configured to determine that no preset violation action exists in the stage of the business handling apparatus operated by the client by the business personnel when the will category belongs to the help-seeking category.
In some embodiments, the behavior detection device further comprises:
the behavior determining module 205 is further configured to determine whether a service staff has a preset auxiliary behavior at a stage of the client operating the service handling apparatus according to the relative position when the intention category belongs to the help-seeking category;
and the instruction sending module is used for sending a blocking instruction to the control center when the auxiliary behavior exists so that the control center does not respond to the service confirmation submission information sent by the service handling device according to the blocking instruction.
In some embodiments, the behavior detection device further comprises:
and the warning module is used for warning and reminding the service personnel through a preset warning device if the service personnel have preset illegal behaviors in the stage of operating the service handling device by the client.
The embodiment of the application discloses a behavior detection device, which is used for acquiring a video of a shooting range including a business handling device if a client operates the business handling device in a current business scene through a video acquisition module 201; the position determining module 202 is used for performing hand recognition according to the video and determining the target hand position of a service person in a service scene in the video; the position identification module 203 is used for handling the position of the device in the video according to the video identification service; a calculating module 204, configured to calculate relative positions of a hand position and a device position of a target in the same video frame; and the action determining module 205 is used for determining whether a preset violation action exists in the stage of operating the business handling device by the client according to the relative position. Therefore, the detection efficiency of the violation behaviors of the service personnel is improved.
Correspondingly, the embodiment of the application also provides a computer device, and the computer device can be a terminal. As shown in fig. 3, fig. 3 is a schematic structural diagram of a computer device according to an embodiment of the present application. The computer apparatus 300 includes a processor 301 having one or more processing cores, a memory 302 having one or more computer-readable storage media, and a computer program stored on the memory 302 and executable on the processor. The processor 301 is electrically connected to the memory 302. Those skilled in the art will appreciate that the computer device configurations illustrated in the figures are not meant to be limiting of computer devices and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components.
The processor 301 is a control center of the computer apparatus 300, connects various parts of the entire computer apparatus 300 by various interfaces and lines, performs various functions of the computer apparatus 300 and processes data by running or loading software programs and/or modules stored in the memory 302, and calling data stored in the memory 302, thereby monitoring the computer apparatus 300 as a whole.
In the embodiment of the present application, the processor 301 in the computer device 300 loads instructions corresponding to processes of one or more application programs into the memory 302, and the processor 301 executes the application programs stored in the memory 302 according to the following steps, so as to implement various functions:
if the current service scene is in a stage of operating the service handling device by a client, acquiring a video of which the shooting range comprises the service handling device;
identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video;
identifying the position of the device in the video according to the video;
calculating the relative positions of the target hand position and the device position in the same video frame;
and determining whether the business personnel have preset illegal behaviors at the stage of operating the business handling device by the client according to the relative position.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Optionally, as shown in fig. 3, the computer device 300 further includes: a touch display 303, a radio frequency circuit 304, an audio circuit 305, an input unit 306, and a power source 307. The processor 301 is electrically connected to the touch display 303, the radio frequency circuit 304, the audio circuit 305, the input unit 306, and the power source 307. Those skilled in the art will appreciate that the computer device configuration illustrated in FIG. 3 does not constitute a limitation of computer devices, and may include more or fewer components than those illustrated, or some components may be combined, or a different arrangement of components.
The touch display screen 303 may be used for displaying a graphical user interface and receiving operation instructions generated by a user acting on the graphical user interface. The touch display screen 303 may include a display panel and a touch panel. Among other things, the display panel may be used to display messages entered by or provided to a user as well as various graphical user interfaces of the computer device, which may be made up of graphics, text, icons, video, and any combination thereof. Alternatively, the display panel may be configured in the form of a Liquid crystal display (LCD, liquid crystal display client account l display client account y), an organic Light-Emitting Diode (OLED), or the like. The touch panel may be used to collect touch operations of a user on or near the touch panel (for example, operations of the user on or near the touch panel using any suitable object or accessory such as a finger, a stylus pen, and the like), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Alternatively, the touch panel may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives the touch message from the touch sensing device, converts the touch message into touch point coordinates, sends the touch point coordinates to the processor 301, and can receive and execute commands sent by the processor 301. The touch panel may overlay the display panel, and when the touch panel detects a touch operation thereon or nearby, the touch panel transmits the touch operation to the processor 301 to determine the type of the touch event, and then the processor 301 provides a corresponding visual output on the display panel according to the type of the touch event. In the embodiment of the present application, the touch panel and the display panel may be integrated into the touch display screen 303 to realize input and output functions. However, in some embodiments, the touch panel and the touch panel can be implemented as two separate components to perform the input and output functions. That is, the touch display screen 303 may also be used as a part of the input unit 306 to implement an input function.
The rf circuit 304 may be used for transceiving rf signals to establish wireless communication with a network device or other computer device via wireless communication, and for transceiving signals with the network device or other computer device.
The audio circuit 305 may be used to provide an audio interface between the user and the computer device through speakers, microphones. The audio circuit 305 may transmit the electrical signal converted from the received audio data to a speaker, and convert the electrical signal into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electric signal, which is received by the audio circuit 305 and converted into audio data, which is then processed by the audio data output processor 301, and then transmitted to, for example, another computer device via the radio frequency circuit 304, or output to the memory 302 for further processing. The audio circuit 305 may also include an earbud jack to provide communication of a peripheral headset with the computer device.
The input unit 306 may be used to receive input numbers, character messages, or user characteristic messages (e.g., fingerprints, irises, facial messages, etc.), and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control.
The power supply 307 is used to power the various components of the computer device 300. Optionally, the power supply 307 may be logically connected to the processor 301 through a power management system, so as to implement functions of managing charging, discharging, and power consumption management through the power management system. Power supply 307 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuitry, power converters or inverters, power status indicators, and the like.
Although not shown in fig. 3, the computer device 300 may further include a camera, a sensor, a wireless fidelity module, a bluetooth module, etc., which are not described in detail herein.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
As can be seen from the above, in the computer device provided in this embodiment, if the client is in the stage of operating the service handling apparatus in the current service scene, the video of which the shooting range includes the service handling apparatus is obtained; identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video; identifying the position of the device in the video according to the video; calculating the relative positions of the target hand position and the device position in the same video frame; and determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored, where the computer programs can be loaded by a processor to execute the steps in any one of the behavior detection methods provided in the present application. For example, the computer program may perform the steps of:
if the current service scene is in a stage of operating the service handling device by a client, acquiring a video of which the shooting range comprises the service handling device;
identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video;
identifying the position of the device in the video according to the video;
calculating the relative positions of the target hand position and the device position in the same video frame;
and determining whether the business personnel have preset violation behaviors at the stage of operating the business handling device by the client according to the relative position.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Wherein the storage medium may include: a read Only Memory (ROM, re client account d Only Memory), a random access Memory (R client account M, R client account and access Memory), a magnetic disk or an optical disk, and the like.
Since the computer program stored in the storage medium can execute the steps in any behavior detection method provided in the embodiments of the present application, beneficial effects that can be achieved by any behavior detection method provided in the embodiments of the present application can be achieved, and detailed descriptions are omitted here for the foregoing embodiments.
The behavior detection method, the behavior detection device, the storage medium, and the computer device provided in the embodiments of the present application are described in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (9)
1. A method of behavior detection, the method comprising:
receiving stage information of a stage in a banking business handling process, which is sent by a business handling device, confirming that the current business scene is in the stage of a client operating the business handling device if the stage information conforms to the preset stage information of the stage of the client operating the business handling device, and shooting the current business scene to obtain a video of which the shooting range comprises the business handling device;
carrying out face recognition on people in the first frames of the video, and determining the relative positions of the people between the business people and the clients based on the face recognition results of the people in the first frames;
identifying hands according to the video, and determining the target hand positions of the business personnel of the business scene in the video;
initializing the key point position and key point sequence of the service handling device, and determining the rough position of the service handling device in the video according to the key point position, the key point sequence and the shape characteristics of the service handling device;
based on the rough position, dividing a local area of the business handling device in the video, and determining a point set of the business handling device in the video according to the edge of the local area;
obtaining a target shape meeting the shape characteristics of the business handling device in the video by utilizing a minimum error method based on the point set, and determining the device position of the business handling device in the video according to the target shape;
calculating the relative positions of the target hand position and the device position in the same video frame;
determining whether the business personnel has preset violation behaviors at the stage of operating the business handling device by the client according to the relative position;
the identifying the hands according to the video and determining the target hand positions of the business personnel of the business scene in the video comprise:
identifying and tracking hands according to the video, and determining the relative position of the hands between the service personnel and the client in the video;
determining the continuous frame number of the phenomenon that the relative positions of the hands and the personnel are not matched according to the relative positions of the hands and the personnel between the business personnel and the client in the video;
when the continuous frame number is larger than a preset frame number threshold value, carrying out human body posture estimation on the service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel of the service scene in the video based on the human body posture estimation result of the service personnel.
2. The method of claim 1, after obtaining the video whose shooting range includes the business handling device, further comprising:
detecting the personnel in the video and determining the quantity of the personnel in the service scene;
correspondingly, the identifying the hands according to the video and determining the target hand position of the business personnel of the business scene in the video comprises the following steps:
identifying hands according to the video, and determining the number of the hands in the current video frame of the video;
when the number of the hands is not matched with the number of the personnel, carrying out human body posture estimation on the service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel;
and determining the target hand position of the service personnel of the service scene in the video based on the human body posture estimation result of the service personnel.
3. The method of claim 1, wherein said determining whether the business person has a predetermined violation at the stage of the customer operating a business transaction apparatus based on the relative position comprises:
when the distance between the relative positions is smaller than a first preset distance, determining that a preset violation behavior exists in the stage that the business personnel operates the business handling device by the client;
when the distance between the relative positions is larger than a first preset distance and smaller than a second preset distance, acquiring the hand posture corresponding to the service personnel;
and if the posture of the hand of the user accords with a preset posture, determining that the business personnel has a preset violation behavior in the stage of operating the business handling device by the client.
4. The method of claim 1, after obtaining the video whose shooting range includes the business transaction device, further comprising:
identifying the sound information in the video to obtain the sound information of the client;
performing intention identification on the client voice information to obtain the intention type of the client when the client operates the service handling device;
and when the intention type belongs to the help seeking type, determining that no preset violation behavior exists in the stage of operating the service handling device by the client.
5. The method of claim 4, further comprising:
when the intention type belongs to a help seeking type, determining whether the business personnel has a preset auxiliary behavior at the stage of operating the business handling device by the client according to the relative position;
and when the auxiliary behavior exists, sending a blocking instruction to a control center so that the control center does not respond to the service confirmation submission information sent by the service handling device according to the blocking instruction.
6. The method of any of claims 1 to 5, further comprising:
and if the business personnel have preset violation behaviors in the stage of operating the business handling device by the client, carrying out alarm reminding on the business personnel through a preset alarm device.
7. A behavior detection device, characterized in that the device comprises:
the video acquisition module is used for receiving the stage information of the stage in the banking business handling process sent by the business handling device, confirming the stage of the client operating the business handling device in the current business scene if the stage information accords with the preset stage information of the stage of the client operating the business handling device, shooting the current business scene, and acquiring the video of which the shooting range comprises the business handling device;
the face recognition module is used for carrying out face recognition on the personnel in the first frames of the video and determining the relative position of the personnel between the service personnel and the client based on the face recognition results of the personnel in the first frames;
the position determining module is used for identifying hands according to the video and determining the target hand position of the business personnel of the business scene in the video;
the position identification module is used for initializing the key point position and key point sequence of the service handling device and determining the rough position of the service handling device in the video according to the key point position, the key point sequence and the shape characteristics of the service handling device; based on the rough position, dividing a local area of the business handling device in the video, and determining a point set of the business handling device in the video according to the edge of the local area; obtaining a target shape meeting the shape characteristics of the business handling device in the video by utilizing a minimum error method based on the point set, and determining the device position of the business handling device in the video according to the target shape;
the calculation module is used for calculating the relative positions of the target hand position and the device position in the same video frame;
a behavior determining module, configured to determine whether a preset violation behavior exists in a stage where the client operates the service handling apparatus according to the relative position;
the position determination module is specifically configured to: identifying and tracking hands according to the video, and determining the relative position of the hands between the service personnel and the client in the video; determining the continuous frame number of the phenomenon that the relative positions of the hands and the personnel are not matched according to the relative positions of the hands and the personnel between the business personnel and the client in the video; when the continuous frame number is larger than a preset frame number threshold value, performing human body posture estimation on the service personnel in the current video frame of the video to obtain a human body posture estimation result of the service personnel; and determining the target hand position of the service personnel of the service scene in the video based on the human body posture estimation result of the service personnel.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and running on the processor, wherein the processor implements the behavior detection method according to any of claims 1 to 6 when executing the program.
9. A storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the behavior detection method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211081587.4A CN115171222B (en) | 2022-09-06 | 2022-09-06 | Behavior detection method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211081587.4A CN115171222B (en) | 2022-09-06 | 2022-09-06 | Behavior detection method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115171222A CN115171222A (en) | 2022-10-11 |
| CN115171222B true CN115171222B (en) | 2022-12-27 |
Family
ID=83480597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211081587.4A Active CN115171222B (en) | 2022-09-06 | 2022-09-06 | Behavior detection method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115171222B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861940B (en) * | 2023-02-24 | 2023-04-28 | 珠海金智维信息科技有限公司 | Work scene behavior evaluation method and system based on human body tracking and recognition technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871675A (en) * | 2019-02-26 | 2019-06-11 | 蒙志标 | A kind of bank counter operating system that can volume reception comprehensively and pay |
| CN111325128A (en) * | 2020-02-13 | 2020-06-23 | 上海眼控科技股份有限公司 | Illegal operation detection method and device, computer equipment and storage medium |
| CN111726586A (en) * | 2020-06-29 | 2020-09-29 | 上海药明生物技术有限公司 | Production system operation standard monitoring and reminding system |
| CN114943455A (en) * | 2022-05-30 | 2022-08-26 | 中国银行股份有限公司 | Method and device for preventing rule violation of foreground and background, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9183746B2 (en) * | 2013-03-12 | 2015-11-10 | Xerox Corporation | Single camera video-based speed enforcement system with a secondary auxiliary RGB traffic camera |
| CN112818868A (en) * | 2021-02-03 | 2021-05-18 | 招联消费金融有限公司 | Behavior sequence characteristic data-based violation user identification method and device |
| CN113536980A (en) * | 2021-06-28 | 2021-10-22 | 浙江大华技术股份有限公司 | Shooting behavior detection method and device, electronic device and storage medium |
-
2022
- 2022-09-06 CN CN202211081587.4A patent/CN115171222B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109871675A (en) * | 2019-02-26 | 2019-06-11 | 蒙志标 | A kind of bank counter operating system that can volume reception comprehensively and pay |
| CN111325128A (en) * | 2020-02-13 | 2020-06-23 | 上海眼控科技股份有限公司 | Illegal operation detection method and device, computer equipment and storage medium |
| CN111726586A (en) * | 2020-06-29 | 2020-09-29 | 上海药明生物技术有限公司 | Production system operation standard monitoring and reminding system |
| CN114943455A (en) * | 2022-05-30 | 2022-08-26 | 中国银行股份有限公司 | Method and device for preventing rule violation of foreground and background, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115171222A (en) | 2022-10-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112329740B (en) | Image processing method, image processing apparatus, storage medium, and electronic device | |
| CN115171222B (en) | Behavior detection method and device, computer equipment and storage medium | |
| CN107481447A (en) | A kind of processing method, system, equipment and storage medium forgotten after card taking | |
| WO2019184593A1 (en) | Method and apparatus for generating environment model, and storage medium | |
| CN112818733B (en) | Information processing method, device, storage medium and terminal | |
| CN111339920A (en) | Cash adding behavior detection method, device and system, storage medium and electronic terminal | |
| CN116307394A (en) | Product user experience scoring method, device, medium and equipment | |
| CN116542740A (en) | Live broadcasting room commodity recommendation method and device, electronic equipment and readable storage medium | |
| CN115618232A (en) | Data prediction method, device, storage medium and electronic equipment | |
| WO2022012595A1 (en) | Order generation method for software interface and system | |
| CN111143441A (en) | Gender determination method, device, equipment and storage medium | |
| CN111813639B (en) | Method and device for evaluating equipment operation level, storage medium and electronic equipment | |
| CN110322289A (en) | A kind of anti-cheat detection method, device, server, terminal and storage medium | |
| CN115631523A (en) | Data processing method and device for business place, computer equipment and storage medium | |
| CN115018496A (en) | Carbon account management method and device, computer equipment and storage medium | |
| WO2022158178A1 (en) | Image processing device, image processing method, and program | |
| CN115330514A (en) | Loan behavior risk management method and device, computer equipment and storage medium | |
| CN115063720A (en) | Image processing method and device, computer equipment and storage medium | |
| CN115514850A (en) | Value analysis method and device for outbound project, computer equipment and storage medium | |
| CN115186018A (en) | Interface blood relationship graph generation method and device, electronic equipment and storage medium | |
| CN115798059A (en) | Living body detection method and device, computer equipment and storage medium | |
| CN115204631A (en) | Product marketing management method and device, computer equipment and storage medium | |
| CN115422517A (en) | Identity authentication method, device, medium and equipment based on credit card | |
| CN115080405A (en) | System health detection method and device, electronic equipment and storage medium | |
| CN115344484A (en) | Method, device, medium and equipment for optimizing test resources based on bill inspection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |