CN115965648A

CN115965648A - Off-duty detection method and device, electronic equipment and storage medium

Info

Publication number: CN115965648A
Application number: CN202211622446.9A
Authority: CN
Inventors: 张瑞雪; 李捷; 徐敏; 周丹雅; 张斌
Original assignee: Shanghai Pudong Development Bank Co Ltd
Current assignee: Shanghai Pudong Development Bank Co Ltd
Priority date: 2022-12-16
Filing date: 2022-12-16
Publication date: 2023-04-14

Abstract

The embodiment of the invention discloses an off-post detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring at least one currently existing human body track and a current frame acquired by aiming at an on-duty area of a person to be detected, and performing human body detection on the current frame to obtain at least one human body detection frame; aiming at each human body track in at least one human body track, determining the coincidence proportion between a target detection frame and a corresponding area corresponding to the on-duty area of the personnel in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame; and under the condition that the coincidence proportion meets a preset first off-Shift condition, determining that the person represented by the human body track leaves the person on-Shift area. According to the technical scheme of the embodiment of the invention, off-duty detection of full-time full-range coverage can be realized with lower labor cost, and the detection result does not need to depend on manpower.

Description

Off-duty detection method and device, electronic equipment and storage medium

Technical Field

The embodiment of the invention relates to the field of computer vision, in particular to an off-duty detection method and device, electronic equipment and a storage medium.

Background

In a bank outlet scene, it is very important to accurately detect whether a teller leaves a post. At present, inspection personnel mainly carry out off-duty inspection in a field mode, an off-site mode and the like.

However, the manual inspection scheme has the problems of high labor cost, high dependence on manual work on detection results, and difficulty in full-time and full-range coverage, and needs to be improved.

Disclosure of Invention

The embodiment of the invention provides an off-duty detection method and device, electronic equipment and a storage medium, and solves the problems that the labor cost is high, the detection result is highly dependent on manpower, and the full-time and full-range coverage is difficult.

According to an aspect of the present invention, there is provided an off-Shift detection method, which may include:

acquiring at least one currently existing human body track and a current frame acquired by aiming at an on-duty area of a person to be detected, and performing human body detection on the current frame to obtain at least one human body detection frame;

aiming at each human body track in at least one human body track, determining the coincidence proportion between a target detection frame and a corresponding area corresponding to the on-duty area of the personnel in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame;

determining that the person represented by the human body track leaves the person on duty area under the condition that the coincidence proportion meets a preset first off duty condition;

the human body track is represented based on at least one historical detection frame of a person, each historical detection frame in the at least one historical detection frame is obtained by detecting the human body of a historical frame corresponding to the historical detection frame, and the historical frame is collected before a current frame.

According to another aspect of the present invention, there is provided an off duty detection apparatus, which may include:

the human body detection frame obtaining module is used for obtaining at least one currently existing human body track and a current frame collected by a to-be-detected person in an on-duty area, and performing human body detection on the current frame to obtain at least one human body detection frame;

a coincidence proportion determining module, configured to determine, for each human body trajectory in the at least one human body trajectory, a coincidence proportion between a target detection frame and a corresponding region corresponding to a person on duty region in the current frame, in a case where the target detection frame matching the human body trajectory exists in the at least one human body detection frame;

the off-duty detection module is used for determining that the person represented by the human body track leaves the person on-duty area under the condition that the coincidence proportion meets a preset first off-duty condition;

the human body track is represented based on at least one historical detection frame of a person, each historical detection frame in the at least one historical detection frame is obtained by detecting a human body of a historical frame corresponding to the historical detection frame, and the historical frame is collected before a current frame.

According to another aspect of the present invention, there is provided an electronic device, which may include:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor, when executed, to implement the off Shift detection method provided by any of the embodiments of the present invention.

According to another aspect of the present invention, there is provided a computer-readable storage medium having stored thereon computer instructions for causing a processor to execute a method for off Shift detection provided by any of the embodiments of the present invention.

According to the technical scheme of the embodiment of the invention, at least one current human body track and a current frame collected by a person to be detected in an on-duty area are obtained, and human body detection is carried out on the current frame to obtain at least one human body detection frame; aiming at each human body track in at least one human body track, determining the coincidence proportion between a target detection frame and a corresponding area corresponding to the on-duty area of the personnel in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame; and under the condition that the coincidence proportion meets a preset first off-duty condition, determining that the person represented by the human body track leaves the person on-duty area. According to the technical scheme, the coincidence proportion between the target detection frame obtained by human body detection and the corresponding region of the on-duty region of the personnel is determined based on human body detection and tracking, and the personnel off-duty detection is carried out by judging whether the coincidence proportion meets the first off-duty condition, so that the off-duty detection can be realized without manpower, and the problems that the labor cost is high, the detection result is highly dependent on manpower, and the full-time-interval and full-range coverage is difficult are solved.

It should be understood that the statements in this section do not necessarily identify key or critical features of any embodiment of the present invention, nor do they necessarily limit the scope of the present invention. Other features of the present invention will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a flowchart of an off-Shift detection method according to a first embodiment of the present invention;

fig. 2 is a schematic diagram of a corresponding area in an off duty detection method according to an embodiment of the present invention;

fig. 3 is a flowchart of an off-Shift detection method according to a second embodiment of the present invention;

fig. 4 is a schematic diagram of a process of determining an object detection frame in an off-Shift detection method according to a second embodiment of the present invention;

fig. 5 is a flowchart of an off-Shift detection method provided in the third embodiment of the present invention;

fig. 6 is a flowchart illustrating a workflow of an end-side device and a cloud server in an off-Shift detection method according to a third embodiment of the present invention;

fig. 7 is a flowchart of an alternative example of an off Shift detection method provided in the third embodiment of the present invention;

fig. 8 is a block diagram of an off-duty detection apparatus according to a fourth embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device implementing the off duty detection method according to the embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. The cases of "target", "original", etc. are similar and will not be described in detail herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example one

Fig. 1 is a flowchart of an off post detection method according to a first embodiment of the present invention. This embodiment can be applicable to personnel and leave the circumstances of post automated inspection. The method may be performed by the off-Shift detection apparatus provided in the embodiment of the present invention, and the apparatus may be implemented by software and/or hardware, and the apparatus may be integrated on an electronic device, and the electronic device may be an end-side device, for example, an end-side device deployed at a banking site.

Referring to fig. 1, the method of the embodiment of the present invention specifically includes the following steps:

s110, obtaining at least one currently existing human body track and a current frame collected by a person to be detected in an on-duty area, and performing human body detection on the current frame to obtain at least one human body detection frame.

The current frame can be understood as a video frame obtained by current acquisition when a person is acquired in an on-duty area in real time. The personnel on duty area is understood to be the area where the position where the person is required is located, which is the area in the actual three-dimensional space, and may be, for example, the area where the position of the person is located. The person can understand the object of the requirement judgment whether the person is off duty (namely, the person is out of the on duty area), for example, the object can be a worker needing to work on a workstation. The human body trajectory may be understood as a trajectory formed according to the movement route of the person. The human body detection frame can be understood as a human body frame of a certain person obtained by performing human body detection on the current frame, and in practical application, optionally, the number of the human body detection frames can be one, two or more, and different human body detection frames correspond to different persons respectively; optionally, the shape of the human body detection frame may be a human body outline, or may be a square, a rectangle, a circle, or the like.

In the embodiment of the invention, at least one current human body track and a current frame acquired aiming at the on duty area of a person to be detected are acquired, wherein the current frame can be understood as a video frame acquired by acquiring an image of the person in the on duty area through a camera or other modes; and carrying out human body detection on the current frame to obtain at least one human body detection frame. In the embodiment of the present invention, the manner of detecting human body is not limited.

S120, aiming at each human body track in the at least one human body track, determining the coincidence proportion between the target detection frame and the corresponding region corresponding to the on-duty region of the personnel in the current frame under the condition that the target detection frame matched with the human body track exists in the at least one human body detection frame, wherein the human body track is represented based on at least one historical detection frame of the personnel, each historical detection frame in the at least one historical detection frame is obtained by carrying out human body detection on the historical frame corresponding to the historical detection frame, and the historical frame is collected before the current frame.

Wherein each of the at least one body trajectory is subsequently processed in the same manner. Specifically, the target detection frame may be understood as a human body detection frame of a person corresponding to a human body trajectory in the current frame. The corresponding region may be understood as a region corresponding to the on Shift region of the person in the current frame, for example, referring to the current frame shown in fig. 2, the corresponding region is a region in a black frame circle. The overlapping proportion can be understood as the proportion between the part of the target detection frame overlapping with the corresponding area and the target detection frame.

In the embodiment of the present invention, for a currently processed human body trajectory, in a case that a target detection frame matched with the human body trajectory exists in at least one human body detection frame, that is, it is described that a person corresponding to the human body trajectory exists in a current frame, in order to further determine whether the person is off duty (that is, leaves the person on duty area), a coincidence proportion between the target detection frame and the corresponding area may be determined, and the coincidence proportion may reflect whether the position of the person corresponding to the target detection frame coincides with the person on duty area or not and the coincidence proportion, so as to facilitate to accurately determine whether the person is off duty subsequently.

It can be understood that, in the embodiment of the present invention, in a case that there is a target detection frame matching the human body trajectory in the at least one human body detection frame, the coincidence ratio between the target detection frame and the corresponding region corresponding to the people on duty region in the current frame is determined, rather than directly determining the coincidence ratio between each human body detection frame in the at least one human body detection frame and the corresponding region corresponding to the people on duty region in the current frame. According to the scheme, the target detection frame matched with the human body track is determined to exist in the at least one human body detection frame, the human body track of the target detection frame is ensured to correspond to the target detection frame, the possibility that people corresponding to the target detection frame are human bodies but not other objects can be improved, and therefore the phenomenon that due to the false detection of human body detection, the object which does not belong to the human body is detected into the human body and whether the object which does not belong to the human body leaves the post or not is judged in the follow-up process can be avoided.

It is noted that the above-mentioned human body trajectory is represented based on at least one historical detection box of the person it characterizes. For example, before the current frame, image acquisition is performed on the on Shift region of the person, at least one history frame may be acquired, and the history frame may be understood as a video frame obtained by performing image acquisition on the on Shift region of the person before the current frame by using a camera or in other manners; detecting a human body of at least one historical frame, so as to detect a historical detection frame corresponding to the person in each historical frame; the historical detection frames respectively correspond to position information on the corresponding historical frames, and the position information can be represented by coordinate information of the historical detection frames, so that the human body track of a person can be determined according to the position information corresponding to the historical detection frames respectively and the time information of the historical frames corresponding to the historical detection frames respectively. The history detection frame can be understood as a human body frame corresponding to a person corresponding to a human body track detected by a human body in the history frame.

S130, determining that the person represented by the human body track leaves the person on duty area under the condition that the superposition proportion meets a preset first off duty condition.

The first off-duty condition may be a preset condition that can determine whether the person is off duty according to the overlapping proportion, for example, the person off-duty condition may be that the overlapping proportion is less than or equal to 0.3. In the embodiment of the present invention, the content of the first off duty condition is not specifically limited.

In the embodiment of the present invention, it is determined that the person represented by the human body trajectory has left the person on duty area when the coincidence ratio satisfies the preset first off duty condition, for example, when the person off duty condition is that the coincidence ratio is less than or equal to 0.3 and the coincidence ratio is 0.2, that is, the coincidence ratio satisfies the preset first off duty condition, it may be determined that the person represented by the human body trajectory has left the person on duty area. And under the condition that the coincidence proportion does not meet the preset first off-Shift condition, determining that the person represented by the human body track does not leave the person on-Shift area.

According to the technical scheme of the embodiment of the invention, at least one current human body track and a current frame collected by a person to be detected in an on-duty area are obtained, and human body detection is carried out on the current frame to obtain at least one human body detection frame; aiming at each human body track in at least one human body track, determining the coincidence proportion between a target detection frame and a corresponding area corresponding to the on-duty area of the personnel in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame; and under the condition that the coincidence proportion meets a preset first off-Shift condition, determining that the person represented by the human body track leaves the person on-Shift area. According to the technical scheme, the coincidence proportion between the target detection frame obtained by human body detection and the corresponding region of the on-duty region of the personnel is determined based on human body detection and tracking, and the personnel off-duty detection is carried out by judging whether the coincidence proportion meets the first off-duty condition, so that the off-duty detection can be realized without manpower, and the problems that the labor cost is high, the detection result is highly dependent on manpower, and the full-time-interval and full-range coverage is difficult are solved.

An optional technical solution, performing human body detection on a current frame to obtain at least one human body detection frame, includes: acquiring a human body detection model which is trained in advance, inputting a current frame into the human body detection model, and obtaining at least one human body detection frame according to an output result of the human body detection model; wherein the human body detection model comprises a YOLO model.

It should be noted that if a video frame such as a current frame needs to be detected, different acquisition devices for acquiring the video frame may exist, for example, resolution differences of video frames acquired by different cameras are obvious, and an angle of the acquisition device is too high, and a person can only acquire a half of the person when facing the acquisition device or facing away from the acquisition device, which may cause a great challenge to human body detection and tracking; and if the video frame collected by the collecting equipment has the conditions of object shielding, glass reflection and the like, the human body detection and tracking can be interfered.

In order to solve the above problem, in the embodiment of the present invention, a human body detection model for performing human body detection may be trained in advance, and in a case that human body detection is required, the human body detection model trained in advance may be obtained, and the current frame is input into the human body detection model, and at least one human body detection frame is obtained according to an output result of the human body detection model. The human body detection model can comprise a YOLO (you only look once) model, for example, a YOLOV3 model, wherein YOLOV3 is a one-stage detection algorithm, the algorithm does not need to separately search a candidate region, can directly and uniformly perform dense sampling at different positions of an image, and then adopts a Full Convolution Network (FCN) to extract features and then directly classify and regress, and the whole human body detection process only needs one step and can meet the real-time processing requirement.

In the embodiment of the present invention, if the off-Shift detection method is applied to the end-side device, the YOLO model may be converted into an onnx/om model supported by the end-side device under the condition that the human body detection model includes the YOLO model, so that the model is more suitable for the end-side device and the model performance is not greatly affected.

In the embodiment of the invention, the YOLO model is used for detecting the at least one human body detection frame, so that the method is suitable for the condition that a plurality of persons exist, the efficiency of determining the at least one human body detection frame can be improved, and the instantaneity of determining the at least one human body detection frame is ensured.

Another optional technical solution, the off-duty detection method further includes: aiming at the tracking loss times corresponding to the human body track, carrying out zero setting on the tracking loss times under the condition that a target detection frame exists; and under the condition that the target detection frame does not exist, carrying out incremental processing on the tracking loss times, and under the condition that the tracking loss times after the incremental processing meet a preset second off-duty condition, determining that the person leaves the person on-duty area and deleting the human body track.

The tracking loss times can be understood as the times that the target detection frame corresponding to the person is not detected for the person corresponding to the human body track. The second off duty condition may be understood as a preset condition capable of determining whether the person is off duty according to the number of times of tracking loss, for example, the person off duty condition may be that the number of times of tracking loss is greater than or equal to 3 times.

In the embodiment of the invention, under the condition that the target detection frame exists, that is, the tracking loss of the personnel corresponding to the human body track is not caused, the zero setting processing can be carried out on the tracking loss times corresponding to the human body track. When the target detection frame does not exist, that is, when the person corresponding to the human body track is not detected at the current time point corresponding to the current frame, the tracking loss times can be increased, and when the tracking loss times after the increasing processing meet a preset second off-duty condition, the person corresponding to the human body track can be considered to be lost, and the person is determined to have left the person on-duty area and the human body track is deleted. According to the technical scheme, the situation that the person represented by the human body track leaves the person on duty area can be determined under the condition that the human body track exists but the target detection frame does not exist.

Another optional technical solution, the off-duty detection method further includes: aiming at the human body detection frame which is not successfully matched with any human body track in the at least one human body detection frame, generating a new track based on the human body detection frame which is not successfully matched; and under the condition that a human body detection frame matched with the newly increased track is detected in the future frame, updating at least one human body track by taking the newly increased track as the human body track, wherein the future frame is acquired after the current frame and is separated from the current frame by a preset number of video frames.

The future frame can be obtained by acquiring an image of the on-duty area of the person behind the current frame through a camera or other modes, and is separated from the current frame by a preset number of video frames. The preset number may be a preset number of interval video frames. A video frame may be understood as a frame in a video captured for an on Shift area for a person to be detected.

It can be understood that if off-duty detection is performed on each frame of video frames, the calculation pressure is high, and real-time performance cannot be guaranteed. In addition, a preset number of video frames can be spaced between the historical frame and the current frame; if the number of the historical frames is multiple, a preset number of video frames may be spaced between the historical frames.

It can be understood that, for example, the number of the human body tracks is 4, the number of the human body detection frames is 5, the 4 human body tracks are successfully matched with the 4 human body detection frames respectively, and a case that the human body tracks of the 1 human body detection frame are not matched with the human body detection frame also exists, that is, a case that the person corresponding to the human body detection frame may be a new person and the human body tracks are not matched with the new person exists. In order to solve the above problem, a newly added trajectory may be generated based on the unsuccessfully matched human body detection frame for the human body detection frame that is not successfully matched with any human body trajectory in the at least one human body detection frame, where the newly added trajectory is a trajectory corresponding to the human body detection frame that is not matched with the human body trajectory.

In the embodiment of the present invention, when a human body detection frame matched with a new trajectory is detected in a future frame, it can be stated that a human body detection frame that is not successfully matched with any human body trajectory in at least one human body detection frame is not a false detection for other non-human body objects, a person corresponding to the new trajectory is a newly-appeared person, the new trajectory can be used as a human body trajectory, and the at least one human body trajectory is updated based on the human body detection frame matched with the new trajectory detected in the future frame, so as to establish and track the human body trajectory of the newly-appeared person.

In the embodiment of the present invention, under the condition of establishing and tracking a human body trajectory of a newly-appeared person, a person on duty area corresponding to the person may not be known, and in order to solve the above problem, a coincidence proportion between a target detection frame and a corresponding area corresponding to the person on duty area in a current frame may be determined under the condition that a target detection frame corresponding to the person on duty area is not determined and there is coincidence with the corresponding area of the person on duty area, and under the condition that the coincidence proportion is greater than a preset proportion threshold, for example, the coincidence proportion is greater than 0.7, the person on duty area having the coincidence proportion with the target detection frame greater than the preset proportion threshold is used as the person on duty area corresponding to the person. For example, the person track corresponds to a corresponding attribute file, an Identity (id) of the person on duty region corresponding to the person track is recorded in the attribute file, and the id of the person on duty region whose coincidence ratio with the target detection frame is greater than a preset ratio threshold may be recorded in the attribute file corresponding to the person track.

Another optional technical solution is that, the number of the people on duty areas is at least two, and determining a coincidence ratio between the target detection frame and a corresponding area corresponding to the people on duty area in the current frame includes: aiming at corresponding areas in the current frame, which correspond to at least two on-duty areas of people, obtaining target areas to which the people belong in the corresponding areas; and determining the coincidence proportion between the target detection frame and the target area.

The target area may be understood as a corresponding area corresponding to a person in at least two corresponding areas.

It will be appreciated that where the number of people on Shift areas is at least two, rather than a person being in any one of the people on Shift areas, it may be desirable to determine whether a person is in the person's corresponding people on Shift area. In order to solve the above problem, in the embodiment of the present invention, for corresponding regions in a current frame, which correspond to at least two people on duty regions, target regions to which people belong in the corresponding regions respectively are obtained; for example, the on duty areas of the person are an area a and an area B, the area corresponding to the area a is an area a, the area corresponding to the area B is an area B, and the on duty area corresponding to the person is an area B, that is, for the area a and the area B corresponding to the area a and the area B in the current frame, the area B to which the person belongs in the area a and the area B may be obtained, and the area B is the target area corresponding to the person. Determining the coincidence proportion between the target detection frame and the target area, namely determining the coincidence proportion between the corresponding areas corresponding to the target detection frame and the personnel, thereby realizing that the personnel can be detected off duty under the condition that the number of the personnel on duty areas is at least two, and in addition, avoiding that the personnel are located in other personnel on duty areas which do not belong to the personnel under the condition that the number of the personnel on duty areas is at least two, and the condition that the personnel are off duty can not be determined.

Example two

Fig. 3 is a flowchart of another off duty detection method provided in the second embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the off-post detection method further includes: determining a tracking detection frame from the at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame in the at least one history detection frame; for each human body detection frame in at least one human body detection frame, calculating the intersection ratio between the human body detection frame and the tracking detection frame according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in the historical frame corresponding to the tracking detection frame; and determining whether a target detection frame exists in at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame. The explanations of the same or corresponding terms as those in the above embodiments are omitted.

Referring to fig. 3, the method of this embodiment may specifically include the following steps:

s210, acquiring at least one currently existing human body track and a current frame acquired by a person to be detected in an on-duty area, and performing human body detection on the current frame to obtain at least one human body detection frame.

S220, aiming at each human body track in at least one human body track, determining a tracking detection frame from at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame in the at least one history detection frame, wherein the human body track is represented based on the at least one history detection frame, each history detection frame in the at least one history detection frame is obtained by carrying out human body detection on the history frame corresponding to the history detection frame, and the history frame is acquired before the current frame.

Wherein, the collection time can be understood as the time for collecting the historical frame corresponding to the historical detection frame. The tracking detection frame may be understood as a history detection frame that can be used to determine whether or not a target detection frame exists in each history detection frame, and the tracking detection frame may be, for example, a history detection frame corresponding to an acquisition time at which a time difference between acquisition times of the current frame is the minimum, or a history detection frame corresponding to an acquisition time at which a time difference between acquisition times of the current frame is a preset time difference.

And S230, aiming at each human body detection frame in at least one human body detection frame, calculating the intersection ratio between the human body detection frame and the tracking detection frame according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in the historical frame corresponding to the tracking detection frame.

Wherein, at least one historical detection frame corresponds to the person represented by the human body track.

It can be understood that, because the time interval between the historical frame and the current frame is very small, even if a person corresponding to the human body trajectory moves greatly, the position difference between the human body detection frame and the tracking detection frame is still very small, and therefore, according to the above characteristics, for each human body detection frame in at least one human body detection frame, the Intersection and parallel ratio (IoU) between the human body detection frame and the tracking detection frame is calculated according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in the historical frame corresponding to the tracking detection frame. Particularly, when the off-duty detection method of the embodiment of the present invention is applied to other devices with limited computing power, such as a device on the end side, because the Central Processing Unit (CPU) of the other devices with limited computing power, such as the device on the end side, is limited, in order to avoid a large loss of tracking performance, an IoU algorithm may be adopted, which has a small computation amount, and is suitable for deployment of the device on the end side.

For example, referring to fig. 4, a and B in fig. 4 are tracking detection frames, a 'and B' are human detection frames of a current frame, and an IoU between the human detection frames and the tracking detection frames is calculated, so that whether a target detection frame exists in at least one human detection frame is determined according to a calculation result of the IoU, and if a detection frame with the highest calculation result of calculating the IoU with a is a ', a' may be used as the target detection frame.

In the embodiment of the present invention, a tracking algorithm based on machine learning, such as a Kalman algorithm or a Kernel Correlation Filtering (KCF) algorithm, may also be used to determine whether a target detection box exists in the at least one human detection box.

In the embodiment of the present invention, the position information may include coordinate positions of all corner points of the human body detection frame or the tracking detection frame, or coordinate positions of corner points of the upper left corner and the lower right corner, so as to calculate the intersection-parallel ratio between the human body detection frame and the tracking detection frame.

S240, determining whether a target detection frame matched with the human body track exists in at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame.

It can be understood that, in the case that the intersection ratio corresponding to the human body detection frame is large, that is, it indicates that the position difference between the human body detection frame and the tracking detection frame is small, and a target detection frame may exist in the human body detection frame, therefore, it may be determined whether a target detection frame exists in at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame, and the process of determining the target detection frame may also be understood as a process of tracking the person corresponding to the human body track corresponding to the target detection frame at the acquisition time corresponding to the current frame.

And S250, under the condition that the target detection frame exists in at least one human body detection frame, determining the coincidence proportion between the target detection frame and the corresponding region corresponding to the on-duty region of the person in the current frame.

And S260, under the condition that the coincidence proportion meets a preset first off duty condition, determining that the person represented by the human body track leaves the person on duty area, wherein the person corresponds to at least one historical detection frame.

According to the technical scheme of the embodiment of the invention, the tracking detection frame is determined from the at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame in the at least one history detection frame; aiming at each human body detection frame in at least one human body detection frame, calculating the intersection ratio between the human body detection frame and the tracking detection frame according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in the historical frame corresponding to the tracking detection frame; and determining whether a target detection frame exists in at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame. According to the technical scheme, whether the target detection frame exists in at least one human body detection frame is determined by calculating the intersection ratio between the human body detection frame and the tracking detection frame and according to the intersection ratio corresponding to each human body detection frame, so that whether the target detection frame exists in the human body detection frame can be determined to correspond to the human body track, and therefore the person in the current frame can correspond to the human body track.

An optional technical solution, determining whether a target detection frame exists in at least one human detection frame according to a cross-over ratio respectively corresponding to each human detection frame, includes: determining the intersection ratio with the maximum numerical value in the intersection ratios respectively corresponding to each human body detection frame; determining whether a target detection frame exists in at least one human body detection frame according to whether the intersection ratio with the maximum value exceeds a preset intersection ratio threshold value; the off-post detection method further comprises the following steps: and under the condition that the target detection frame exists, taking the human body detection frame corresponding to the intersection ratio with the largest numerical value in at least one human body detection frame as the target detection frame.

The intersection ratio threshold may be understood as a preset threshold for determining whether a target detection frame exists in at least one human detection frame through an intersection ratio.

In the embodiment of the invention, the intersection ratio with the maximum value in the intersection ratios respectively corresponding to each human body detection frame can be determined, and the intersection ratio is more likely to be the target detection frame than the corresponding human body detection frame. Considering the situation that the intersection ratio of the largest numerical value is still small although the numerical value of the corresponding human body detection frame in the corresponding intersection ratio of each human body detection frame is the largest, namely the situation that the target detection frame does not exist, whether the intersection ratio of the largest numerical value exceeds a preset intersection ratio threshold value or not can be judged, and under the situation that the intersection ratio exceeds the preset intersection ratio threshold value, the target detection frame exists in at least one human body detection frame; and under the condition that the preset intersection ratio threshold value is not exceeded, the target detection frame does not exist in the at least one human body detection frame. And under the condition that the target detection frame exists, taking the human body detection frame corresponding to the intersection ratio with the largest numerical value in at least one human body detection frame as the target detection frame. By the technical scheme, the accuracy rate of determining whether the target detection frame exists can be improved, and the target detection frame can be accurately determined under the condition that the target detection frame exists.

Another optional technical solution, the off-duty detection method further includes: under the condition that the target detection frame exists, updating the human body track based on the target detection frame; determining a tracking detection frame from the at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame in the at least one history detection frame, comprising: acquiring the acquisition time of a historical frame corresponding to each historical detection frame in at least one historical detection frame, and determining the adjacent time with the minimum time difference between the acquisition time of the historical frame and the acquisition time of the current frame; and taking the history detection frame corresponding to the adjacent time in the at least one history detection frame as a tracking detection frame.

In the embodiment of the invention, under the condition that the target detection frame exists, the human body track is updated based on the target detection frame, so that the off-duty detection can be carried out based on the updated human body track in the future.

In the embodiment of the present invention, the acquisition times of the historical frames corresponding to each of the at least one historical detection frame may be obtained, and the adjacent time with the smallest time difference between the acquisition times of the historical frames and the acquisition time of the current frame in the acquisition times of the historical frames corresponding to each of the historical detection frames is determined, where the adjacent time is the acquisition time with the smallest time difference between the acquisition times of the historical frames and the acquisition time of the current frame in the acquisition times of the historical frames corresponding to each of the historical frames, and the historical frame corresponding to the acquisition time may be adjacent to the current frame, that is, there is no other historical frame between the historical frame corresponding to the acquisition time and the current frame; and taking the history detection frame corresponding to the adjacent time in the at least one history detection frame as a tracking detection frame.

According to the technical scheme of the embodiment of the invention, under the condition that the target detection frame exists, the human body track is updated based on the target detection frame; acquiring the acquisition time of a historical frame corresponding to each historical detection frame in at least one historical detection frame, and determining the adjacent time with the minimum time difference between the acquisition time of the historical frame and the acquisition time of the current frame; and taking the history detection frame corresponding to the adjacent time in the at least one history detection frame as a tracking detection frame. The method and the device not only facilitate off-post detection in the future, but also improve the accuracy of determining whether the target detection frame exists in at least one human body detection frame based on the determined tracking detection frame.

Before describing the third embodiment of the present invention, an application scenario thereof is exemplarily described. For example, it can be understood that there may be a large number of target nodes requiring off-duty detection, and there may also be a large number of acquisition devices such as cameras capable of acquiring video frames of each target node, and if the cloud server is used to perform the off-duty detection uniformly, the requirement on network bandwidth may be high, and it is difficult to implement full-time monitoring. In order to solve The above problems, in The embodiment of The present invention, the off-duty detection method may be applied to an end-side device, where The end-side device is deployed in a target site corresponding to an on-duty area of a person, where The end-side device may be, for example, a mobile device or an Internet of Things (IOT) device, and The target site may be understood as a region where The on-duty area of The person is located, such as a bank site or a branch company of a certain company, so as to meet a requirement for avoiding network bandwidth pressure caused by uploading a video frame to a cloud server, and meet a requirement for real-time off-duty detection processing of multiple target sites in a case of implementing real-time video frame acquisition of multiple acquisition devices of The target site.

EXAMPLE III

Fig. 5 is a flowchart of another off duty detection method provided in the third embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, after determining that the person represented by the human body trajectory has left the person on duty area, the off duty detection method further includes: and reporting the message event to a cloud server so that the cloud server processes the message event, wherein the message event is used for indicating that the personnel leave the on-duty area. The explanations of the same or corresponding terms as those in the above embodiments are omitted.

Referring to fig. 5, the method of this embodiment may specifically include the following steps:

s310, acquiring at least one currently existing human body track and a current frame acquired by a to-be-detected person in an on-duty area, and performing human body detection on the current frame to obtain at least one human body detection frame.

S320, aiming at each human body track in the at least one human body track, under the condition that a target detection frame matched with the human body track exists in the at least one human body detection frame, determining the coincidence proportion between the target detection frame and a corresponding region corresponding to the on-duty region of the personnel in the current frame, wherein the human body track is represented based on at least one historical detection frame of the personnel, each historical detection frame in the at least one historical detection frame is obtained by carrying out human body detection on a historical frame corresponding to the historical detection frame, and the historical frame is collected before the current frame.

S330, determining that the person represented by the human body track leaves the person on duty area under the condition that the superposition proportion meets a preset first off duty condition.

And S340, reporting the message event to a cloud server so that the cloud server processes the message event, wherein the message event is used for indicating that the personnel leave the on-duty area.

It can be understood that the end-side device is only used for determining whether the person represented by the human body track leaves the person on duty area, and the determination of whether the person is abnormal off duty depends on the cloud server. Therefore, when the off-duty detection method is applied to the end-side device, after the end-side device determines that the person represented by the human body trajectory leaves the person on duty area, a message event needs to be reported to the server, where the message event is an event corresponding to the relevant information that the person leaves the person on duty area, so that the cloud server processes the message event.

Optionally, reporting the message event to the cloud server may include: and recording the leaving time of the personnel, if the personnel are still in the state of leaving the personnel on Shift area after the preset leaving time after the leaving time, and reporting the message event and the leaving time of the personnel to the cloud server. Because the situation that the personnel leave for a short time may exist, for example, the personnel leave the personnel on duty area to receive water or go to a toilet, the leaving time of the personnel leaving the personnel on duty area can be recorded, and the personnel are located in the personnel on duty area within the preset leaving time after the leaving time, so that the message event is not reported to the cloud server; after the preset leaving time length after the leaving time, the personnel are still in the state of leaving the personnel on Shift area, and the message event and the leaving time of the personnel can be reported to the cloud server, so that unnecessary message events are avoided being reported.

According to the technical scheme, the off-duty detection method is applied to the end-side equipment, the end-side equipment is deployed at a target network point corresponding to the on-duty area of the personnel, and after the fact that the personnel represented by the human body track leave the on-duty area of the personnel is determined, the message event is reported to the cloud server so that the cloud server can process the message event, wherein the message event is used for indicating that the personnel leave the on-duty area of the personnel. According to the technical scheme, the off-duty detection is carried out by adopting the end-side equipment, frequent network transmission is not needed, full-time monitoring can be realized, and a hardware detection device is not needed to be additionally installed, so that personnel are prevented from being interfered by electronic equipment such as an infrared sensor and a pressure sensor for a long time, and the damage to a human body is avoided; and after determining that the person represented by the human body track leaves the person on duty area, reporting the message event to the cloud server so that the cloud server processes the message event, so that the cloud server can automatically report the message event to the cloud server under the condition that the person leaves the person on duty area, and the cloud server can perform subsequent processing, thereby further avoiding the dependence on manual participation.

An optional technical solution, so that a cloud server processes a message event, includes: the cloud server acquires the business information of the personnel corresponding to the message event, and determines whether the personnel abnormally leave the on-duty area of the personnel according to the business information; the off-duty detection method further comprises the following steps: and sending out an early warning prompt when the personnel leave the on-duty area abnormally.

The service information may be understood as information capable of reflecting a service condition of a person, and may be, for example, leave information, information related to that the person needs to leave the post to handle other services, or a sign-off lock screen record, so as to determine whether the person leaves the post area abnormally.

It can be understood that there may be a case that a person has a leave request behavior in advance, or the person needs to leave the region on duty to handle other services for the client, and in the above case, even if the person leaves the region on duty, the person belongs to the region on duty for the person leaving normally, and in order to determine whether the person leaves the region on duty abnormally, in the embodiment of the present invention, the cloud server may obtain the service information of the person corresponding to the message event, and determine whether the person leaves the region on duty abnormally according to the service information.

In the embodiment of the invention, when a person leaves the on-duty area abnormally, an early warning prompt can be sent out, wherein the early warning prompt is a related prompt for the person leaving the on-duty area abnormally; the early warning prompt can be an indicator light prompt or a voice prompt; the early warning prompt can be a prompt for personnel, and can also be a prompt for personnel management personnel. In the embodiment of the present invention, the manner of the warning prompt and the object are not specifically limited.

For example, referring to fig. 6, real Time Streaming Protocol (RTSP) stream data including a current frame, collected by a collecting device such as a general camera or a smart camera, may be obtained to a terminal device; the end-side equipment performs off-post detection real-time analysis through human body detection and tracking based on RTSP streaming data, and sends a message event to the cloud server according to an analysis result; the method comprises the steps that a cloud management platform on a cloud server sends a message event to a management node through a real-time message queue, the management node obtains sign-off records and business records of personnel corresponding to the message event from a database node, whether the personnel leave an on-duty area abnormally is determined based on the sign-off records and the business records, and an early warning prompt is sent out under the condition that the personnel leave the on-duty area abnormally.

In the embodiment of the invention, the early warning prompt can be sent out according to whether the personnel leave the on-duty area abnormally or not through the cloud server, and under the condition that the personnel leave the on-duty area abnormally. The reminding device can remind people in time when the people leave the on-duty area abnormally.

In order to better understand the technical solution of the above embodiment of the present invention, an alternative example is provided herein. In an exemplary manner, the first and second electrodes are,

the method comprises the steps of firstly, obtaining a video stream address of a target network point, and aiming at each camera in at least one camera in the target network point, forming a configuration file containing an RTSP address and range information of a corresponding region corresponding to an on-Shift region of a person in a video frame shot by the framing camera, wherein each camera in at least one camera corresponds to a corresponding region of the on-Shift region of at least one person needing to be subjected to off-Shift detection.

Step two: the end-side equipment reads the configuration files corresponding to the at least one camera respectively and decodes the RTSP stream according to the RTSP address in the configuration files; and respectively sending the acquired current frames to different processes of the end-side equipment by at least one camera, and respectively acquiring human body detection frames corresponding to the personnel in the current frames by each process based on a previously trained Yolov3 model according to the read configuration information in the configuration file of the corresponding camera.

Step three: the method comprises the steps of carrying out human body detection once every 5 video frames, tracking each human body track in at least one current human body track through an IoU algorithm based on a tracking detection frame determined in at least one historical detection frame corresponding to the human body track, and storing the human body track, wherein the human body track can be formed by coordinates of each historical detection frame corresponding to a person, each coordinate can be presented in a list form, and the time difference between the acquisition time of the historical frame corresponding to the tracking detection frame and the acquisition time of a current frame is minimum. The specific process of tracking the human body detection frame through the IoU algorithm is as follows:

acquiring all human body detection frames in the current frame;

for each human body track in at least one currently existing human body track, finding a human body detection frame with the maximum IoU corresponding to the tracking detection frame corresponding to the human body track in the current frame, judging whether the maximum IoU exceeds a preset intersection ratio threshold value, if so, determining that a target detection frame exists, otherwise, determining that the target detection frame does not exist;

if the target detection frame exists, updating the target detection frame into a target track, determining that the tracking of the personnel in the current frame is successful, and carrying out zero setting processing on tracking loss times;

under the condition that no target detection frame exists, incremental processing is carried out on tracking loss times, and under the condition that the incremental processed tracking loss times meet 3 times, the situation that a person leaves a person on duty area is determined, and a human body track is deleted;

for the human body detection frame which is not successfully matched with any human body track in the at least one human body track, the human body detection frame is added into the human body track queue as a new track, in order to reduce the influence of false detection, if the new track is continuously tracked for more than 2 times, the new track is determined to be the human body track, and the personnel corresponding to the new track are effective tracking personnel.

Step four: according to the method, for a current frame corresponding to each camera in at least one camera, judging whether a person leaves a person on duty area or not under the condition that a target detection frame matched with a human body track exists in at least one human body detection frame according to range information of a corresponding area in a configuration file, starting timing when the person is in a state of leaving, and generating a message event containing the camera information, the range information, the state change condition and the leaving time if no person appears in the person on duty area within two minutes.

The main flow for judging whether the person leaves the on Shift area is as follows:

when a person appears in a person on duty area for the first time and the person is an effective tracker, judging whether the coincidence proportion of a human body detection frame corresponding to the person and a corresponding area corresponding to the person on duty area is greater than 0.7, if so, marking the area identity identification number in the person attribute file as the id of the person on duty area to indicate that the person and the person appearing in the person on duty area;

for the personnel with the marked area identity identification numbers, if the coincidence proportion of the human body detection frame corresponding to the personnel and the corresponding area of the marked personnel on duty area is more than 0.3, the personnel does not leave the personnel on duty area;

and if the personnel do not meet the conditions, determining that the personnel are not in the corresponding personnel on duty area.

Step five: the end-side equipment sends the message event to the cloud server, the cloud server compares the sign-off screen locking records of the personnel, and if the personnel do not sign off or lock the screen before leaving, an early warning prompt is sent.

In order to better understand the technical solutions of the embodiments of the present invention, another alternative example is provided herein. Exemplarily, referring to fig. 7, an embodiment of the present invention may be deployed at a peer-side device: the off-duty detection is realized by the data module, the human body detection module, the human body tracking module and the post-processing module, and the data module can acquire at least one human body track currently existing and a current frame acquired by aiming at an on-duty area of a person to be detected as a pre-processing process; the human body detection module can perform human body detection on the current frame to obtain at least one human body detection frame, and the human body tracking module can determine that a target detection frame matched with the human body track exists in the at least one human body detection frame and determine the target detection frame as a detection and tracking process aiming at each human body track in the at least one human body track; the post-processing module can determine the coincidence proportion between the target detection frame and the corresponding region corresponding to the on-duty region of the person in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame, and determine that the person represented by the human body track leaves the on-duty region of the person under the condition that the coincidence proportion meets a preset first off-duty condition, and record the leaving condition of the person as a post-processing process.

Example four

Fig. 8 is a block diagram of an off-Shift detection apparatus according to a fourth embodiment of the present invention, which is configured to execute the off-Shift detection method according to any of the above embodiments. The device and the off-post detection method of each embodiment belong to the same inventive concept, and details which are not described in detail in the embodiment of the off-post detection device can refer to the embodiment of the off-post detection method. Referring to fig. 8, the apparatus may specifically include: a human body detection frame obtaining module 410, a coincidence proportion determining module 420 and an off-duty detecting module 430.

The human body detection frame obtaining module 410 is configured to obtain at least one currently existing human body track and a current frame acquired for an on duty area of a person to be detected, and perform human body detection on the current frame to obtain at least one human body detection frame;

a coincidence ratio determining module 420, configured to determine, for each human body trajectory in the at least one human body trajectory, a coincidence ratio between the target detection frame and a corresponding region corresponding to the person on duty region in the current frame in the case that a target detection frame matching the human body trajectory exists in the at least one human body detection frame;

the off-duty detection module 430 is configured to determine that the person represented by the human body trajectory has left the on-duty area when the coincidence proportion meets a preset first off-duty condition;

Optionally, the human body detection frame obtaining module 410 may include:

a human body detection frame obtaining unit, configured to obtain a human body detection model that is trained in advance, input the current frame into the human body detection model, and obtain at least one human body detection frame according to an output result of the human body detection model;

wherein the human body detection model comprises a YOLO model.

Optionally, the off-duty detection apparatus may further include:

the tracking detection frame determining module is used for determining a tracking detection frame from the at least one historical detection frame according to the acquisition time of the historical frame corresponding to each historical detection frame in the at least one historical detection frame;

the intersection and comparison determining module is used for calculating the intersection and comparison between the human body detection frame and the tracking detection frame according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in the historical frame corresponding to the tracking detection frame aiming at each human body detection frame in the at least one human body detection frame;

and the target detection frame determining module is used for determining whether a target detection frame exists in at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame.

On the basis of the foregoing scheme, optionally, the target detection frame determining module may include:

the intersection ratio determining unit is used for determining the intersection ratio with the largest median value in the intersection ratios respectively corresponding to each human body detection frame;

the target detection frame determining unit is used for determining whether a target detection frame exists in at least one human body detection frame according to whether the intersection ratio with the largest numerical value exceeds a preset intersection ratio threshold value;

off duty detection device can also include:

and the target detection frame is used as a module for taking the human body detection frame corresponding to the largest intersection ratio of the numerical values in at least one human body detection frame as the target detection frame under the condition that the target detection frame exists.

On the basis of the above scheme, optionally, the off-post detection device may further include:

the first human body track updating module is used for updating the human body track based on the target detection frame under the condition that the target detection frame exists;

the tracking detection box determination module may include:

the adjacent time determining unit is used for acquiring the acquisition time of the historical frame corresponding to each historical detection frame in the at least one historical detection frame and determining the adjacent time with the minimum time difference value between the acquisition time of the historical frame corresponding to each historical detection frame and the acquisition time of the current frame;

and the tracking detection frame is used as a unit for taking the history detection frame corresponding to the adjacent time in at least one history detection frame as the tracking detection frame.

Optionally, the off-duty detection apparatus may further include:

the zero setting processing module is used for carrying out zero setting processing on tracking loss times under the condition that a target detection frame exists according to the tracking loss times corresponding to the human body track;

and the human body track deleting module is used for carrying out incremental processing on the tracking loss times under the condition that the target detection frame does not exist, and determining that the person leaves the on-duty area and deleting the human body track under the condition that the tracking loss times after the incremental processing meets a preset second off-duty condition.

Optionally, the off-duty detection apparatus may further include:

the newly added track generation module is used for generating a newly added track based on the unsuccessfully matched human body detection frame aiming at the human body detection frame which is not successfully matched with any human body track in the at least one human body detection frame;

and the second human body track updating module is used for updating at least one human body track by taking the newly added track as the human body track under the condition that the human body detection frame matched with the newly added track is detected in the future frame, wherein the future frame is acquired after the current frame and is separated from the current frame by a preset number of video frames.

Optionally, the number of the people in the on duty areas is at least two, and the overlap ratio determining module 420 may include:

the target area acquisition unit is used for acquiring target areas of the personnel in the corresponding areas respectively corresponding to the at least two personnel on-duty areas in the current frame;

and the coincidence proportion determining unit is used for determining the coincidence proportion between the target detection frame and the target area.

Optionally, the off-duty detection apparatus is integrated in the end-side device, the end-side device is deployed at a target site corresponding to the on-duty area of the person, and the off-duty detection apparatus may further include:

and the message event processing module is used for reporting the message event to the cloud server after determining that the person represented by the human body track leaves the person on duty area, so that the cloud server processes the message event, wherein the message event is used for indicating that the person leaves the person on duty area.

On the basis of the foregoing solution, optionally, the message event processing module may include:

the abnormal leaving determining unit is used for enabling the cloud server to acquire the business information of the personnel corresponding to the message event and determining whether the personnel leave the on-duty area abnormally according to the business information;

off-post detection device can also include:

and the early warning prompt sending module is used for sending out an early warning prompt under the condition that the personnel leave the on-duty area abnormally.

In the off-duty detection device provided by the fourth embodiment of the present invention, at least one currently existing human body track and a current frame collected for an on-duty area of a person to be detected are obtained through the human body detection frame obtaining module, and human body detection is performed on the current frame to obtain at least one human body detection frame; aiming at each human body track in at least one human body track, determining the coincidence proportion between a target detection frame and a corresponding area corresponding to the personnel on duty area in the current frame under the condition that the target detection frame matched with the human body track exists in at least one human body detection frame through a coincidence proportion determining module; determining that the person represented by the human body track leaves the person on duty area by the off duty detection module under the condition that the coincidence proportion meets a preset first off duty condition; the human body track is represented based on at least one historical detection frame of a person, each historical detection frame in the at least one historical detection frame detects the human body through a historical frame corresponding to the historical detection frame, and the historical frame is collected before a current frame. The device determines the coincidence proportion between the target detection frame obtained by human body detection and the corresponding region of the on-duty region of the personnel based on human body detection and tracking, and carries out personnel off-duty detection by judging whether the coincidence proportion meets the first off-duty condition, so that off-duty detection can be realized without manpower, and the problems that the labor cost is high, the detection result is highly dependent on manpower and the full-time-interval and full-range coverage is difficult are solved.

The off-post detection device provided by the embodiment of the invention can execute the off-post detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

It should be noted that, in the embodiment of the off duty detection apparatus, each unit and each module included in the embodiment are only divided according to functional logic, but are not limited to the above division, as long as the corresponding function can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.

EXAMPLE five

FIG. 9 illustrates a block diagram of an electronic device 10 that may be used to implement embodiments of the present invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 9, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, and the like, wherein the memory stores a computer program executable by the at least one processor, and the processor 11 can perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from a storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data necessary for the operation of the electronic apparatus 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.

A number of components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, or the like; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, or the like. The processor 11 performs the various methods and processes described above, such as the off Shift detection method.

In some embodiments, the off Shift detection method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the off Shift detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the off Shift detection method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.

The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for off-duty detection is characterized by comprising the following steps:

for each human body track in the at least one human body track, under the condition that a target detection frame matched with the human body track exists in the at least one human body detection frame, determining the coincidence proportion between the target detection frame and a corresponding area corresponding to the on-duty area of the person in the current frame;

determining that the person represented by the human body track leaves the person on duty area under the condition that the superposition proportion meets a preset first off duty condition;

the human body track is represented based on at least one historical detection frame of the person, each historical detection frame in the at least one historical detection frame is obtained by performing human body detection on a historical frame corresponding to the historical detection frame, and the historical frame is acquired before the current frame.

2. The method of claim 1, wherein the performing human detection on the current frame to obtain at least one human detection frame comprises:

acquiring a human body detection model which is trained in advance, inputting the current frame into the human body detection model, and obtaining at least one human body detection frame according to an output result of the human body detection model;

wherein the human detection model comprises a YOLO model.

3. The method of claim 1, further comprising:

determining a tracking detection frame from the at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame in the at least one history detection frame;

for each human body detection frame in the at least one human body detection frame, calculating an intersection-parallel ratio between the human body detection frame and the tracking detection frame according to the position information of the human body detection frame in the current frame and the position information of the tracking detection frame in a historical frame corresponding to the tracking detection frame;

and determining whether the target detection frame exists in the at least one human body detection frame according to the intersection ratio corresponding to each human body detection frame.

4. The method according to claim 3, wherein the determining whether the target detection frame exists in the at least one human detection frame according to the intersection ratio respectively corresponding to each human detection frame comprises:

determining the intersection ratio with the largest median value in the intersection ratios respectively corresponding to each human body detection frame;

determining whether the target detection frame exists in the at least one human body detection frame according to whether the intersection ratio with the maximum numerical value exceeds a preset intersection ratio threshold value;

the method further comprises the following steps:

and under the condition that the target detection frame exists, taking the human body detection frame corresponding to the largest intersection ratio of the numerical values in the at least one human body detection frame as the target detection frame.

5. The method of claim 3, further comprising:

under the condition that the target detection frame exists, updating the human body track based on the target detection frame;

the determining a tracking detection frame from the at least one history detection frame according to the acquisition time of the history frame corresponding to each history detection frame of the at least one history detection frame includes:

acquiring the acquisition time of a historical frame corresponding to each historical detection frame in the at least one historical detection frame, and determining the adjacent time with the minimum time difference between the acquisition time of the historical frame and the acquisition time of the current frame;

and taking a history detection frame corresponding to the adjacent time in the at least one history detection frame as a tracking detection frame.

6. The method of claim 1, further comprising:

aiming at the tracking loss times corresponding to the human body track, carrying out zero setting processing on the tracking loss times under the condition that the target detection frame exists;

and under the condition that the target detection frame does not exist, performing incremental processing on the tracking loss times, and under the condition that the tracking loss times after the incremental processing meet a preset second off-duty condition, determining that the person leaves the on-duty area and deleting the human body track.

7. The method of claim 1, further comprising:

aiming at the human body detection frame which is not successfully matched with any human body track in the at least one human body detection frame, generating a new track based on the human body detection frame which is not successfully matched;

and under the condition that a human body detection frame matched with the newly added track is detected in a future frame, updating the at least one human body track by taking the newly added track as the human body track, wherein the future frame is acquired after the current frame and has a preset number of video frames with the current frame.

8. The method of claim 1, wherein the number of people on Shift regions is at least two, and wherein said determining a proportion of coincidence between the target detection box and a corresponding region in the current frame corresponding to the people on Shift region comprises:

aiming at corresponding areas in the current frame, which correspond to the at least two people on duty areas respectively, obtaining target areas to which the people belong in the corresponding areas respectively;

and determining the coincidence proportion between the target detection frame and the target area.

9. The method of claim 1, applied to an end-side device deployed at a target site corresponding to the people on duty area, further comprising, after said determining that the person characterized by the human trajectory has left the people on duty area:

and reporting a message event to a cloud server so that the cloud server processes the message event, wherein the message event is used for indicating that the personnel leave the personnel on-duty area.

10. The method of claim 9, wherein the causing the cloud server to process the message event comprises:

enabling the cloud server to acquire the business information of the personnel corresponding to the message event, and determining whether the personnel abnormally leave the personnel on-duty area according to the business information;

the method further comprises the following steps:

and sending out an early warning prompt under the condition that the personnel leaves the on-duty area abnormally.

11. The utility model provides a detection device off duty which characterized in that includes:

the human body detection frame obtaining module is used for obtaining at least one currently existing human body track and a current frame collected aiming at an on-duty area of a person to be detected, and carrying out human body detection on the current frame to obtain at least one human body detection frame;

a coincidence proportion determining module, configured to determine, for each human body trajectory in the at least one human body trajectory, a coincidence proportion between a target detection frame and a corresponding area corresponding to the people on duty area in the current frame, if the target detection frame matching the human body trajectory exists in the at least one human body detection frame;

the off-duty detection module is used for determining that the person represented by the human body track leaves the on-duty area of the person under the condition that the coincidence proportion meets a preset first off-duty condition;

the human body track is represented based on at least one historical detection frame of the person, each historical detection frame in the at least one historical detection frame is obtained by detecting a human body of a historical frame corresponding to the historical detection frame, and the historical frame is collected before the current frame.

12. An electronic device, comprising:

at least one processor; and

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to cause the at least one processor to perform the off Shift detection method of any one of claims 1-10.

13. A computer-readable storage medium having stored thereon computer instructions for causing a processor to perform the off Shift detection method of any of claims 1-10 when executed.