CN113111847A - Automatic monitoring method, device and system for process circulation - Google Patents
Automatic monitoring method, device and system for process circulation Download PDFInfo
- Publication number
- CN113111847A CN113111847A CN202110476850.9A CN202110476850A CN113111847A CN 113111847 A CN113111847 A CN 113111847A CN 202110476850 A CN202110476850 A CN 202110476850A CN 113111847 A CN113111847 A CN 113111847A
- Authority
- CN
- China
- Prior art keywords
- frame
- tracking
- tunnel
- dynamic event
- frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 118
- 230000008569 process Effects 0.000 title claims abstract description 67
- 238000012544 monitoring process Methods 0.000 title claims abstract description 34
- 238000001514 detection method Methods 0.000 claims abstract description 44
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 28
- 238000013135 deep learning Methods 0.000 claims abstract description 19
- 238000007781 pre-processing Methods 0.000 claims abstract description 8
- 238000011410 subtraction method Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 9
- 230000009471 action Effects 0.000 claims description 7
- 239000000203 mixture Substances 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000012806 monitoring device Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 description 15
- 238000010276 construction Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000009430 construction management Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000011112 process operation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/109—Time management, e.g. calendars, reminders, meetings or time accounting
- G06Q10/1091—Recording time for administrative or management purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Strategic Management (AREA)
- General Health & Medical Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Educational Administration (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method, a device and a system for automatically monitoring process circulation, which comprise the following steps: acquiring a video stream of a tunnel portal monitoring area, and preprocessing the video stream to obtain a plurality of frame pictures; judging whether a dynamic event occurs in a multi-frame picture, and if so, marking the frame picture with the dynamic event; carrying out tracking prediction on the marked frame picture by using face detection based on opencv and depsort based on deep learning; carrying out face information recognition on the tracked operator by a face recognition model trained by an AM-softmax algorithm; when a plurality of operators of the same work type are recognized to enter or leave the tunnel within a certain time range, the current process start or end of the work type is recorded. The invention identifies the information of the operating personnel by detecting and identifying a plurality of operating personnel who have the same work type in a certain time range to enter and exit the tunnel, automatically records the time of starting or ending the working procedure operation, does not need to manually record the condition that the operating personnel enter and exit the tunnel in the whole process, and improves the efficiency of the working procedure management.
Description
Technical Field
The invention belongs to the technical field of target pedestrian tracking, and particularly relates to an automatic monitoring method, device and system for process circulation.
Background
With the rapid development of national traffic construction, along with the continuous development of construction career and continuous innovation of construction technology in China, more and more operation projects such as tunnels and the like are provided, the construction cost is high, the construction period is long, the span is large, the process management in the construction process is an important link for enhancing the construction quality of the project, and the inspection and comparison are mainly carried out on the construction progress and the construction quality of the project so as to ensure the progress of the construction plan and the construction quality of the project.
In the prior art, manual recording and comparison tests are adopted for data collection, arrangement and summarization in the tunnel construction process management process, for example, when an operator who goes in and out of a tunnel is managed, the time for the operator of a certain work type to enter the tunnel to perform operation and the time for the operator of the work type to leave the tunnel are recorded in a manual recording mode, and the time interval from the time when the operator of the work type enters the tunnel to perform operation to the time when the operator leaves the tunnel is recorded as a process cycle, so as to realize process management. However, the manual process data recording mode not only greatly increases the consumption of manpower, material resources, financial resources and time in the process management process, but also inevitably causes various errors such as similar calculation, statistics and analysis due to human participation, thereby causing data distortion and influencing the construction quality and progress.
Disclosure of Invention
The present invention is directed to a method, apparatus and system for automatically monitoring process cycles to address at least one of the problems of the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the present invention provides a method of automatically monitoring a process cycle, the method comprising:
acquiring a video stream of a tunnel portal, and preprocessing the video stream to obtain a plurality of frame pictures;
judging whether a dynamic event occurs in a plurality of frames of pictures, and if so, marking the frames of pictures with the dynamic event;
carrying out tracking prediction on the marked frame picture by using a face detection model based on opencv and a depsort model based on deep learning;
carrying out face information recognition on the tracked operator based on a face recognition model trained by an AM-softmax algorithm;
when a plurality of operators of the same work type are recognized to enter or leave the tunnel within a certain time range, the current process start or end of the work type is recorded.
In one possible design, the method further includes:
acquiring time intervals of a plurality of operating personnel of the same work type entering the tunnel and leaving the tunnel;
the time interval is recorded as the time of one process cycle.
In one possible design, the determining whether a dynamic event occurs in multiple frames of the frame picture includes:
and judging whether a dynamic event occurs in the frame pictures of the plurality of frames by adopting one algorithm of a two-frame difference frame method, a three-frame difference frame method, a video background subtraction method based on Gaussian mixture learning or a python background model subtraction method.
In a possible design, the determining whether a dynamic event occurs in multiple frames of the picture by using a two-frame difference frame method includes:
carrying out difference on pixel values at the same position of adjacent frames to obtain a difference image;
and carrying out binarization on the differential image, wherein when the pixel value change of the differential image is greater than a preset threshold value, a dynamic event occurs in the adjacent frame.
In one possible design, the human face detection model based on opencv and the deepsort model based on deep learning perform operator tracking prediction on the marked frame picture, including:
inputting the marked frame picture into the face detection model based on opencv for rapid face detection;
and inputting the frame picture of the detected face into the deep learning-based deepsort model for tracking the operator, predicting the action track of the operator, updating the tracking result and outputting the tracking result.
In one possible design, inputting a frame picture of a detected face into the deep learning based depsort model for operator tracking, and predicting an action track of an operator, the method includes:
respectively initializing the ID numbers of the operators of the target detection frames in the frame pictures of the detected human faces;
predicting the position of the target detection frame by adopting a Kalman filtering algorithm to obtain a state parameter of the target at the next moment;
and extracting the appearance characteristics of the object in the target detection frame based on a convolutional neural network, and matching and associating the target detection frame, the state parameters of the tracked target object at the next moment and the extracted appearance characteristics of the object based on a Hungarian cascade matching algorithm.
In a second aspect, the present invention provides an apparatus for automatically monitoring a process cycle, the apparatus comprising:
the image processing module is used for acquiring a video stream of a tunnel portal monitoring area and preprocessing the video stream to obtain a plurality of frames of pictures;
the dynamic event judging module is used for judging whether a dynamic event occurs in a plurality of frames of pictures, and if the dynamic event occurs, marking the frames of pictures with the dynamic event;
the target tracking prediction module is used for tracking the operators on the marked frame pictures based on a face detection model of opencv and a deepsort model based on deep learning;
the face information recognition module is used for carrying out face information recognition on the tracked operating personnel based on a face recognition model trained by an AM-softmax algorithm;
and the process starting and stopping recording module is used for recording the start or the end of the current process of the same work type when a plurality of operators of the same work type enter or leave the tunnel within a certain time range.
In one possible design, the apparatus further includes:
the time interval acquisition unit is used for acquiring the time intervals of the beginning or the ending of the working procedures represented by the entering and leaving of a plurality of working personnel of the same work type;
and the process cycle recording unit is used for recording the time interval as the time of one process cycle.
In a third aspect, the present invention provides an apparatus for automatically monitoring a process cycle, the apparatus comprising: a memory, a processor and a transceiver, which are in communication with each other in sequence, wherein the memory is used for storing a computer program, the transceiver is used for transmitting and receiving messages, and the processor is used for reading the computer program and executing the automatic monitoring method of the process cycle according to the first aspect.
In a fourth aspect, the invention provides an automatic monitoring system of process circulation, which comprises an image acquisition device and a face detection tracking and recognition subsystem, wherein the image acquisition device transmits acquired video streams of a tunnel portal monitoring area to the face detection tracking and recognition subsystem, the face detection tracking and recognition subsystem carries out tracking prediction on an operator through a face detection model based on opencv and a deeplearning-based deepsort model, and carries out face information recognition on the tracked operator through a face recognition model trained based on an AM-softmax algorithm.
Has the advantages that: after the video stream is collected, whether a dynamic event occurs in a plurality of frame pictures is judged, and if the dynamic event occurs, the frame pictures with the dynamic event are marked; then, carrying out tracking prediction on the marked frame picture by using a face detection model based on opencv and a depsort model based on deep learning; then, carrying out face information recognition on the tracked operator based on a face recognition model trained by an AM-softmax algorithm; and finally, when a plurality of operators of the same work type are recognized to enter or leave the tunnel within a certain time range, marking the start or the end of the current process of the work type. The invention can automatically acquire the images of tunnel operators entering and exiting the tunnel, automatically record the beginning or the end of the current process operation when a plurality of operators with the same work type entering and exiting the tunnel within a certain time range are identified by face detection tracking, and the condition that the operators enter and exit the tunnel is not required to be recorded manually in the whole process, thereby greatly improving the efficiency of process management and providing support for realizing the intellectualization of tunnel construction management.
Drawings
FIG. 1 is a flow chart of a method for automatically monitoring a process cycle provided by the present invention;
FIG. 2 is a schematic structural diagram of an automatic monitoring device for process cycles provided by the present invention;
FIG. 3 is a schematic structural diagram of an automatic monitoring device for another process cycle according to the present invention;
fig. 4 is a schematic structural diagram of an automatic monitoring system for process cycles provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments in the present description, belong to the protection scope of the present invention.
Examples
As shown in fig. 1, the method for automatically monitoring a process cycle provided in the first aspect of the embodiment of the present application includes, but is not limited to, the following steps S101 to S106.
S101, acquiring a video stream of a tunnel portal, and preprocessing the video stream to obtain a plurality of frame pictures;
in the step S101, a video stream of a tunnel portal monitoring area may be acquired through a conventional image capturing device, where the image capturing device includes, but is not limited to, a high definition camera and a camera, and other devices with a photographing function, such as a mobile phone and a tablet computer; the preprocessing of the video stream can be executed by a server, specifically, the server performs scale conversion on the acquired video stream in the tunnel portal monitoring area, so as to reduce the time consumption of the whole algorithm and make a basic preparation for the real-time performance of the algorithm, and the images after the scale conversion are preprocessed in sequence, so that a frame difference method is subsequently adopted to detect the dynamic objects in the frame pictures.
S102, judging whether a dynamic event occurs in a plurality of frames of pictures, and if so, marking the frames of pictures with the dynamic event;
as a possible design, in the step 102, the determining whether a dynamic event occurs in multiple frames of the picture includes:
and judging whether a dynamic event occurs in the frame pictures of the plurality of frames by adopting one algorithm of a two-frame difference frame method, a three-frame difference frame method, a video background subtraction method based on Gaussian mixture learning or a python background model subtraction method.
In step 102, the frame difference method (also called interframe difference method) is a method for obtaining a moving object contour by performing a difference operation on two adjacent frames in a video image sequence, and is suitable for a case where a plurality of moving objects exist and a camera moves. When abnormal object motion occurs in a monitored scene, a frame is obviously different from a frame, the two frames are subtracted to obtain an absolute value of the brightness difference of the two frames, whether the absolute value is greater than a threshold value or not is judged to analyze the motion characteristic of a video or an image sequence, and whether object motion exists in the image sequence or not is determined.
In a possible design, the determining whether a dynamic event occurs in multiple frames of the picture by using a two-frame difference frame method includes:
carrying out difference on pixel values at the same position of adjacent frames to obtain a difference image;
it should be noted that the adjacent frames may be two continuous frames of images selected from a video image sequence, and a difference image is obtained by subtracting pixel values at the same position of a previous frame of image and a current frame of image.
Preferably, before the step 1021, the method further includes: and performing median filtering pretreatment on the sequence image, and removing random noise of the image to reduce the complexity of subsequent operation and overcome the interference of the noise on the image processing result.
And carrying out binarization on the differential image, wherein when the pixel value change of the differential image is greater than a preset threshold value, a dynamic event occurs in the adjacent frame.
Based on the above disclosure, the occurrence of the dynamic event is judged by the two-frame difference method, so that the calculation is simple, the calculation speed is high, and the occupied server resources are less.
In a possible design, the determining whether the dynamic event occurs in the frames of the multiple frames of pictures by using the three-frame difference method is based on the two-frame difference method, and an and operation is taken from two frame difference pictures of the adjacent three frames of pictures to obtain a difference image, so that the detected moving object is more accurate.
In one possible design, the principle of determining whether a dynamic event occurs in multiple frames of the frame picture based on the python background model subtraction method is as follows: different from the frame difference method, the moving object is determined by the pixel difference between the front frame picture and the rear frame picture, the background model subtraction method hopes to obtain the outline of the moving object by establishing a background model and then subtracting each frame picture from the background model. The method specifically comprises the following steps: accumulating each frame of picture through the weight to reconstruct a video background; calculating the absolute value difference between a current frame picture and a reconstructed background picture in a video stream to obtain a difference image; binarizing the difference image; removing noise by using median filtering and expansion corrosion, and removing isolated noise points or thinner noise lines; then, the contour of the moving target is found out, and the moving object is captured.
In one possible design, the principle of determining whether a dynamic event occurs in multiple frames of the frame pictures by the gaussian mixture learning-based video background subtraction includes:
for each point in the video stream (updated over time), a K gaussian model is used to model, the distribution of which by default conforms to a gaussian mixture distribution: for background points, the pixel values p are distributed around the mean value of the gaussian model (| p- μ | < ═ 2.5 σ); for the motion target point, its pixel value p deviates from the mean value (| p- μ | >2.5 σ); meanwhile, a moving object may become a new background (for example, a vehicle is considered to be a moving object before stopping, and is considered to be a new background after stopping), so we need to add a partial weight α × p of the moving object to the mean value of the background distribution, where α is the learning rate; for a Gaussian model with pixel values continuously distributed around a mean value, the default can better accord with expected distribution, the mean value and the standard deviation are updated, the standard deviation becomes smaller, and the weight is increased in the Gaussian mixture model; for a Gaussian model with non-uniform pixel value distribution, reducing the weight of the Gaussian model, and considering that the distribution possibly needs to be updated; if a certain point does not accord with the distribution of the K models, the mixed model is considered to be incapable of well describing the background at the moment, so that the distribution with the minimum weight needs to be deleted, and new distribution is added according to the mean value of the pixel points at the moment.
103, carrying out tracking prediction on the marked frame picture by using a face detection model based on opencv and a depsort model based on deep learning;
step 1031, inputting the marked frame picture into the face detection model based on opencv to perform rapid face detection;
and step 1032, inputting the frame picture with the detected face into the deep learning-based depsort model for tracking the operator, predicting the action track of the operator, updating the tracking result and outputting the tracking result.
Inputting the frame picture of the detected face into the deep learning-based deepsort model for tracking the operator, and predicting the action track of the operator, wherein the action track comprises the following steps:
in one possible design, respectively initializing the ID numbers of the operators of the target detection frames in the frame pictures of the detected human faces;
predicting the position of the target detection frame by adopting a Kalman filtering algorithm to obtain a state parameter of the target at the next moment;
and extracting the appearance characteristics of the object in the target detection frame based on a convolutional neural network, and matching and associating the target detection frame, the state parameters of the tracked target object at the next moment and the extracted appearance characteristics of the object based on a Hungarian cascade matching algorithm.
In this embodiment of the present application, the kalman filtering algorithm mainly includes: a prediction phase and an update phase. In the prediction stage, the algorithm firstly inputs the position state of the object at the last moment K-1 into a predefined physical model, and obtains the position state predicted by the physical model at the moment K by the physical model after calculation of the physical model. In the updating stage, after the predicted target physical state is obtained, weighting calculation is carried out on the predicted physical variable and the observed value of the object at the current moment by using a weighting variable predefined by the algorithm, the final correction value of the current K moment corrected by the Kalman filtering algorithm is finally obtained, and the internal parameters of the Kalman filtering algorithm are updated after the correction is finished.
In the embodiment of the present application, the working principle of predicting the target position by the kalman filter algorithm is as follows: the predicted values use the coordinates x, y, w, h of the target in the picture, i.e. its observations. The real values of the target state are x, y, w, h, Vx, Vy, Vw, Vh of the target bbox. The used Kalman theory is that after a target true value is obtained by a classical Kalman theory, a physical model of the Kalman theory is used for predicting the next state of the corrected value, namely, the target state value of the K moment after being corrected by the classical Kalman filtering is used, then a state transfer equation is used for carrying out state transfer to the next moment, the predicted value of the target at the K +1 moment based on the true value is obtained, and then an observation equation is used for solving to obtain the observed state value, so that the predicted state of the target at the K +1 moment can be obtained.
In the embodiment of the application, a Hungarian matching algorithm is used for comparing the characteristics of the image targets in the two frames aiming at the prediction frame of the frame and the detection frame of the next frame, if the characteristic distance is smaller than a preset threshold distance, the frames in the two frames are considered to correspond to the same ID number, namely, one-time target association is completed, and if the characteristic distance is larger than the preset threshold, the two ID numbers at the time are considered to be inconsistent, and the next rectangular frame is matched. If the matching is finished, the IOU matching is used for the current two frames, if the matching is still finished, the state of the target is updated by using a Kalman updating formula, and the updated state is used for predicting the motion state of the target of the next frame; if the matching is not complete (i.e. the CNN feature without the trace box matches the CNN feature of the current detection box), it is considered as a new track and given a new ID number.
S104, carrying out face information recognition on the tracked operating personnel based on a face recognition model trained by an AM-softmax algorithm;
it should be noted that the face information includes a name, an identification number, a work type, a time of entering or leaving a tunnel, and a corresponding current process of the operator, and is not limited specifically.
And S105, when a plurality of operators of the same work type are recognized to enter or leave the tunnel within a certain time range, recording the start or the end of the current process of the work type.
In one possible design, the method further includes:
acquiring time intervals of a plurality of operating personnel of the same work type entering the tunnel and leaving the tunnel;
the time interval is recorded as the time of one process cycle.
In a second aspect, as shown in fig. 2, the present invention provides an apparatus for automatically monitoring a process cycle, the apparatus comprising:
the image processing module is used for acquiring a video stream of the tunnel portal and preprocessing the video stream to obtain a multi-frame picture;
the dynamic event judging module is used for judging whether a dynamic event occurs in a plurality of frames of pictures, and if the dynamic event occurs, marking the frames of pictures with the dynamic event;
the target tracking prediction module is used for tracking the operators on the marked frame pictures based on a face detection model of opencv and a deepsort model based on deep learning;
the face information recognition module is used for carrying out face information recognition on the tracked operating personnel based on a face recognition model trained by an AM-softmax algorithm;
and the process starting and stopping recording module is used for recording the start or the end of the current process of the same work type when a plurality of operators of the same work type enter or leave the tunnel within a certain time range.
In one possible design, the apparatus further includes:
the time interval acquisition unit is used for acquiring the time intervals of the beginning or the ending of the working procedures represented by the entering and leaving of a plurality of working personnel of the same work type;
and the process cycle recording unit is used for recording the time interval as the time of one process cycle.
In a possible design, the dynamic event determining module determines whether a dynamic event occurs in multiple frames of the frame picture by using one of a two-frame difference frame method, a three-frame difference frame method, a video background subtraction method based on gaussian mixture learning, or a python background model subtraction method.
For the working process, the working details and the technical effects of the foregoing apparatus provided in the second aspect of this embodiment, reference may be made to the first aspect or any one of the automatic monitoring methods that may be designed for the process cycle in the first aspect, and details are not described herein again.
In a third aspect, as shown in fig. 3, the present invention provides an apparatus for automatically monitoring a process cycle, the apparatus comprising: a memory, a processor and a transceiver, which are in communication with each other in sequence, wherein the memory is used for storing a computer program, the transceiver is used for transmitting and receiving messages, and the processor is used for reading the computer program and executing the automatic monitoring method of the process cycle according to the first aspect.
For the working process, the working details and the technical effects of the foregoing apparatus provided in the second aspect of this embodiment, reference may be made to the first aspect or any one of the automatic monitoring methods that may be designed for the process cycle in the first aspect, and details are not described herein again.
As shown in fig. 4, the present invention provides an automatic monitoring system with a cyclic process, which includes an image acquisition device and a face detection tracking recognition subsystem, wherein the image acquisition device transmits acquired video streams of a tunnel portal monitoring area to the face detection tracking recognition system, and the face detection tracking recognition system performs tracking prediction on an operator through an opencv-based face detection model and a deep learning-based deepsort model, and performs face information recognition on the tracked operator through an AM-softmax algorithm-trained face recognition model.
For the working process, the working details and the technical effects of the foregoing system provided in the fourth aspect of this embodiment, reference may be made to the first aspect or any one of the automatic monitoring methods that may be designed for the process cycle in the first aspect, and details are not described herein again.
The embodiment of the application can automatically acquire the images of tunnel operators passing in and out of the tunnel, when the human face detection, tracking and identification are carried out on a plurality of operators having the same work type within a certain time range, the operation of the next process is considered to be started or ended and recorded, the time, the work type, the number of people and the like of the operators passing in and out of the tunnel are monitored in the whole process, and the data of the process circulation can be acquired, so that the condition that the operators pass in and out of the tunnel is not required to be manually recorded, the efficiency of process management is greatly improved, and support is provided for realizing the intellectualization of tunnel construction management.
Finally, it should be noted that: the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for automatically monitoring a process cycle, the method comprising:
acquiring a video stream of a tunnel portal, and preprocessing the video stream to obtain a plurality of frame pictures;
judging whether a dynamic event occurs in a plurality of frames of pictures, and if so, marking the frames of pictures with the dynamic event;
carrying out tracking prediction on the marked frame picture by using a face detection model based on opencv and a depsort model based on deep learning;
carrying out face information recognition on the tracked operator based on a face recognition model trained by an AM-softmax algorithm;
when a plurality of operators of the same work type are recognized to enter or leave the tunnel within a certain time range, the current process start or end of the work type is recorded.
2. The method for automatically monitoring a process cycle according to claim 1, further comprising:
acquiring time intervals of a plurality of operating personnel of the same work type entering the tunnel and leaving the tunnel;
the time interval is recorded as the time of one process cycle.
3. The method of claim 1, wherein the determining whether a dynamic event occurs in a plurality of frames of the pictures comprises:
and judging whether a dynamic event occurs in the frame pictures of the plurality of frames by adopting one algorithm of a two-frame difference frame method, a three-frame difference frame method, a video background subtraction method based on Gaussian mixture learning or a python background model subtraction method.
4. The method of claim 3, wherein said determining whether a dynamic event occurs in a plurality of frames of said frame picture by using a two-frame difference frame method comprises:
carrying out difference on pixel values at the same position of adjacent frames to obtain a difference image;
and carrying out binarization on the differential image, wherein when the pixel value change of the differential image is greater than a preset threshold value, a dynamic event occurs in the adjacent frame.
5. The method for automatically monitoring process cycle according to claim 1, wherein the performing of operator tracking prediction on the marked frame picture by using the opencv-based face detection model and the deep learning-based deepsort model comprises:
inputting the marked frame picture into the face detection model based on opencv for rapid face detection;
and inputting the frame picture of the detected face into the deep learning-based deepsort model for tracking the operator, predicting the action track of the operator, updating the tracking result and outputting the tracking result.
6. The method according to claim 5, wherein the step of inputting the frame picture with the detected face into the deep learning-based deepsort model for operator tracking to predict the action trajectory of the operator comprises:
respectively initializing the ID numbers of the operators of the target detection frames in the frame pictures of the detected human faces;
predicting the position of the target detection frame by adopting a Kalman filtering algorithm to obtain a state parameter of the target at the next moment;
and extracting the appearance characteristics of the object in the target detection frame based on a convolutional neural network, and matching and associating the target detection frame, the state parameters of the tracked target object at the next moment and the extracted appearance characteristics of the object based on a Hungarian cascade matching algorithm.
7. An apparatus for automatically monitoring a process cycle, the apparatus comprising:
the image processing module is used for acquiring a video stream of the tunnel portal and preprocessing the video stream to obtain a multi-frame picture;
the dynamic event judging module is used for judging whether a dynamic event occurs in a plurality of frames of pictures, and if the dynamic event occurs, marking the frames of pictures with the dynamic event;
the target tracking prediction module is used for tracking the operators on the marked frame pictures based on a face detection model of opencv and a deepsort model based on deep learning;
the face information recognition module is used for carrying out face information recognition on the tracked operating personnel based on a face recognition model trained by an AM-softmax algorithm;
and the process starting and stopping recording module is used for recording the start or the end of the current process of the same work type when a plurality of operators of the same work type enter or leave the tunnel within a certain time range.
8. The automatic process cycle monitoring device of claim 7, further comprising:
the time interval acquisition unit is used for acquiring the time intervals of the beginning or the ending of the working procedures represented by the entering and leaving of a plurality of working personnel of the same work type;
and the process cycle recording unit is used for recording the time interval as the time of one process cycle.
9. An apparatus for automatically monitoring a process cycle, the apparatus comprising: a memory, a processor and a transceiver communicatively coupled in sequence, wherein the memory is configured to store a computer program, the transceiver is configured to transmit and receive messages, and the processor is configured to read the computer program and perform the method for automatic monitoring of a process cycle according to any of claims 1-6.
10. The automatic monitoring system of process circulation is characterized by comprising an image acquisition device and a face detection, tracking and recognition subsystem, wherein the image acquisition device sends acquired video streams of a tunnel portal monitoring area to the face detection, tracking and recognition subsystem, the face detection, tracking and recognition subsystem carries out tracking and prediction on operators through a face detection model based on opencv and a deppsort model based on deep learning, and carries out face information recognition on the tracked operators through a face recognition model based on AM-softmax algorithm training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476850.9A CN113111847A (en) | 2021-04-29 | 2021-04-29 | Automatic monitoring method, device and system for process circulation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110476850.9A CN113111847A (en) | 2021-04-29 | 2021-04-29 | Automatic monitoring method, device and system for process circulation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113111847A true CN113111847A (en) | 2021-07-13 |
Family
ID=76720536
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110476850.9A Pending CN113111847A (en) | 2021-04-29 | 2021-04-29 | Automatic monitoring method, device and system for process circulation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113111847A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115657580A (en) * | 2022-12-14 | 2023-01-31 | 北京交科公路勘察设计研究院有限公司 | Tunnel fire pool monitoring method and system based on combined algorithm |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107871345A (en) * | 2017-09-18 | 2018-04-03 | 深圳市盛路物联通讯技术有限公司 | Information processing method and related product |
| CN108363997A (en) * | 2018-03-20 | 2018-08-03 | 南京云思创智信息科技有限公司 | It is a kind of in video to the method for real time tracking of particular person |
| CN110569809A (en) * | 2019-09-11 | 2019-12-13 | 淄博矿业集团有限责任公司 | coal mine dynamic face recognition attendance checking method and system based on deep learning |
| CN111353338A (en) * | 2018-12-21 | 2020-06-30 | 国家电网有限公司客户服务中心 | Energy efficiency improvement method based on business hall video monitoring |
| CN111460985A (en) * | 2020-03-30 | 2020-07-28 | 华中科技大学 | On-site worker track statistical method and system based on cross-camera human body matching |
| CN111460884A (en) * | 2020-02-09 | 2020-07-28 | 天津博宜特科技有限公司 | Multi-face recognition method based on human body tracking |
| CN111698464A (en) * | 2020-04-29 | 2020-09-22 | 泉州禾逸电子有限公司 | Intelligent camera following monitoring alarm method and device |
| CN111988571A (en) * | 2020-08-26 | 2020-11-24 | 杭州海康威视数字技术股份有限公司 | Method and device for detecting access information |
| CN112561387A (en) * | 2020-12-24 | 2021-03-26 | 中交第四航务工程局有限公司 | Work efficiency analysis method and system based on visualization and personnel management |
| CN112560745A (en) * | 2020-12-23 | 2021-03-26 | 南方电网电力科技股份有限公司 | Method for discriminating personnel on electric power operation site and related device |
| CN112633749A (en) * | 2020-12-31 | 2021-04-09 | 河南橡树智能科技有限公司 | Staff working time counting method, system and medium based on face recognition |
-
2021
- 2021-04-29 CN CN202110476850.9A patent/CN113111847A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107871345A (en) * | 2017-09-18 | 2018-04-03 | 深圳市盛路物联通讯技术有限公司 | Information processing method and related product |
| CN108363997A (en) * | 2018-03-20 | 2018-08-03 | 南京云思创智信息科技有限公司 | It is a kind of in video to the method for real time tracking of particular person |
| CN111353338A (en) * | 2018-12-21 | 2020-06-30 | 国家电网有限公司客户服务中心 | Energy efficiency improvement method based on business hall video monitoring |
| CN110569809A (en) * | 2019-09-11 | 2019-12-13 | 淄博矿业集团有限责任公司 | coal mine dynamic face recognition attendance checking method and system based on deep learning |
| CN111460884A (en) * | 2020-02-09 | 2020-07-28 | 天津博宜特科技有限公司 | Multi-face recognition method based on human body tracking |
| CN111460985A (en) * | 2020-03-30 | 2020-07-28 | 华中科技大学 | On-site worker track statistical method and system based on cross-camera human body matching |
| CN111698464A (en) * | 2020-04-29 | 2020-09-22 | 泉州禾逸电子有限公司 | Intelligent camera following monitoring alarm method and device |
| CN111988571A (en) * | 2020-08-26 | 2020-11-24 | 杭州海康威视数字技术股份有限公司 | Method and device for detecting access information |
| CN112560745A (en) * | 2020-12-23 | 2021-03-26 | 南方电网电力科技股份有限公司 | Method for discriminating personnel on electric power operation site and related device |
| CN112561387A (en) * | 2020-12-24 | 2021-03-26 | 中交第四航务工程局有限公司 | Work efficiency analysis method and system based on visualization and personnel management |
| CN112633749A (en) * | 2020-12-31 | 2021-04-09 | 河南橡树智能科技有限公司 | Staff working time counting method, system and medium based on face recognition |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115657580A (en) * | 2022-12-14 | 2023-01-31 | 北京交科公路勘察设计研究院有限公司 | Tunnel fire pool monitoring method and system based on combined algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110334569B (en) | Passenger flow volume in-out identification method, device, equipment and storage medium | |
| CN109145708B (en) | Pedestrian flow statistical method based on RGB and D information fusion | |
| CN110633612B (en) | Monitoring method and system for inspection robot | |
| CN109298785A (en) | A kind of man-machine joint control system and method for monitoring device | |
| WO2021139049A1 (en) | Detection method, detection apparatus, monitoring device, and computer readable storage medium | |
| CN112528861B (en) | Foreign matter detection method and device applied to ballast bed in railway tunnel | |
| CN111161206A (en) | Image capturing method, monitoring camera and monitoring system | |
| CN113947731B (en) | Foreign matter identification method and system based on contact net safety inspection | |
| CN104966304A (en) | Kalman filtering and nonparametric background model-based multi-target detection tracking method | |
| CN111898581A (en) | Animal detection method, device, electronic equipment and readable storage medium | |
| CN111723773A (en) | Remnant detection method, device, electronic equipment and readable storage medium | |
| CN112070053B (en) | Background image self-updating method, device, equipment and storage medium | |
| CN113781526A (en) | Domestic animal count identification system | |
| CN110334568B (en) | Track generation and monitoring method, device, equipment and storage medium | |
| CN110175553B (en) | Method and device for establishing feature library based on gait recognition and face recognition | |
| CN109615641B (en) | Multi-target pedestrian tracking system and tracking method based on KCF algorithm | |
| CN113111847A (en) | Automatic monitoring method, device and system for process circulation | |
| CN113920585A (en) | Behavior recognition method and device, equipment and storage medium | |
| CN114120165A (en) | Gun and ball linked target tracking method and device, electronic device and storage medium | |
| CN111860392B (en) | Thermodynamic diagram statistical method based on target detection and foreground detection | |
| CN106803937B (en) | Double-camera video monitoring method, system and monitoring device with text log | |
| CN112417978A (en) | Vision-based method and device for detecting foreign matters between train and shield door | |
| CN116012949B (en) | People flow statistics and identification method and system under complex scene | |
| CN112561957A (en) | State tracking method and device for target object | |
| CN111784750A (en) | Method, device and equipment for tracking moving object in video image and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210713 |
|
| RJ01 | Rejection of invention patent application after publication |