CN111259718A - Escalator retention detection method and system based on Gaussian mixture model - Google Patents

Escalator retention detection method and system based on Gaussian mixture model Download PDF

Info

Publication number
CN111259718A
CN111259718A CN201911017881.7A CN201911017881A CN111259718A CN 111259718 A CN111259718 A CN 111259718A CN 201911017881 A CN201911017881 A CN 201911017881A CN 111259718 A CN111259718 A CN 111259718A
Authority
CN
China
Prior art keywords
retention
image
escalator
detection
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911017881.7A
Other languages
Chinese (zh)
Other versions
CN111259718B (en
Inventor
胡芬
陈南西
楼阳冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou AIMS Intelligent Technology Co Ltd
Original Assignee
Hangzhou AIMS Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou AIMS Intelligent Technology Co Ltd filed Critical Hangzhou AIMS Intelligent Technology Co Ltd
Priority to CN201911017881.7A priority Critical patent/CN111259718B/en
Publication of CN111259718A publication Critical patent/CN111259718A/en
Application granted granted Critical
Publication of CN111259718B publication Critical patent/CN111259718B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06M—COUNTING MECHANISMS; COUNTING OF OBJECTS NOT OTHERWISE PROVIDED FOR
    • G06M1/00—Design features of general application
    • G06M1/27—Design features of general application for representing the result of count in the form of electric signals, e.g. by sensing markings on the counter drum
    • G06M1/272—Design features of general application for representing the result of count in the form of electric signals, e.g. by sensing markings on the counter drum using photoelectric means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
    • Y02B50/00—Energy efficient technologies in elevators, escalators and moving walkways, e.g. energy saving or recuperation technologies

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of escalator safety monitoring, and discloses a method and a system for detecting escalator detention based on a Gaussian mixture model, which comprises the following steps: A) extracting pedestrian information from the video image by using a deep learning network; B) constructing a classification model by using a machine learning method, and classifying and screening the pedestrian information detected in real time; C) constructing a Gaussian mixture model, and acquiring a retention target image of the current frame; D) preprocessing the retention target image to obtain an image in a retention detection area at an entrance and an exit of the escalator; E) obtaining a candidate detaining target contour by adopting an edge detection algorithm, and judging the area of a candidate detaining frame; F) and (5) performing overlapping judgment on the candidate retention frames, and performing retention alarm. The invention detects the retention of articles and passengers at the entrance and exit of the escalator and gives an alarm in real time, thereby achieving the purpose of monitoring the retention behavior of the entrance and exit area of the escalator and having high retention and detection accuracy.

Description

Escalator retention detection method and system based on Gaussian mixture model
Technical Field
The invention relates to the technical field of escalator safety monitoring and image processing, in particular to an escalator retention detection method and system based on a Gaussian mixture model.
Background
Along with continuous pursuit of people to swift life style, the use of staircase is more and more common, and some safety issue, the smooth and easy operation problem of staircase arouse people's concern more and more, especially staircase mouth position, including whether the staircase mouth passenger flow blocks up, whether have the passenger to fall down and whether have major possession article to be detained etc.. At present, the common practice is to arrange workers at the position of an escalator entrance for nursing and to perform corresponding treatment measures according to the conditions on site. Personnel nursing not only has great labor cost, but also is difficult to monitor the passenger flow condition and the behavior of the escalator in real time in the using process, so that the countermeasure is difficult to be taken at the first time. Therefore, it is necessary to develop a more intelligent monitoring method for escalator operation, and many escalator manufacturers are actively paying attention to and exploring this field.
For example, a "detection system and detection method for automatically detecting passengers staying in a school bus" disclosed in chinese patent literature, which is under the publication No. CN 109484292 a, links a data processor with a school bus signal, and detects passengers getting in and out of a carriage and passengers getting in and out of a seat by first and second opposite emission type photoelectric sensors, respectively; the data processor judges whether the passengers get on or off the vehicle according to the received signal sequence and counts the total number of the passengers in real time; if the number of passengers in the vehicle is not zero when the terminal is reached, the data processor outputs an alarm signal, if the number of passengers is zero, the data processor receives a detection signal of the thermal infrared human body sensor to recheck passengers staying in the vehicle, and if the number of passengers does not exist, the data processor confirms that no passengers stay in the vehicle. The invention adopts the radio photoelectric sensor, only can detect passengers, but cannot detect the retained articles.
Disclosure of Invention
The invention provides a method and a system for detecting retention of an escalator based on a Gaussian mixture model, aiming at solving the problem of intelligentization of escalator operation monitoring. The invention utilizes a machine learning method to construct a classification model, filters out information of non-retention substances, thereby reducing the condition of false detection and improving the accuracy of retention substance detection.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for detecting retention of an escalator based on a Gaussian mixture model comprises the following steps:
A) acquiring a monitoring video image of an escalator region through a camera, extracting pedestrian information from the video image by using a deep learning network, collecting false detection pedestrian information and real pedestrian information, recording the false detection pedestrian information as a negative sample, and recording the real pedestrian information as a positive sample;
B) a classification model is established by using a machine learning method, a negative sample and a positive sample are used for training the classification model, the trained classification model is used for classifying and screening pedestrian information detected in real time, real pedestrian information is reserved, and false detection pedestrian information is removed; judging whether pedestrian information exists in the range of the escalator entrance area, if so, entering the step C), and if not, repeating the step A);
C) constructing a Gaussian mixture model, obtaining a detection area background, obtaining a background difference image by using background subtraction, obtaining a frame difference method image by using an inter-frame difference method, and performing XOR processing on the background difference image and the frame difference method image to obtain a retention target image of a current frame;
D) preprocessing the retention target image, intercepting the preprocessed retention target image to obtain an image in a retention detection area at an entrance and an exit of the escalator;
E) adopting an edge detection algorithm to obtain candidate detained target contours in the detained detection region, setting an area threshold value, obtaining the minimum circumscribed rectangle of each candidate detained target contour, taking the minimum circumscribed rectangle of each candidate detained target contour with the area larger than the area threshold value as a candidate detained frame, setting a detained counter, and setting the detained counter to be 1;
F) setting an overlap threshold and a counter threshold, judging whether the overlap ratio of the candidate retention frame of the previous frame image to the candidate retention frame of the current frame image is greater than the overlap threshold, if not, setting a retention counter to be 1, and if so, adding 1 to the retention counter;
and judging whether the retention counter is larger than the counter threshold value, if so, giving an alarm of retention of articles or personnel at the entrance and exit of the escalator, and if not, repeating the step.
The pedestrian information is extracted from the video image by using the deep learning network, and then the classification model is constructed by using the machine learning method, so that the slowly-changing information of the non-retention is filtered, and the condition of false detection is reduced. The Gaussian mixture model uses a plurality of Gaussian models to represent the characteristics of each pixel point in the video image, the Gaussian mixture model is updated in real time after a new frame of video image is obtained, each pixel point in the current video image is matched with the Gaussian mixture model, if the matching is successful, the point is judged to be a background point, and if not, the point is a foreground point. And E), acquiring a candidate detained target contour in the detained detection area by adopting an edge detection algorithm, and judging and analyzing the minimum circumscribed rectangle area of the candidate detained target contour so as to select the large article detained product. And F) continuously updating by comparing the candidate detention frame of the previous frame image with the candidate detention frame of the current frame image in real time, judging whether the detained article stays in the detention area, if so, continuously accumulating a detention counter along with the time change, and when the value of the detention counter exceeds the threshold value of the counter, namely, judging that the detention time of the detained article is longer, giving a detention alarm.
The escalator region surveillance video images include escalator entrance images and/or escalator exit images.
The deep learning network in the step A) is any one of OpenPose, SSD, Yolo and Faster-CNN.
Further, the machine learning method is a support vector machine.
The support vector machine correctly classifies the sample set by searching a segmentation hyperplane, and the classification effect is better than that of a neural network under the condition of less sample sets.
Further, the Gaussian mixture model is constructed in the step C) by using 500 frames of video images in an unmanned state.
By constructing a Gaussian mixture model, a model that can characterize the "background" is constructed. 500 frames of video images are acquired under the condition that no moving target exists, namely, no person exists, and a Gaussian mixture model is constructed by using the 500 frames of video images, so that the influence of various factors such as illumination change, target movement and the like is eliminated.
Further, in step C), the learning rate parameter in the gaussian mixture model is set to be a low learning rate, and the background difference image D is obtained by background subtraction at the low learning raten(x, y) setting a threshold T according to the formula
Figure BDA0002246272520000031
One by one background difference image DnAnd (x, y) performing binarization processing on each pixel point to obtain a background differential binarization image, wherein the background differential binarization image comprises a real-time moving target and a static target which is more than the background image.
The Gaussian mixture model is mainly determined by two parameters of variance and mean, the learning of the mean and the variance is carried out, and different learning mechanisms are adopted to directly influence the stability, the accuracy and the convergence of the model. Because the invention models the background extraction of the moving object, the variance and mean parameters in the Gaussian mixture model need to be updated in real time. In order to improve the learning capability of the model, updating the mean value and the variance by adopting different learning rates, constructing a detection area background by using a Gaussian mixture model, setting an extremely low learning rate, carrying out differential operation on a current frame image and a background image by using background subtraction to obtain a background differential image, extracting a moving target from the differential image, and ensuring that the system can adapt to slow illumination change. The background subtraction operation process firstly uses a Gaussian mixture model to establish a background image frame B, and records the current image frame as fnSubtracting the gray values of the corresponding pixel points of the background frame and the current frame and taking the absolute value to obtain a differential image Dn(x, y). Setting a threshold value T according to the formula
Figure BDA0002246272520000032
Carrying out binarization processing on the pixel points one by one to obtain a binarized image, wherein the gray value is255 points are foreground points, the foreground points comprise real-time moving objects and static objects which are more than background images, and the points with the gray value of 0 are background points. And then, performing connectivity analysis on the binary image to finally obtain an image containing a complete target.
Further, the learning rate parameter in the Gaussian mixture model is set to be high learning rate in the step C), and the frame difference method image E is obtained by using the inter-frame difference method under the high learning raten(x, y) setting a threshold T2According to the formula
Figure BDA0002246272520000033
Frame-by-frame difference image EnAnd (x, y) carrying out binarization processing on each pixel point to obtain a frame difference method binarization image, wherein the frame difference method binarization image contains a real-time moving target.
The video sequence collected by the camera has the characteristic of continuity. If there are no moving objects in the scene, the change in successive frames is weak, and if there are moving objects, there will be significant changes from frame to frame. As objects in a scene move, the position of the object's imagery in different image frames may also vary. The method comprises the steps of carrying out differential operation on two or three continuous frames of images in time by using an interframe differential method, subtracting corresponding pixel points of different frames, judging the absolute value of gray difference, and judging a moving target when the absolute value exceeds a certain threshold value, thereby realizing the detection function of the target.
Further, the retention target image is preprocessed in step D), wherein the preprocessing is corrosion and/or expansion.
In the preprocessing, a method of erosion and/or expansion in morphological image processing is used to extract image components useful in the shape of the image region from the retained target image.
Further, the edge detection algorithm in the step E) adopts any one of a Sobel operator, a Laplacian operator and a Canny operator.
The edge detection method for image processing greatly reduces the data size, eliminates information which can be considered irrelevant, and retains important structural attributes of the image. The method adopts any one edge detection algorithm of a Sobel operator, a Laplacian operator and a Canny operator to obtain the candidate detained target contour in the detained detection area.
A Gaussian mixture model staircase retention detection system comprises an acquisition module, a pedestrian identification module, a pedestrian analysis module, a retention detection module, a preprocessing module, a retention analysis module and a judgment module;
the acquisition module is used for acquiring a monitoring video image of the escalator area;
the pedestrian identification module is used for extracting pedestrian information from the current frame image by utilizing the deep learning network model;
the pedestrian analysis module is used for collecting false detection and real pedestrian information, printing corresponding labels, constructing normal pedestrians and positive and negative samples of false detection data, constructing a classification model by using a machine learning method, classifying and screening real-time pedestrian detection information, retaining correct pedestrian information and removing false detection pedestrian information;
the retention detection module is used for judging whether pedestrian information exists in the range of the escalator entrance area, constructing a detection area background, acquiring a background difference image and a frame difference method image of the current frame image, and then performing XOR processing on the background difference image and the frame difference method image to acquire a retention target image;
the preprocessing module is used for preprocessing the retention target image to eliminate noise and intercepting an image of the image in a retention detection area at an entrance and an exit of the escalator;
the retention analysis module is used for acquiring candidate retention target outlines in the retention detection area by adopting an edge detection algorithm, setting an area threshold value, acquiring a minimum circumscribed rectangle of each candidate retention target outline, taking the minimum circumscribed rectangle of each candidate retention target outline with the area larger than the area threshold value as a candidate retention frame, setting a retention counter to be 1, setting an overlap threshold value, judging whether the overlap ratio of the candidate retention frame of the previous frame image to the candidate retention frame of the current frame image is larger than the overlap threshold value, and if the overlap ratio is smaller than the overlap threshold value, setting the retention counter to be 1; if so, adding 1 to the retention counter;
and the judging module is used for judging whether the retention counter is larger than the counter threshold value, if so, alarming retention of articles or personnel at the entrance and exit of the escalator, and alarming by sound on site of the system.
Therefore, the invention has the following beneficial effects: the classification model for pedestrian identification is constructed by adopting a machine learning method, the classification model is constructed by utilizing the machine learning method, and information of non-retentate is filtered, so that the condition of false detection is reduced, the accuracy of retentate detection is improved, in addition, a Gaussian mixture model is established to identify and judge the retention behavior, the retention of articles and passengers at the exit and entrance of the escalator is detected and alarmed in real time, the purpose of monitoring the retention behavior of the exit and entrance area of the escalator is achieved, and the retention detection accuracy is high.
Drawings
Fig. 1 is a flow chart of a first embodiment of the present invention.
Fig. 2 is a schematic structural view of an escalator retention detection system according to a first embodiment of the invention.
Fig. 3 is an original video image of an entrance/exit of an escalator in an unmanned state according to a first embodiment of the present invention.
Fig. 4 is a diagram showing the effect of the identification of the escalator entrance and exit detention detection area in the unmanned state according to the first embodiment of the present invention.
Fig. 5 is a diagram illustrating an effect of background subtraction processing according to the first embodiment of the present invention.
Fig. 6 is a diagram illustrating the processing effect of the frame difference method according to the first embodiment of the present invention.
Fig. 7 is a diagram illustrating an effect of the xor processing according to the first embodiment of the present invention.
FIG. 8 is a graph showing the effect of the corrosion expansion treatment according to the first embodiment of the present invention.
Fig. 9 is a diagram illustrating the effect of intercepting a stagnant area in the first embodiment of the present invention.
Fig. 10 is a diagram showing the effect of the warning of the staying of person according to the first embodiment of the present invention.
Fig. 11 is a diagram showing the effect of the article retention alarm according to the first embodiment of the present invention.
100 acquisition module, 200 pedestrian identification module, 300 pedestrian analysis module, 400 retention detection module, 500 preprocessing module, 600 retention analysis module, 700 judgment module.
Detailed Description
The invention is further described with reference to the following detailed description and accompanying drawings.
In a first embodiment, a method for detecting retention of an escalator based on a gaussian mixture model, as shown in fig. 1, includes the steps of: A) acquiring a monitoring video image of an escalator region through a camera, extracting pedestrian information from the video image by utilizing an OpenPose deep learning network, collecting false detection pedestrian information and real pedestrian information, recording the false detection pedestrian information as a negative sample, and recording the real pedestrian information as a positive sample;
B) constructing a classification model by adopting a support vector machine method, training the classification model by utilizing a negative sample and a positive sample, classifying and screening pedestrian information detected in real time through the trained classification model, retaining real pedestrian information, and removing false detection pedestrian information; judging whether pedestrian information exists in the range of the escalator entrance area, if so, entering the step C), otherwise, repeating the step A), and as shown in fig. 4, obtaining an identification effect diagram of the escalator entrance and exit retention detection area in an unmanned state.
C) The Gaussian mixture model is constructed by using 500 frames of video images in an unmanned state, and as shown in FIG. 3, the Gaussian mixture model is an original video image of an entrance and an exit of the escalator in the unmanned state. Obtaining the background of the detection area through a Gaussian mixture model, setting the learning rate parameter in the Gaussian mixture model as a low learning rate, and obtaining a background differential image D by background subtraction under the low learning raten(x, y) is a background difference processing effect diagram as shown in fig. 5. Setting a threshold value T according to the formula
Figure BDA0002246272520000061
One by one background difference image DnAnd (x, y) performing binarization processing on each pixel point to obtain a background differential binarization image, wherein the background differential binarization image comprises a real-time moving target and a static target which is more than the background image.
Setting the learning rate parameter in the Gaussian mixture model as a high learning rate in the step C), and acquiring a frame difference method image E by using an inter-frame difference method under the high learning raten(x, y) as shown in FIG. 6, which is a graph of the effect of frame difference processing. Setting a threshold T2According to the formula
Figure BDA0002246272520000062
Frame-by-frame difference image EnAnd (x, y) carrying out binarization processing on each pixel point to obtain a frame difference method binarization image, wherein the frame difference method binarization image contains a real-time moving target.
The background difference image and the frame difference image are subjected to exclusive or processing, and as shown in fig. 7, the background difference image and the frame difference image are a frame difference processing effect image. And acquiring a retention target image of the current frame, namely acquiring more static targets than the background image.
D) The retained target image is preprocessed, and the preprocessing includes erosion and expansion, as shown in fig. 8, which is an effect graph of erosion and expansion processing. And intercepting the preprocessed retention target image to obtain an image in the retention detection area at the entrance and exit of the escalator, wherein the image is an effect image of the intercepted retention area as shown in fig. 9.
E) Adopting an edge detection algorithm to obtain candidate detained target contours in the detained detection region, setting an area threshold value, obtaining the minimum circumscribed rectangle of each candidate detained target contour, taking the minimum circumscribed rectangle of each candidate detained target contour with the area larger than the area threshold value as a candidate detained frame, setting a detained counter, and setting the detained counter to be 1;
F) setting an overlap threshold and a counter threshold, judging whether the overlap ratio of the candidate retention frame of the previous frame image to the candidate retention frame of the current frame image is greater than the overlap threshold, if not, setting a retention counter to be 1, and if so, adding 1 to the retention counter;
and judging whether the retention counter is larger than the counter threshold value, if so, giving an alarm of retention of articles or personnel at the entrance and exit of the escalator, and if not, repeating the step. Fig. 10 and 11 show the effect of the alarm for the retention of a person and the effect of the alarm for the retention of an article.
A Gaussian mixture model staircase detention detection system is shown in FIG. 2 and comprises an acquisition module 100, a pedestrian recognition module 200, a pedestrian analysis module 300, a detention detection module 400, a preprocessing module 500, a detention analysis module 600 and a judgment module 700;
the acquiring module 100 is used for acquiring a monitoring video image of an escalator region;
a pedestrian recognition module 200, configured to extract pedestrian information from the current frame image by using a deep learning network model;
the pedestrian analysis module 300 is used for collecting false detection and real pedestrian information, marking corresponding labels, constructing normal pedestrians and positive and negative samples of false detection data, constructing a classification model by using a machine learning method, classifying and screening real-time pedestrian detection information, retaining correct pedestrian information and removing false detection pedestrian information;
the retention detection module 400 is used for judging whether pedestrian information exists in the range of the escalator entrance area, constructing a detection area background, acquiring a background difference image and a frame difference method image of the current frame image, and performing exclusive or processing on the background difference image and the frame difference method image to acquire a retention target image;
the preprocessing module 500 is used for preprocessing the retention target image to eliminate noise and intercepting an image of the image in a retention detection area at an entrance and an exit of the escalator;
the retention analysis module 600 acquires candidate retention target contours in the retention detection area by using an edge detection algorithm, sets an area threshold, acquires a minimum circumscribed rectangle of each candidate retention target contour, takes the minimum circumscribed rectangle of each candidate retention target contour with the area larger than the area threshold as a candidate retention frame, sets a retention counter to 1, sets an overlap threshold, determines whether the overlap ratio of the candidate retention frame of the previous frame image to the candidate retention frame of the current frame image is larger than the overlap threshold, and sets the retention counter to 1 if the overlap ratio is smaller than the overlap threshold; if so, adding 1 to the retention counter;
and the judging module 700 is used for judging whether the retention counter is larger than the counter threshold value, if so, alarming the retention of articles or personnel at the entrance and exit of the escalator, and alarming by sound on the site of the system.
The invention utilizes the machine learning method to construct the classification model, filters the information of non-retentate, thereby reducing the condition of false detection and improving the accuracy of retentate detection, and in addition, establishes the Gaussian mixture model to identify and judge the retention behavior, and detects and alarms in real time the retention of articles and passengers at the entrance and exit of the escalator, thereby achieving the purpose of monitoring the retention behavior of the entrance and exit area of the escalator, and the retention detection accuracy is high.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (10)

1. A method for detecting retention of an escalator based on a Gaussian mixture model is characterized by comprising the following steps:
A) acquiring a monitoring video image of an escalator region through a camera, extracting pedestrian information from the video image by using a deep learning network, collecting false detection pedestrian information and real pedestrian information, recording the false detection pedestrian information as a negative sample, and recording the real pedestrian information as a positive sample;
B) a classification model is established by using a machine learning method, a negative sample and a positive sample are used for training the classification model, the trained classification model is used for classifying and screening pedestrian information detected in real time, real pedestrian information is reserved, and false detection pedestrian information is removed; judging whether pedestrian information exists in the range of the escalator entrance area, if so, entering the step C), and if not, repeating the step A);
C) constructing a Gaussian mixture model, obtaining a detection area background, obtaining a background difference image by using background subtraction, obtaining a frame difference method image by using an inter-frame difference method, and performing XOR processing on the background difference image and the frame difference method image to obtain a retention target image of a current frame;
D) preprocessing the retention target image, intercepting the preprocessed retention target image to obtain an image in a retention detection area at an entrance and an exit of the escalator;
E) adopting an edge detection algorithm to obtain candidate detained target contours in the detained detection region, setting an area threshold value, obtaining the minimum circumscribed rectangle of each candidate detained target contour, taking the minimum circumscribed rectangle of each candidate detained target contour with the area larger than the area threshold value as a candidate detained frame, setting a detained counter, and setting the detained counter to be 1;
F) setting an overlap threshold and a counter threshold, judging whether the overlap ratio of the candidate retention frame of the previous frame image to the candidate retention frame of the current frame image is greater than the overlap threshold, if not, setting a retention counter to be 1, and if so, adding 1 to the retention counter;
and judging whether the retention counter is larger than the counter threshold value, if so, giving an alarm of retention of articles or personnel at the entrance and exit of the escalator, and if not, repeating the step.
2. The escalator stagnation detection method based on the Gaussian mixture model as claimed in claim 1, wherein the escalator region monitoring video image comprises an escalator entrance image and/or an escalator exit image.
3. The escalator staying detection method based on the Gaussian mixture model as claimed in claim 1 or 2, wherein the deep learning network in the step A) is any one of OpenPose, SSD, Yolo and Faster-CNN.
4. The escalator retention detection method based on the Gaussian mixture model as claimed in claim 3, wherein,
the machine learning method is a support vector machine.
5. The escalator stoppage detection method based on the Gaussian mixture model as claimed in claims 1 and 4, wherein the Gaussian mixture model is constructed by using 500 frames of video images in an unmanned state in the step C).
6. The escalator retention detection method based on the Gaussian mixture model as claimed in claim 5, wherein,
in step C), the mixture of Gaussian mixture modelsThe learning rate parameter is set to a low learning rate at which the background difference image D is obtained by background subtractionn(x, y) setting a threshold T1According to the formula
Figure FDA0002246272510000021
One by one background difference image DnAnd (x, y) performing binarization processing on each pixel point to obtain a background differential binarization image, wherein the background differential binarization image comprises a real-time moving target and a static target which is more than the background image.
7. The escalator retention detection method based on the Gaussian mixture model as claimed in claim 1 or 6, wherein in step C), the learning rate parameter in the Gaussian mixture model is set as a high learning rate, and the frame difference method image E is obtained by using the inter-frame difference method under the high learning raten(x, y) setting a threshold T2According to the formula
Figure FDA0002246272510000022
Frame-by-frame difference image EnAnd (x, y) carrying out binarization processing on each pixel point to obtain a frame difference method binarization image, wherein the frame difference method binarization image contains a real-time moving target.
8. The escalator staying detection method based on the Gaussian mixture model as claimed in claim 7, wherein in the step D), the staying target image is preprocessed, and the preprocessing is corrosion and/or expansion.
9. The escalator retention detection method based on the Gaussian mixture model as claimed in claim 1 or 8, characterized in that,
the edge detection algorithm in the step E) adopts any one of a Sobel operator, a Laplacian operator and a Canny operator.
10. The escalator detention detection system based on the Gaussian mixture model is suitable for the escalator detention detection method based on the Gaussian mixture model according to any one of claims 1 to 9, and is characterized by comprising an acquisition module, a pedestrian identification module, a pedestrian analysis module, a detention detection module, a preprocessing module, a detention analysis module and a judgment module;
the acquisition module is used for acquiring a monitoring video image of the escalator area;
the pedestrian identification module is used for extracting pedestrian information from the current frame image by utilizing a deep learning network model;
the pedestrian analysis module is used for collecting false detection and real pedestrian information, printing corresponding labels, constructing normal pedestrians and positive and negative samples of false detection data, constructing a classification model by using a machine learning method, classifying and screening real-time pedestrian detection information, retaining correct pedestrian information and removing false detection pedestrian information;
the retention detection module is used for judging whether pedestrian information exists in the range of the escalator entrance area, constructing a detection area background, acquiring a background difference image and a frame difference method image of the current frame image, and then performing XOR processing on the background difference image and the frame difference method image to acquire a retention target image;
the preprocessing module is used for preprocessing the retention target image to eliminate noise and intercepting an image of the image in a retention detection area at an entrance and an exit of the escalator;
the retention analysis module adopts an edge detection algorithm to obtain candidate retention target outlines in the retention detection area, sets an area threshold value, obtains the minimum circumscribed rectangle of each candidate retention target outline, takes the minimum circumscribed rectangle of each candidate retention target outline with the area larger than the area threshold value as a candidate retention frame, sets a retention counter to be 1, sets an overlap threshold value, judges whether the overlap ratio of the candidate retention frame of the previous frame image and the candidate retention frame of the current frame image is larger than the overlap threshold value, and sets the retention counter to be 1 if the overlap ratio is smaller than the overlap threshold value; if so, adding 1 to the retention counter;
and the judging module is used for judging whether the retention counter is larger than the counter threshold value, if so, alarming retention of articles or personnel at the entrance and exit of the escalator, and alarming by sound on site of the system.
CN201911017881.7A 2019-10-24 2019-10-24 Staircase retention detection method and system based on Gaussian mixture model Active CN111259718B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911017881.7A CN111259718B (en) 2019-10-24 2019-10-24 Staircase retention detection method and system based on Gaussian mixture model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911017881.7A CN111259718B (en) 2019-10-24 2019-10-24 Staircase retention detection method and system based on Gaussian mixture model

Publications (2)

Publication Number Publication Date
CN111259718A true CN111259718A (en) 2020-06-09
CN111259718B CN111259718B (en) 2023-05-23

Family

ID=70946697

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911017881.7A Active CN111259718B (en) 2019-10-24 2019-10-24 Staircase retention detection method and system based on Gaussian mixture model

Country Status (1)

Country Link
CN (1) CN111259718B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914943A (en) * 2020-08-14 2020-11-10 广西大学 Information vector machine method and device for comprehensively judging stability of dumping type karst dangerous rock
CN112990155A (en) * 2021-05-12 2021-06-18 深圳金三立视频科技股份有限公司 Image processing-based legacy detection method and terminal
CN113033316A (en) * 2021-03-01 2021-06-25 湖南长海现代实验室设备有限公司 Device and method for detecting and controlling show window state of fume hood
CN117566564A (en) * 2024-01-15 2024-02-20 康力电梯股份有限公司 Method for detecting personnel retention at entrance and exit of escalator for conveying passengers to boarding
CN117994609A (en) * 2024-03-14 2024-05-07 广州邦讯信息系统有限公司 Comprehensive recognition method for multi-target pedestrian potential safety hazard behaviors

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130243322A1 (en) * 2012-03-13 2013-09-19 Korea University Research And Business Foundation Image processing method
CN105678267A (en) * 2016-01-08 2016-06-15 浙江宇视科技有限公司 Scene recognition method and device
CN106683077A (en) * 2016-12-07 2017-05-17 华南理工大学 Escalator floor board large-object retention detection method
CN107507221A (en) * 2017-07-28 2017-12-22 天津大学 With reference to frame difference method and the moving object detection and tracking method of mixed Gauss model
US20190244513A1 (en) * 2018-02-05 2019-08-08 Nec Laboratories America, Inc. False alarm reduction system for automatic manufacturing quality control
CN110348305A (en) * 2019-06-06 2019-10-18 西北大学 A kind of Extracting of Moving Object based on monitor video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130243322A1 (en) * 2012-03-13 2013-09-19 Korea University Research And Business Foundation Image processing method
CN105678267A (en) * 2016-01-08 2016-06-15 浙江宇视科技有限公司 Scene recognition method and device
CN106683077A (en) * 2016-12-07 2017-05-17 华南理工大学 Escalator floor board large-object retention detection method
CN107507221A (en) * 2017-07-28 2017-12-22 天津大学 With reference to frame difference method and the moving object detection and tracking method of mixed Gauss model
US20190244513A1 (en) * 2018-02-05 2019-08-08 Nec Laboratories America, Inc. False alarm reduction system for automatic manufacturing quality control
CN110348305A (en) * 2019-06-06 2019-10-18 西北大学 A kind of Extracting of Moving Object based on monitor video

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914943A (en) * 2020-08-14 2020-11-10 广西大学 Information vector machine method and device for comprehensively judging stability of dumping type karst dangerous rock
CN111914943B (en) * 2020-08-14 2022-04-15 广西大学 Information vector machine method and device for comprehensively judging stability of dumping type karst dangerous rock
CN113033316A (en) * 2021-03-01 2021-06-25 湖南长海现代实验室设备有限公司 Device and method for detecting and controlling show window state of fume hood
CN112990155A (en) * 2021-05-12 2021-06-18 深圳金三立视频科技股份有限公司 Image processing-based legacy detection method and terminal
CN117566564A (en) * 2024-01-15 2024-02-20 康力电梯股份有限公司 Method for detecting personnel retention at entrance and exit of escalator for conveying passengers to boarding
CN117994609A (en) * 2024-03-14 2024-05-07 广州邦讯信息系统有限公司 Comprehensive recognition method for multi-target pedestrian potential safety hazard behaviors

Also Published As

Publication number Publication date
CN111259718B (en) 2023-05-23

Similar Documents

Publication Publication Date Title
CN111259718B (en) Staircase retention detection method and system based on Gaussian mixture model
CN106203274B (en) Real-time pedestrian detection system and method in video monitoring
CN110378179B (en) Subway ticket evasion behavior detection method and system based on infrared thermal imaging
CN108446630B (en) Intelligent monitoring method for airport runway, application server and computer storage medium
Liu et al. A fall detection system using k-nearest neighbor classifier
CN102663452B (en) Suspicious act detecting method based on video analysis
CN111275910B (en) Method and system for detecting border crossing behavior of escalator based on Gaussian mixture model
CN100589561C (en) Dubious static object detecting method based on video content analysis
CN106204640A (en) A kind of moving object detection system and method
CN102982313B (en) The method of Smoke Detection
US20160004912A1 (en) System and method for robust motion detection
CN106600625A (en) Image processing method and device for detecting small-sized living thing
JP5106356B2 (en) Image monitoring device
CN111626203B (en) Railway foreign matter identification method and system based on machine learning
WO2023039781A1 (en) Method for detecting abandoned object, apparatus, electronic device, and storage medium
Su et al. A new local-main-gradient-orientation HOG and contour differences based algorithm for object classification
CN113297926A (en) Behavior detection and identification method and system
CN104899559B (en) A kind of rapid pedestrian detection method based on video monitoring
CN111597919A (en) Human body tracking method in video monitoring scene
CN112104838A (en) Image distinguishing method, monitoring camera and monitoring camera system thereof
WO2012081969A1 (en) A system and method to detect intrusion event
CN113657250A (en) Flame detection method and system based on monitoring video
CN117671799A (en) Human body falling detection method, device, equipment and medium combining depth measurement
KR100967456B1 (en) moving picture checking apparatus for an obstacle on a rail road crossing and method thereof
KR20150002040A (en) The way of Real-time Pedestrian Recognition and Tracking using Kalman Filter and Clustering Algorithm based on Cascade Method by HOG

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant