CN112149588A - Pedestrian attitude estimation-based intelligent elevator dispatching method - Google Patents
Pedestrian attitude estimation-based intelligent elevator dispatching method Download PDFInfo
- Publication number
- CN112149588A CN112149588A CN202011039991.6A CN202011039991A CN112149588A CN 112149588 A CN112149588 A CN 112149588A CN 202011039991 A CN202011039991 A CN 202011039991A CN 112149588 A CN112149588 A CN 112149588A
- Authority
- CN
- China
- Prior art keywords
- elevator
- pedestrian
- human body
- point
- affinity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 27
- 238000012706 support-vector machine Methods 0.000 claims abstract description 25
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 22
- 238000004458 analytical method Methods 0.000 claims abstract description 20
- 238000010586 diagram Methods 0.000 claims abstract description 19
- 210000000988 bone and bone Anatomy 0.000 claims abstract description 7
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000036544 posture Effects 0.000 claims description 19
- 239000013598 vector Substances 0.000 claims description 14
- 210000003414 extremity Anatomy 0.000 claims description 12
- 230000000694 effects Effects 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 10
- 210000000245 forearm Anatomy 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000010354 integration Effects 0.000 claims description 2
- 238000012423 maintenance Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 230000001629 suppression Effects 0.000 claims description 2
- 230000006399 behavior Effects 0.000 description 16
- 238000001514 detection method Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 210000002683 foot Anatomy 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 210000003423 ankle Anatomy 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000002955 isolation Methods 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 238000003909 pattern recognition Methods 0.000 description 2
- 210000000707 wrist Anatomy 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B66—HOISTING; LIFTING; HAULING
- B66B—ELEVATORS; ESCALATORS OR MOVING WALKWAYS
- B66B1/00—Control systems of elevators in general
- B66B1/02—Control systems without regulation, i.e. without retroactive action
- B66B1/06—Control systems without regulation, i.e. without retroactive action electric
- B66B1/14—Control systems without regulation, i.e. without retroactive action electric with devices, e.g. push-buttons, for indirect control of movements
- B66B1/18—Control systems without regulation, i.e. without retroactive action electric with devices, e.g. push-buttons, for indirect control of movements with means for storing pulses controlling the movements of several cars or cages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
- G06V40/25—Recognition of walking or running movements, e.g. gait recognition
Abstract
The invention provides an elevator intelligent scheduling method based on pedestrian attitude estimation, belonging to the field of intelligent transportation and elevator application and comprising the following steps: (1) acquiring a monitoring video outside each elevator car in real time, storing and preprocessing the monitoring video in real time; (2) carrying out attitude estimation on the monitoring video acquired in real time to obtain a human body bone node diagram and the number of human bodies of each passenger; (3) taking 18 key point coordinates of a human skeleton key point diagram as input, sending the input into a behavior analysis model based on a support vector machine, and outputting two results of elevator taking intention and elevator not taking intention; (4) the human body posture estimation module and the elevator taking behavior analysis module finally acquire the information of the number of pedestrians with the elevator taking intention in the monitoring video, and send the information obtained by algorithm analysis to the elevator intelligent dispatching module. The invention provides technical support for intelligent elevator dispatching.
Description
Technical Field
The invention belongs to the field of intelligent transportation and elevator application, and particularly relates to an elevator dispatching method based on pedestrian attitude estimation.
Background
Elevators are indispensable vertical transportation means, and although elevator products have been greatly changed in terms of hardware technology in recent years, it is difficult to make the entire elevator system provide high-quality service without a good elevator dispatching algorithm. Particularly, with the development of society, large buildings and high-rise and super high-rise buildings are more and more, so that a single elevator cannot meet the huge user requirements, and a plurality of elevators operate together and serve together, so that the elevator group control technology is developed at the discretion. An Elevator Group Control System (EGCS) is to connect a plurality of elevators with a computer, acquire passenger requests and Elevator running states acquired from the plurality of elevators by the computer, process data by corresponding dispatching algorithms, and then send Control commands to the elevators. The EGCS can send out reasonable elevator dispatching instructions in real time aiming at the change of passenger traffic flow in the building, and uniformly allocate the elevators of the whole elevator group so as to achieve the purpose of improving the operation efficiency and the service quality.
In order to solve the problem of identifying passenger traffic flow in an elevator group control system, researchers generally divide the elevator traffic flow into several different modes, such as an "up peak mode", a "down peak mode", and an "idle mode", by using an elevator traffic flow mode identification technology. The elevator traffic flow identification has the problem that the system is difficult to acquire detailed information of passenger calls in advance. The more detailed the passenger information pre-fetched by the control center is, the more beneficial the running efficiency of the elevator is. In the existing elevator system, the only operation that passengers can carry out before entering an elevator is to send out an up call instruction or a down call instruction, so that a dispatching center is difficult to acquire more passenger information in advance to optimize dispatching. And the group control system based on image recognition can acquire the passenger number information. At present, an elevator group control system based on image recognition mainly processes monitoring videos of an elevator waiting hall to obtain the approximate number of passengers, and methods adopted by the image recognition are generally based on target detection, but the method can only judge the number of passengers in a monitoring video and cannot accurately judge whether pedestrians are passengers and have elevator taking requirements. Considering whether the pedestrian in the video has the elevator taking requirement or not, the pedestrian can be judged according to the behavior and the posture, so the pedestrian can be processed by adopting a posture estimation technology, and then the posture of the pedestrian is analyzed so as to judge the elevator taking intention of the pedestrian.
Disclosure of Invention
The invention aims to provide an elevator intelligent dispatching method based on pedestrian attitude estimation, aiming at solving the problem of passenger traffic flow identification of an elevator group control system.
The technical scheme of the invention is implemented according to the following steps:
s1, acquiring the monitoring video outside each elevator car in real time, storing and preprocessing the monitoring video in real time;
s2, carrying out attitude estimation on the monitoring video acquired in real time to obtain a human body bone node map and the number of human bodies of each passenger;
s3, taking 18 key point coordinates of the human skeleton key point diagram as input, sending the input into a behavior analysis model based on a support vector machine, wherein the model is a two-classification model, and outputting results of elevator riding intention and elevator non-riding intention;
s4, a human body posture estimation module and an elevator taking behavior analysis module based on a support vector machine algorithm are included, the pedestrian number information with the elevator taking intention in the monitoring video is finally obtained, and the information obtained through algorithm analysis is sent to an elevator intelligent scheduling module;
further, in step S2, the human joint detection algorithm flow is as follows:
s2.1, detecting key points of bones: the skeleton key point thermodynamic diagram is composed of a series of two-dimensional points and is used for measuring the confidence coefficient of the key points appearing at a certain position of an image, and the position with the highest confidence coefficient is the final position of the key points. If there is only one person in the image, the particular keypoint has only one peak in the thermodynamic diagram, for the case of multiple persons, there are multiple peaks for the keypoint. The confidence of each location can be determined using a Gaussian function, with x, for the jth keypoint of the kth individualj,kRepresenting the actual position of the key point, the confidence S of the pixel points around the key point p is:
where the standard deviation sigma is used to control the range of the distribution of confidence values. For the multi-person case, the specific key point for each person is the maximum value within the range:
s2.2, a group of detected body parts is given, a possibility measurement needs to be carried out on the connection of each body part, the body parts are guaranteed to belong to the same person, and the body parts are finally assembled into the complete body postures of a plurality of persons. Each section of limb is specified to correspond to an affinity field, each pixel point on the limb is represented by a unit vector, the position and direction information of the limb is kept, and the affinity field is formed by a series of unit vectors. x is the number ofj1,kAnd xj2,kAre two key points on the k-th individual's forearm. If point p falls on the forearm, the affinity value at point p
Unit vectors for nodes j1 through j 2. If a point is outside the forearm, the affinity value at that point is a zero vector. The mathematical form is expressed as:
affinity field constraints:
σlis a distance threshold; lc,k＝||xj2,k-xj1,k||2The length of the whole small arm; v represents, v⊥Is a vector perpendicular to the forearm. The overlapping affinity regions of the lower arm are represented as:
nc(p) is the number of times the affinity fields of different people at position p are superimposed there, i.e. where the non-zero vectors of all people are superimposed. The confidence in the association between any two keypoints is the linear integration of the affinity field between the two points. For two keypoint locations dj1And dj2On a line segment consisting of two key points, the affinity field L is alignedcSampling is performed, and the confidence of association between two keypoints is integrated on the line segment for the affinity field:
p (u) is at two nodes dj1And dj2The difference between:
p(u)＝(1-u)dj1+udj2 (7)
s2.3, multi-person analysis: non-maximum suppression is used on the confidence map to obtain a set of candidate keypoints, and due to multiple persons or misjudgments, multiple candidate points exist for each part, and the candidate points define a possible limb set.
Defining a pair of key points j1, j2 for the c-th limb, finding the best match is a simplified bipartite graph problem. And obtaining the optimal matching by using the Hungarian algorithm.
The constraint conditions are as follows:
lc,k＝||xj2,k-xj1,k||2 (9)
Ecis the total weight of the limb c, ZcIs a subset of Z, represents
And
whether or not to connect. EmnIs that
And
affinity field confidence of, DJIs a set of candidate keypoints.
The optimization is expanded to the optimization of key points of multiple persons, and two relaxation conditions are added according to the self structure of the human body: only the connection of adjacent key points is considered, and the human body is represented by a tree structure; optimization is not performed globally, but only on each torso type, and the decomposition is a bipartite graph matching problem. The optimization can be simplified as follows:
the method has the advantages that the np-hard problem is converted into a plurality of bipartite graphs which are easy to solve for optimization, the overall optimal solution can be effectively approached, meanwhile, the complexity of the algorithm is greatly reduced, and the purpose of real-time multi-person attitude estimation is achieved.
Further, in step S3, a Support Vector Machine (SVM) is a Machine learning method with excellent performance, and the main idea is: given a training sample, a hyperplane is established as a decision surface such that the isolation margin between positive and negative examples is maximized. If the samples are linearly inseparable, the SVM uses a so-called kernel function to map the samples into a high-dimensional feature space through nonlinear mapping, thereby efficiently performing nonlinear classification. The SVM is one of the most commonly used classifiers with the best effect at present, is often applied to the two-classification problem, has unique advantages in solving small sample, nonlinear and high-dimensional pattern recognition, and has better generalization capability. The skeletal features to be processed in the text are high-dimensional data, and the convergence rate of the algorithm required by the instantaneity of an application scene is high, so that the classifier is obtained by learning and training through the SVM algorithm, and the problem of two classifications of distinguishing the waiting behavior from the walking behavior is solved. The SVM model training process comprises the following steps:
s3.1, the experimental dataset is a short video dataset of collective activities, which contains 5 different collective activities, respectively, cross (Crossing), walk (Walking), wait (Walking), talk (Talking), queue (Queueing), and 44 short video sequences. The method only needs to distinguish two behaviors of waiting and walking of people, so three types of waiting, queuing and talking in a data set are selected as samples of waiting states, two types of walking and crossing are selected as samples of walking states, the two types are 100 samples respectively, 80% of the two types of samples are used as training sets, and 20% of the two types of samples are used as verification sets.
S3.2, joint point coordinate normalization: because the distance from the person to the camera is changed when the person passes through the camera, the closer the person is to the camera, the larger the joint point diagram of the person is, and the farther the person is from the camera, the smaller the joint point diagram of the person is. To solve this problem, we need to normalize the extracted bone features to scale the human body posture to the same height. Because the neck node of the human body node point map is stable, the neck node is selected as the origin, and the distance between the neck and the hip center is taken as the basis, so that the skeleton characteristic change caused by the change of the distance between a person and a camera is eliminated. The normalization formula is as follows:
wherein P isi∈R2Is the coordinate of the skeletal joint point i, PiIs' is PiNormalized coordinates of (1), PneckIs the neck node coordinate, PhipAre the hip node coordinates. PneckWhen the origin is set, the formula is simplified as follows:
s3.3, the waiting people and the walking people can be distinguished through the SVM classifier, and the passengers who are expected to take the elevator are excluded through the body orientation of the pedestrians in the last step. The pedestrian facing the elevator is judged to have the intention of taking the elevator, and the pedestrian facing away from the elevator is judged to have no intention of taking the elevator.
Further, after the information obtained by the algorithm analysis in the step S4 is sent to the elevator intelligent dispatching module, the elevator intelligent dispatching module dispatches the elevator in real time according to a certain dispatching rule.
Has the advantages that:
the invention provides an elevator intelligent scheduling method based on pedestrian attitude estimation. Firstly, carrying out posture estimation on a human body in a video in real time by using a human body posture estimation calculation method to obtain a human body skeleton key point diagram of each pedestrian, then using a skeleton node diagram as the input of an SVM classifier to identify and classify two behaviors of waiting and walking of the pedestrian, and eliminating the people without the need of taking a ladder by judging the body orientation of the passenger. And finally, the elevator taking demand information of the passengers is sent to the elevator intelligent dispatching module, and technical support is provided for elevator intelligent dispatching.
Drawings
FIG. 1 is a system framework diagram of a method for elevator ride intent prediction and elevator dispatch based on human pose estimation and behavior analysis in accordance with an embodiment of the present invention;
FIG. 2 is a diagram illustrating the effect of the posture estimation according to the embodiment of the present invention, wherein a is one of the pedestrians;
FIG. 3 is a diagram of a support vector machine based behavior analysis framework in accordance with an embodiment of the present invention;
FIG. 4 is a schematic diagram of a rectangular coordinate system of a top view according to an embodiment of the present invention;
Detailed Description
In order to make the present invention better understood by those skilled in the art, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, a flowchart illustrating steps of an intelligent elevator dispatching method based on pedestrian attitude estimation according to an embodiment of the present invention is shown, which includes the following steps:
s1, acquiring the monitoring video outside each elevator car in real time, storing and preprocessing the monitoring video in real time;
the above steps are to obtain the pedestrian before the elevator shot in each floor by the most suitable angle through the elevator taking person monitoring video image data acquisition unit including but not limited to the monitoring camera, the industrial camera and the like placed in the monitoring area of the appointed elevator, which is installed in the passageway waiting for each floor of the staircase.
S2, carrying out attitude estimation on the monitoring video acquired in real time, and carrying out pretreatment in real time to obtain a human body skeleton node diagram and the number of human bodies of each passenger, as shown in figure 2;
s3, taking 18 key point coordinates of the human skeleton key point diagram as input, sending the input into a behavior analysis model based on a support vector machine, wherein the model is a two-classification model, and outputting results of elevator riding intention and elevator non-riding intention;
the human body posture estimation module and the elevator-taking behavior analysis module based on the support vector machine algorithm are mainly operated on a GPU server or other high-computation-capacity embedded devices, including but not limited to latest GPU embedded devices TX1 and TX2 and the like introduced by Invitta.
S4, a human body posture estimation module and an elevator taking behavior analysis module based on a support vector machine algorithm are included, the pedestrian number information with the elevator taking intention in the monitoring video is finally obtained, and the information obtained through algorithm analysis is sent to an elevator intelligent scheduling module;
the steps are processed by an elevator intelligent scheduling unit, wherein the elevator intelligent scheduling unit comprises but is not limited to an elevator monitoring platform of a residential area or a shopping mall property and a sensor arranged in an elevator. The control center for elevator dispatching includes but is not limited to processing devices such as CPUs, FPGAs and the like. The elevator operation monitoring module mainly comprises an elevator monitoring operation unit, an elevator operation and maintenance platform and other equipment based on a CPU, a DSP, an FPGA and other processors.
In a specific application example, the human body posture estimation module in S2 and S3 and the elevator riding behavior analysis module based on the support vector machine algorithm are core modules, and the working processes of the two modules are further detailed below.
In step S2, the body posture estimation module provides 18 or 25 joint points detection, wherein the main difference between the 18 or 25 joint points is that the joint points of the left and right feet are more detected than the joint points of the 25 joint points. Since the information about the joints of the foot is of little significance for passenger identification, and the detection of the joints of the foot has a large influence on the running speed of the program, the information about 18 human joints is selected as the input of the passenger identification algorithm. The 18 human body joint points are respectively: 0: nose, 1: neck, 2: right shoulder, 3: right elbow, 4: right wrist, 5: left shoulder, 6: left elbow, 7: left wrist, 8: right hip, 9: right knee, 10: right ankle, 11: left hip, 12: left knee, 13: the left ankle. Fig. 2 is an effect diagram of posture estimation of a pedestrian in a video.
Further, in step S3, a Support Vector Machine (SVM) is a Machine learning method with excellent performance, and the main idea is: given a training sample, a hyperplane is established as a decision surface such that the isolation margin between positive and negative examples is maximized. If the samples are linearly inseparable, the SVM uses a so-called kernel function to map the samples into a high-dimensional feature space through nonlinear mapping, thereby efficiently performing nonlinear classification. The SVM is one of the most commonly used classifiers with the best effect at present, is often applied to the two-classification problem, has unique advantages in solving small sample, nonlinear and high-dimensional pattern recognition, and has better generalization capability. The skeletal features to be processed in the text are high-dimensional data, and the convergence rate of the algorithm required by the instantaneity of an application scene is high, so that the classifier is obtained by learning and training through the SVM algorithm, and the problem of two classifications of distinguishing the waiting behavior from the walking behavior is solved. The SVM model training process comprises the following steps:
s3.1, the experimental dataset is a short video dataset of collective activities, which contains 5 different collective activities, respectively, cross (Crossing), walk (Walking), wait (Walking), talk (Talking), queue (Queueing), and 44 short video sequences. The method only needs to distinguish two behaviors of waiting and walking of people, so three types of waiting, queuing and talking in a data set are selected as samples of waiting states, two types of walking and crossing are selected as samples of walking states, the two types are 100 samples respectively, 80% of the two types of samples are used as training sets, and 20% of the two types of samples are used as verification sets.
S3.2, joint point coordinate normalization: because the distance from the person to the camera is changed when the person passes through the camera, the closer the person is to the camera, the larger the joint point diagram of the person is, and the farther the person is from the camera, the smaller the joint point diagram of the person is. To solve this problem, the extracted bone features need to be normalized to scale the human body posture to the same height. Since the human body closes the node map 0: the neck node is stable, the neck node is selected as an origin point, and the distance between the neck and the hip center is taken as a basis, so that the change of the bone characteristics caused by the change of the distance between a person and a camera is eliminated. The normalization formula is as follows:
wherein P isi∈R2Is the coordinate of the skeletal joint point i, PiIs' is PiNormalized coordinates of (1), PneckIs the neck node coordinate, PhipAre the hip node coordinates. PneckWhen the origin is set, the formula is simplified as follows:
s3.3, judging the direction of the human body: by means of the SVM classifier, waiting people and walking people can be distinguished, and the last step is to eliminate passengers who intend to take the elevator through the body orientation of pedestrians. Under the motion constraint of human body structure, the orientation of human head and the orientation of human body can not exceed 180 deg.,and according to statistics, the head and body directions are consistent in most cases, so that the directions of the bodies of the pedestrians can be determined according to the head directions of the pedestrians, and the head directions can be determined according to the following conditions that the head directions are 0: neck and 1: vector formed by two key points of nose
And (4) determining. Knowing the position of the elevator in the video, the pedestrian neck node can be used as the origin, and the direction entering the elevator (which can be calibrated by the actual shooting angle) is used as the y-axis to establish a coordinate system under the overlooking view angle, as shown in fig. 4.
Projected on a coordinate system as
The angle with the x-axis is denoted as θ. Then stipulate
When the pedestrian is moving to the elevator,
when the pedestrian walks backwards to the elevator. Referring to fig. 2, a pedestrian a is selected, and the head node coordinate is detected to be (292,243), and the neck node coordinate is detected to be (301,202). The direction of the elevator entrance is horizontal to the left, and the angle formed by the connecting line of the head node coordinate and the neck node coordinate is less than 90 degrees, so that the person is judged to have the elevator riding intention.
Claims (3)
1. An elevator intelligent scheduling method based on pedestrian attitude estimation is characterized in that:
the technical scheme of the invention is implemented according to the following steps:
s1, acquiring the monitoring video outside each elevator car in real time, and storing and preprocessing the monitoring video in real time;
s2, carrying out posture estimation on the pedestrians in the monitoring video acquired in real time to obtain a human body skeleton node graph and the number of human bodies of each pedestrian;
s2.1-determine confidence for each location using Gaussian function, with x for the jth keypoint of the kth individualj，kRepresenting the actual position of the key point, the confidence of the pixel points around the key point is as follows:
the standard deviation sigma controls the distribution range of the confidence value; for the multi-person case, the specific key point for each person is the maximum value within the range:
s2.2, a group of detected body parts are given, and the connection of each body part needs to be measured, so that the body parts belong to the same person, and the body parts are finally assembled into the complete body postures of a plurality of persons; each section of limb is specified to correspond to an affinity field, each pixel point on the limb is represented by a unit vector, the position and direction information of the limb is kept, and the affinity field is formed by a series of unit vectors; let xj1，kAnd xj2，kAre two key points on the kth individual's forearm; if point p falls on the forearm, the affinity value at point p
Unit vectors for nodes j1 through j 2; if the point is outside the forearm, the affinity value at the point is a zero vector; the mathematical form is expressed as:
affinity field constraints:
σlis a distance threshold; lc，k＝||xj2，k-xj1，k||2The length of the whole small arm; v is a vector extending in the direction of the forearm, v⊥Is a vector perpendicular to the forearm; the overlapping affinity regions of the lower arm are represented as:
nc(p) is the number of times the affinity fields of different people at position p are superimposed there, i.e. where the non-zero vectors of all people are superimposed;
the confidence of the association between any two key points is the linear integration of the affinity field between the two points; for two keypoint locations dj1And dj2On a line segment consisting of two key points, the affinity field L is alignedcSampling is performed, and the confidence of association between two keypoints is integrated on the line segment for the affinity field:
p (u) is at two nodes dj1And dj2The difference between:
p(u)＝(1-u)dj1+udj2 (7)
s2.3 multi-person analysis: adopting non-maximum suppression on the confidence map to obtain a group of candidate key points, wherein multiple candidate points exist in each part due to multiple persons or misjudgment, and the candidate points define a possible limb set;
defining a pair of key points j1 and j2 of the c-th limb, and obtaining the optimal matching by using the Hungarian algorithm:
the constraint conditions are as follows:
lc，k＝||xj2，k-xj1，k||2 (9)
Ecis the total weight of the limb c, ZcIs a subset of Z, represents
And
whether to connect; emnIs that
And
affinity field confidence of, DJIs a set of candidate keypoints;
the optimization is expanded to the optimization of key points of multiple persons, and two relaxation conditions are added according to the self structure of the human body: only the connection of adjacent key points is considered, and the human body is represented by a tree structure; global optimization is not carried out, optimization is only carried out on each trunk type, and the problem of bipartite graph matching is resolved; the optimization can be simplified as follows:
therefore, the global optimal solution can be effectively approached, and meanwhile, the complexity of the algorithm is greatly reduced, so that the aim of real-time multi-person attitude estimation is fulfilled;
s3, taking 18 key point coordinates of the human skeleton key point diagram as input, sending the input into a behavior analysis model based on a support vector machine, and outputting results of elevator riding intention and elevator non-riding intention;
s3.1 the experimental dataset is a short video dataset of collective activities, which comprises 5 different collective activities, namely Crossing (Crossing), Walking (Walking), Waiting (Walking), Talking (Talking), queuing (queuing) and 44 short video sequences; therefore, three types of waiting, queuing and talking in a data set are selected as samples of waiting states, two types of walking and crossing are selected as samples of walking states, the two types of samples are respectively 100, 80% of the two types of samples are used as training sets, and 20% of the two types of samples are used as verification sets;
s3.2, joint point coordinate normalization: normalizing the extracted bone features to zoom the human body posture to the same height; because the neck node of the human body is stable, the neck node is selected as the origin, and the distance between the neck and the hip center is taken as the basis, the normalization formula is as follows:
wherein P isi∈R2Is the coordinate of the skeletal joint point i, PiIs' is PiNormalized coordinates of (1), PneckIs the neck node coordinate, PhipIs the hip node coordinates; pneckWhen the origin is set, the formula is simplified as follows:
s3.3, waiting people and walking people can be distinguished through an SVM classifier, and the passengers who are expected to take the elevator are eliminated through the body orientation of the pedestrians in the last step; the pedestrian facing the elevator is judged to have the intention of taking the elevator, and the pedestrian facing away from the elevator is judged to have no intention of taking the elevator;
s4, the human body posture estimation module and the behavior analysis module based on the support vector machine finally acquire the pedestrian number information with the elevator riding intention in the monitoring video, and send the information obtained by algorithm analysis to the elevator intelligent scheduling module.
2. An elevator intelligent scheduling method based on pedestrian attitude estimation is characterized in that:
and after the information obtained by the algorithm analysis in the step S4 is sent to the elevator intelligent dispatching module, the elevator intelligent dispatching module dispatches the elevator in real time according to a certain dispatching rule.
3. An elevator intelligent scheduling method based on pedestrian attitude estimation is characterized in that: the steps are processed by an elevator intelligent scheduling unit, wherein the elevator intelligent scheduling unit comprises but is not limited to an elevator monitoring platform of a residential area or a shopping mall property and a sensor arranged in an elevator; the control center for elevator dispatching comprises but is not limited to processing equipment such as a CPU (central processing unit), an FPGA (field programmable gate array) and the like; the elevator operation monitoring module mainly comprises an elevator monitoring operation unit, an elevator operation and maintenance platform and other equipment based on a CPU, a DSP, an FPGA and other processors.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011039991.6A CN112149588A (en) | 2020-09-28 | 2020-09-28 | Pedestrian attitude estimation-based intelligent elevator dispatching method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011039991.6A CN112149588A (en) | 2020-09-28 | 2020-09-28 | Pedestrian attitude estimation-based intelligent elevator dispatching method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112149588A true CN112149588A (en) | 2020-12-29 |
Family
ID=73895746
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011039991.6A Pending CN112149588A (en) | 2020-09-28 | 2020-09-28 | Pedestrian attitude estimation-based intelligent elevator dispatching method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112149588A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906667A (en) * | 2021-04-07 | 2021-06-04 | 南开大学 | Human body posture layered classification method based on three-dimensional human body key points |
| CN113371561A (en) * | 2021-05-31 | 2021-09-10 | 浙江金轴机电科技有限公司 | Intelligent elevator control method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090057068A1 (en) * | 2006-01-12 | 2009-03-05 | Otis Elevator Company | Video Aided System for Elevator Control |
| CN109368434A (en) * | 2018-11-23 | 2019-02-22 | 张勇 | A kind of elevator control system with video surveillance |
| CN109460702A (en) * | 2018-09-14 | 2019-03-12 | 华南理工大学 | Passenger's abnormal behaviour recognition methods based on human skeleton sequence |
| CN109829390A (en) * | 2019-01-09 | 2019-05-31 | 浙江新再灵科技股份有限公司 | A kind of elevator intelligent scheduling system and method based on deep learning |
| CN110342355A (en) * | 2019-07-12 | 2019-10-18 | 苏州健雄职业技术学院 | A kind of intelligent building elevator dispatching plateform system and its working method |
| CN111292534A (en) * | 2020-02-13 | 2020-06-16 | 北京工业大学 | Traffic state estimation method based on clustering and deep sequence learning |
-
2020
- 2020-09-28 CN CN202011039991.6A patent/CN112149588A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090057068A1 (en) * | 2006-01-12 | 2009-03-05 | Otis Elevator Company | Video Aided System for Elevator Control |
| CN109460702A (en) * | 2018-09-14 | 2019-03-12 | 华南理工大学 | Passenger's abnormal behaviour recognition methods based on human skeleton sequence |
| CN109368434A (en) * | 2018-11-23 | 2019-02-22 | 张勇 | A kind of elevator control system with video surveillance |
| CN109829390A (en) * | 2019-01-09 | 2019-05-31 | 浙江新再灵科技股份有限公司 | A kind of elevator intelligent scheduling system and method based on deep learning |
| CN110342355A (en) * | 2019-07-12 | 2019-10-18 | 苏州健雄职业技术学院 | A kind of intelligent building elevator dispatching plateform system and its working method |
| CN111292534A (en) * | 2020-02-13 | 2020-06-16 | 北京工业大学 | Traffic state estimation method based on clustering and deep sequence learning |
Non-Patent Citations (2)
| Title |
|---|
| 臧坤;章国宝;: "智能电梯视频监测系统的设计与实现", 工业控制计算机, no. 04, 25 April 2019 (2019-04-25) * |
| 许忠雄;张睿哲;石晓军;岳贵杰;刘弋锋;: "深度学习实时多人姿态估计与跟踪", 中国电子科学研究院学报, no. 04, 20 August 2018 (2018-08-20) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112906667A (en) * | 2021-04-07 | 2021-06-04 | 南开大学 | Human body posture layered classification method based on three-dimensional human body key points |
| CN112906667B (en) * | 2021-04-07 | 2023-05-02 | 南开大学 | Human body posture layered classification method based on three-dimensional human body key points |
| CN113371561A (en) * | 2021-05-31 | 2021-09-10 | 浙江金轴机电科技有限公司 | Intelligent elevator control method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109460702B (en) | Passenger abnormal behavior identification method based on human body skeleton sequence | |
| CN109522793B (en) | Method for detecting and identifying abnormal behaviors of multiple persons based on machine vision | |
| Wang et al. | Fall detection based on dual-channel feature integration | |
| CN109948453B (en) | Multi-person attitude estimation method based on convolutional neural network | |
| CN110473232A (en) | Image-recognizing method, device, storage medium and electronic equipment | |
| CN110738154A (en) | pedestrian falling detection method based on human body posture estimation | |
| CN112149588A (en) | Pedestrian attitude estimation-based intelligent elevator dispatching method | |
| WO2021218671A1 (en) | Target tracking method and device, and storage medium and computer program | |
| CN112528812A (en) | Pedestrian tracking method, pedestrian tracking device and pedestrian tracking system | |
| CN113763429A (en) | Pig behavior recognition system and method based on video | |
| CN116092199B (en) | Employee working state identification method and identification system | |
| CN113516005A (en) | Dance action evaluation system based on deep learning and attitude estimation | |
| Arif et al. | Human pose estimation and object interaction for sports behaviour | |
| CN112132157A (en) | Raspberry pie-based gait face fusion recognition method | |
| CN111079481B (en) | Aggressive behavior recognition method based on two-dimensional skeleton information | |
| Benabbas et al. | Action recognition using direction models of motion | |
| CN114581990A (en) | Intelligent running test method and device | |
| Sheu et al. | Improvement of human pose estimation and processing with the intensive feature consistency network | |
| CN116229507A (en) | Human body posture detection method and system | |
| CN114758286A (en) | Intelligent edge safety monitoring method and device based on work ticket event | |
| CN113408435B (en) | Security monitoring method, device, equipment and storage medium | |
| CN109753934A (en) | A kind of method and identification device identifying image true-false | |
| Zhao et al. | An alphapose-based pedestrian fall detection algorithm | |
| CN114943873A (en) | Method and device for classifying abnormal behaviors of construction site personnel | |
| CN115187660A (en) | Knowledge distillation-based multi-person human body posture estimation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |