CN108222749B - Intelligent automatic door control method based on image analysis - Google Patents
Intelligent automatic door control method based on image analysis Download PDFInfo
- Publication number
- CN108222749B CN108222749B CN201711480203.5A CN201711480203A CN108222749B CN 108222749 B CN108222749 B CN 108222749B CN 201711480203 A CN201711480203 A CN 201711480203A CN 108222749 B CN108222749 B CN 108222749B
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- image
- automatic door
- area
- intelligent automatic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- E—FIXED CONSTRUCTIONS
- E05—LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
- E05F—DEVICES FOR MOVING WINGS INTO OPEN OR CLOSED POSITION; CHECKS FOR WINGS; WING FITTINGS NOT OTHERWISE PROVIDED FOR, CONCERNED WITH THE FUNCTIONING OF THE WING
- E05F15/00—Power-operated mechanisms for wings
- E05F15/70—Power-operated mechanisms for wings with automatic actuation
- E05F15/73—Power-operated mechanisms for wings with automatic actuation responsive to movement or presence of persons or objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- E—FIXED CONSTRUCTIONS
- E05—LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
- E05Y—INDEXING SCHEME RELATING TO HINGES OR OTHER SUSPENSION DEVICES FOR DOORS, WINDOWS OR WINGS AND DEVICES FOR MOVING WINGS INTO OPEN OR CLOSED POSITION, CHECKS FOR WINGS AND WING FITTINGS NOT OTHERWISE PROVIDED FOR, CONCERNED WITH THE FUNCTIONING OF THE WING
- E05Y2900/00—Application of doors, windows, wings or fittings thereof
- E05Y2900/10—Application of doors, windows, wings or fittings thereof for buildings or parts thereof
- E05Y2900/13—Application of doors, windows, wings or fittings thereof for buildings or parts thereof characterised by the type of wing
- E05Y2900/132—Doors
Abstract
The invention discloses an intelligent automatic door control method based on image analysis, belonging to the technical field of data processing, and comprising the steps of carrying out background modeling on a monitoring area of a camera arranged on an automatic door to obtain an initial background image; differentiating the image currently acquired by the camera with the initial background image to obtain a foreground image; detecting and positioning the pedestrian in the foreground image to obtain a central point position corresponding to the region where the pedestrian is located; tracking the position of the pedestrian according to the position of the corresponding central point of the area where the pedestrian is located in the continuous multi-frame images to obtain the walking direction of the pedestrian; judging whether the walking direction of the pedestrian is parallel to the horizontal direction; if the automatic door is controlled to open, otherwise, the automatic door is not opened. Through handling the image that contains the pedestrian, according to the opening/closing of the direction control intelligence automatically-controlled door of pedestrian's walking for in the complex environment such as automatically-controlled door is applicable to the corridor, reduced the spoilage of intelligence automatically-controlled door.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to an intelligent automatic door control method based on image analysis.
Background
Automatic doors based on infrared sensors have been used in various areas of life in society. Currently, infrared automatic doors are not suitable in some situations. For example, when a certain infrared automatic door is installed on the side of a corridor, many people passing through the corridor do not want to go out, but the infrared door is opened because the walking direction of the pedestrian cannot be distinguished by the current infrared automatic door. Moreover, when the flow of people in the corridor is large enough, the infrared automatic door is in a high-frequency opening and closing state. On the one hand, the damage of the automatic infrared door is accelerated. On the other hand, the high-frequency opening of the automatic door also causes great noise interference to the surrounding environment, and influences the work and study of workers.
Disclosure of Invention
The invention aims to provide an intelligent automatic door control method based on image analysis, which improves the intelligent level of an intelligent automatic door.
In order to realize the purpose, the invention adopts the technical scheme that:
an intelligent automatic door control method based on image analysis is adopted, and comprises the following steps:
carrying out background modeling on a monitoring area of a camera installed on the intelligent automatic door to obtain an initial background image;
differentiating the image currently acquired by the camera with the initial background image to obtain a foreground image;
detecting and positioning the pedestrian in the foreground image to obtain a central point position corresponding to the region where the pedestrian is located;
tracking the position of the pedestrian according to the position of the corresponding central point of the area where the pedestrian is located in the continuous multi-frame images to obtain the walking direction of the pedestrian;
judging whether the walking direction of the pedestrian is parallel to the horizontal direction;
if so, controlling the intelligent automatic door to open the door, otherwise, controlling the intelligent automatic door not to open the door.
Preferably, after the walking direction of the pedestrian is parallel to the horizontal direction, the method further comprises the following steps:
(a) acquiring a moving track of a central point position corresponding to an area where a pedestrian is located in continuous multi-frame images to obtain the walking speed of the pedestrian;
(b) judging whether the walking speed of the pedestrian is less than a set threshold value, if so, executing the step (c), otherwise, executing the step (f);
(c) analyzing the orientation of the pedestrian, judging whether the orientation of the pedestrian face is a front face, if so, executing the step (d), and if not, executing the step (f);
(d) analyzing the deflection angle of the face of the pedestrian, and judging whether the deflection angle of the face of the pedestrian is within a set range, if so, executing the step (e), otherwise, executing the step (f);
(e) controlling the intelligent automatic door to open;
(f) the intelligent automatic door does not open.
Preferably, after the background modeling is performed on the monitoring area of the camera installed on the intelligent automatic door to obtain an initial background image, the method further includes:
extracting the current image and the initial background image acquired by the camera according to the set area range to obtain the current image and the initial background image with the reduced range;
correspondingly, the differentiating the image currently shot by the camera with the initial background image to obtain a foreground image includes:
and carrying out difference on the image with the reduced range and the initial background image with the reduced range to obtain a foreground image.
Preferably, background modeling is performed on a monitoring area of a camera installed on the intelligent automatic door to obtain a background image, and the method specifically includes:
acquiring monitoring area images of a camera to acquire N frames of monitoring area images;
detecting whether a moving target exists in the monitored area image by adopting a frame difference method;
if so, acquiring the monitoring area image of the camera again until N frames of monitoring area images without moving targets are acquired as background images;
averaging pixels at the same pixel point position in the N frames of background images to obtain an initial background image;
analyzing each pixel point in the collected current frame monitoring area image, and judging whether the pixel point belongs to the initial background image or not;
and if a certain pixel point belongs to the background pixel, updating the pixel point into the initial background image in a weighting mode so as to realize the updating of the initial background image.
Preferably, in the foreground image, the detecting and positioning of the pedestrian are performed to obtain a central point position corresponding to the area where the pedestrian is located, and the method specifically includes:
training the constructed first neural network by using a first training data set formed by a single pedestrian image sample to obtain a pedestrian detector;
extracting the region part of each pedestrian from the collected image with the reduced range;
and inputting each individual pedestrian area as an individual pedestrian detector, and outputting the central point position of the pedestrian in the current collected image after the prediction of the pedestrian detector.
Preferably, according to the position of the central point corresponding to the area where the pedestrian is located in the continuous multi-frame image, the tracking of the position of the pedestrian is performed to obtain the walking direction of the pedestrian, and the method specifically comprises the following steps:
sending all pedestrian areas in continuous multi-frame collected images to a pedestrian detector for prediction to obtain a central point position corresponding to each pedestrian area;
matching pedestrian areas in adjacent frames by adopting a minimum distance method to obtain pedestrian areas belonging to the same pedestrian in the adjacent frames;
associating pedestrian areas belonging to the same person in adjacent frames, and connecting the central positions of the areas corresponding to the same pedestrian in the adjacent frames to obtain the moving track of the pedestrian;
and judging the walking direction of the pedestrian according to the moving track of the pedestrian.
Preferably, the method for acquiring the moving track of the central point position corresponding to the area where the pedestrian is located in the continuous multi-frame image to obtain the walking speed of the pedestrian specifically comprises the following steps:
continuously acquiring the moving track of a certain pedestrian in the continuous M-frame image range, and recording the initial point and the end point of the track;
and dividing the difference value of the distances between the track end point and the starting point by the number of the frame images to obtain the average speed of the pedestrian.
Preferably, the analyzing of the pedestrian orientation specifically includes:
constructing a second neural network and a second training data set, and training the constructed second neural network by using the second training data set to obtain a face orientation detector;
normalizing the input image frame, and taking the normalized image as the input of a human face orientation detector to obtain the proportion that the human face orientation in the input image frame is a front face and the proportion that the human face orientation is a side face;
the face orientation with a higher ratio is selected as the detection result of the face orientation detector from the ratio of the face orientation to the front and the ratio of the face orientation to the side.
Preferably, the analysis of the deflection angle of the face of the pedestrian specifically includes:
constructing a third neural network and a third training data set, and training the third neural network by using the third training data set to obtain a human face deflection angle detector;
cutting an input image frame to obtain a local image area with the head of a pedestrian;
normalizing the local image area to obtain a normalized local image area;
and taking the normalized local image area as the input of a human face deflection angle detector to obtain the human face deflection angle.
Preferably, in the foreground image, after detecting and locating the pedestrian and obtaining the center point position corresponding to the region where the pedestrian is located, the method further includes:
and cutting the pedestrian area and reserving the arm area of the human body.
Carrying out thresholding and connected domain extraction operations on an arm region of a human body to obtain all connected domains of the arm region;
selecting a connected domain corresponding to the largest area of the connected domain, and solving a circumscribed rectangle of the connected domain;
when the length of the horizontal direction of the external rectangle is smaller than the angle of the vertical direction, the intelligent automatic door is controlled to be opened.
Compared with the prior art, the invention has the following technical effects: according to the intelligent automatic door, the camera arranged on the intelligent automatic door is used for collecting the image of the monitored area, analyzing the collected image to obtain the walking direction of the passing pedestrian, and controlling the intelligent automatic door to be opened when the walking direction of the pedestrian is vertical to the camera. When the infrared automatic door is arranged on the side face of the corridor, the intelligent automatic door is controlled to be opened when the walking direction of the pedestrian is judged to be parallel to the horizontal direction. This scheme is through handling the image that contains the pedestrian, and according to the opening/closing of the direction control intelligence automatically-controlled door of pedestrian's walking, compare with the mode that has only the pedestrian to open at the corridor through just controlling intelligence automatically-controlled door among the prior art, can be very big reduction the possibility that the automatically-controlled door mistake was opened for during the automatically-controlled door is applicable to complex environment such as corridor, reduced the spoilage of intelligence automatically-controlled door.
Drawings
The following detailed description of embodiments of the invention refers to the accompanying drawings in which:
FIG. 1 is a schematic flow diagram of a method for intelligent automatic door control based on image analysis;
FIG. 2 is a schematic diagram of the definition of neighboring pixels;
FIG. 3 is a schematic diagram of a neural network architecture;
FIG. 4 is a schematic diagram of a single neuron structure;
FIG. 5 is a schematic general flow diagram of a method for intelligent automatic door control based on image analysis;
fig. 6 is a flow chart of hand waving detection.
Detailed Description
To further illustrate the features of the present invention, refer to the following detailed description of the invention and the accompanying drawings. The drawings are for reference and illustration purposes only and are not intended to limit the scope of the present disclosure.
As shown in fig. 1, the present embodiment discloses an intelligent automatic door control method based on image analysis, which includes the following steps:
s101, performing background modeling on a monitoring area of a camera installed on an intelligent automatic door to obtain an initial background image;
s102, differentiating an image currently acquired by a camera with an initial background image to obtain a foreground image;
s103, detecting and positioning the pedestrian in the foreground image to obtain a central point position corresponding to the area where the pedestrian is located;
s104, tracking the position of the pedestrian according to the position of the corresponding central point of the area where the pedestrian is located in the continuous multi-frame images to obtain the walking direction of the pedestrian;
s105, judging whether the walking direction of the pedestrian is parallel to the horizontal direction;
s106, if so, controlling the intelligent automatic door to open;
and S107, otherwise, the intelligent automatic door is not opened.
As a further preferable scheme, because the camera on the intelligent automatic door is fixed, background modeling can be performed on the monitoring area of the camera to obtain a background image, so as to reduce invalid search. In this embodiment, in step S101, background modeling is performed by using a multi-frame average and real-time iteration method, which specifically includes:
(1) a total of N frames (e.g., 300 frames) of monitored area images are acquired.
(2) And detecting whether the moving target exists in the N frames of monitoring area images by adopting a frame difference method, and if so, acquiring again until the number of the monitoring area images without the moving target meets the requirement. The process of detecting whether the moving object exists in the monitoring area image is as follows:
a1, frame difference processing: monitoring area image P for the ith frame acquired currentlyi+1And the previous frame of monitoring area image PiMaking frame difference, and recording the result of the frame difference as Di+1，i≥0：
Di+1(x,y)＝|Pi+1(x,y)-Pi(x,y)|，
Wherein, Pi+1(x, y) is represented at the P-thi+1Pixel value, p, corresponding to point (x, y) in the image of the monitored area of the framei(x, y) is represented at the P-thiPixel value, D, corresponding to point (x, y) in the image of the frame monitoring areai+1(x, y) is represented at Di+1In the frame difference image, the pixel value corresponding to the point (x, y).
b1, thresholding operation: for a frame difference image D with a threshold of 20i+1Carrying out thresholding operation, wherein the specific principle is as follows:
in-frame difference image Di+1For each pixel (x, y) if Di+1If the value of (x, y) is greater than 20, the point (x, y) is retained as a foreground pixel while the pixel value of the point is changed to 1, if D is greater than 1i+1(x, y) is less than 20, the pixel value at this point is changed to 0, and the final image is designated as DXi+1。
It should be noted that the threshold value of 20 is an empirical threshold value for performing the thresholding operation, which is obtained by a lot of experiments by those skilled in the art.
b1, generating a connected domain: pixel-by-pixel traversal image DXi+1And if the pixel values of two adjacent pixels are not 0, dividing the two pixels into the same connected domain, thus obtaining a plurality of connected domains. Where two pixels are adjacent as defined in fig. 2, for pixel x, pixels 1-8 are all their neighbors.
d1, calculating the area of the connected domain: analyzing the area of each connected domain, and if the area (the number of white pixel points) of one connected domain is larger than 30 pixels, determining that the area of the connected domain is too large, namely determining the No. Pi+1The frame contains a moving object and is dropped. If the area of each connected domain is less than or equal to 30 pixels, consider the P-th pixeli+1The frame does not contain a moving object, and the frame is reserved and used as a background image and is marked as Sj，1≤j≤N。
Continue to couple frame Pi+2And frame Pi+1And d, repeating the steps b to d until the monitoring area image without the moving target meets N frames.
(3) Calculating an average value at each pixel point position by using the obtained N frames of background images to obtain an initial background image, and recording the initial background image as B:
where B (x, y) represents the pixel value at the point (x, y) position of the initial background image B, Si(x, y) is shown in the image SiThe pixel value at the midpoint (x, y) location.
(4) In the real-time detection process, each pixel in the current frame monitoring area image is analyzed, if a certain pixel point belongs to a background pixel, the current pixel is updated to the background image in a weighting mode, and the process is as follows:
a2, setting the threshold value for judging whether the pixel belongs to the background pixel as e-20, wherein 20 is an empirical threshold value.
b2, in the process of real-time monitoring, assuming current frame monitoringControl region image is PjThen to PjAnd analyzing pixel by pixel, if the pixel meets the following conditions: | pjIf (x, y) -B (x, y) | < e, performing background updating operation on the pixel point, otherwise, not performing background updating operation on the pixel point. Wherein, Pj(x, y) is represented in the current frame PjAnd (5) the size of the corresponding pixel value at the middle pixel point (x, y).
c2, for the initial background image B, the image information is relatively stable, so in the process of updating the background, it should keep a large proportion, the background updating operation of a certain pixel point (x, y) in the current frame monitoring area image is:
B(x,y)＝B(x,y)*0.9+0.1*Pj(x,y)。
as a further preferable scheme, after step S101, the method further includes:
extracting the current image and the initial background image acquired by the camera according to the set area range to obtain the current image and the initial background image with the reduced range;
correspondingly, the differentiating the image currently shot by the camera with the initial background image to obtain a foreground image includes:
and carrying out difference on the image with the reduced range and the initial background image with the reduced range to obtain a foreground image.
It should be noted that, for the intelligent automatic door system, a camera is provided right above the automatic door to monitor the surrounding environment, and detect whether people pass around, thereby controlling the opening of the intelligent automatic door. However, the monitoring area of the camera is not limited to the corridor but also includes the area where people do not appear. These areas where people are not present can increase the time the system diagnoses whether opening of the automatic door is required. Based on this, the present embodiment manually sets a region range (the region range is set according to experience and practical situations) for the image frames acquired by the camera in real time, and people's activities will rarely occur outside the region range. The image frame is cut by utilizing the area range, so that the detection speed can be greatly improved, and unnecessary interference is reduced.
As a further preferable mode, in step S103: in the foreground image, detect and fix a position the pedestrian, obtain the central point position that the pedestrian belongs to the regional correspondence, specifically include:
(1) and training the neural network fast-RCNN by utilizing the first training data set to obtain the pedestrian detector. The first training data set comprises a public data set and a data set collected by the first training data set, wherein the data set is an image of a single pedestrian, the scale of the data set is 10000 images of the pedestrian, and the specific training steps are as follows:
A. the 10000 pedestrian images are manually calibrated to obtain any image DiThe central point position of the middle pedestrian, namely the abdomen area position of the pedestrian, is marked as Yi10000 groups (D) were finally obtainedi,Yi) And (4) carrying out pairing.
B. With group 10000 of DiAs input 10000 groups YiAs an output, and DiInput represented by and YiThe output represented is a set of mapping data to fit a mapping relationship F. After the mapping F is obtained, for any given input, the result, i.e., the output of the mapping F, is predicted based on the mapping F.
In this embodiment, a BP (back propagation) algorithm is adopted to fit the mapping relationship. For fast-RCNN, which is a network composed of multiple neurons, as shown in fig. 3, the BP algorithm for a single neuron is as follows:
it should be noted that the structure of the simple small neural network can be shown in fig. 4, where each circle represents a neuron, w1And w2Representing the weight between neurons, b representing the bias, g (z) being the activation function, such that the output becomes non-linear, a representing the output, x1And x2Representing the input, then for the current structure, the output can be represented as:
a＝g(x1*w1+x2*w2+1*b)，
it can be seen that the value a of the neural network output is related to the weight and bias with the input data and activation function unchanged. By adjusting different weights and biases, the output of the neural network will also have different results.
Therefore, the mapping F is obtained from weights and offsets between neurons, and the mapping F is different for weights and offsets of different values. Thus, the process of fitting the mapping relationship F: a weight and an offset are found so that the mapping F is optimal under such weight and offset. I.e. input D for a given first training set of samplesiF (D) obtainedi) And YiThe error is minimal.
For a known neural network output value (predicted value) a, assuming that its corresponding true value is a', its BP algorithm performs the following process:
(B-1) randomly initializing per-connection line weight (w)1And w2) And an offset b;
(B-2) for input data x1、x2The BP algorithm executes forward transmission to obtain a predicted value a;
(B-3) estimating the error between the true value a' and the predicted value a
The inverse feedback updates the weight of each connecting line and the bias of each layer in the neural network. The updating method of the weight and the bias is as follows:
i.e. using E to calculate w respectively1、w2b, wherein η represents the learning rate and is a preset parameter.
(B-4) continuously repeating the steps (B-1) to (B-3) until the network converges, namely the value of E is minimum or basically kept unchanged, which indicates that the network is trained completely.
(2) After the training of the fast-RCNN (which is an existing pedestrian detection framework and does not need to be constructed), the image S acquired by the camera in real timeiAfter cropping, the area image of the reduced area obtained is recorded as Ci. In the image CiIn (C), there may be a plurality of pedestrians, and thus, from (C)iA region part of each pedestrian is extracted, and then each individual pedestrian region will be used as an input of an individual fast-RCNN. The specific implementation is as follows:
A. frame difference with the initial background image: for the initial background image B, the clipping operation is also carried out to obtain the initial background image with the reduced range, which is marked as Bs. At this time BsAnd CiIs the same in size, a frame difference operation can be performed, and the frame difference result is recorded as ZiThe frame difference principle is the same as above.
B. And (3) binarization operation: to frame difference image ZiPerforming binarization operation, wherein the threshold value is 20 (empirical threshold value), and the image after binarization is recorded as Ei。
C. Extracting a connected domain: for image EiAnd performing the operation of generating the connected domain.
D. Judging the shape and size: for EiAnd (3) solving an external rectangle of each connected domain of the image, if the area of the external rectangle of the connected domain is larger than 100 pixels (experience threshold value), determining that the connected domain corresponds to the pedestrian, recording the coordinates of the upper left corner of the external rectangle, keeping the coordinates, and otherwise, deleting the connected domain.
For the connected component left, since the coordinates of the top left corner of its bounding rectangle are known, it is in image C according to its coordinate position and rectangle sizeiCorresponding regions are extracted, and the extracted regions are used as the neural network input of the fast-RCNN.
(3) After prediction by the fast-RCNN, the output of the network represents the location of the pedestrian's center point in the input image.
In practical applications, since the walking of the pedestrian is continuous, the pedestrian can be tracked in continuous multiple frames, and the walking direction of the pedestrian is obtained, then the step S104: according to the corresponding central point position of the pedestrian region in the continuous multi-frame images, tracking the position of the pedestrian to obtain the walking direction of the pedestrian, and the method specifically comprises the following steps:
(1) in continuous multi-frame collected images, the current frame is assumed to be Pi，PijThe j-th pedestrian detection area under the current frame. Will PijAfter the prediction is sent to the fast-RCNN, the output data of the penultimate layer of the network is recorded and marked as Tij(which is stored in the form of a vector) as a feature, thereby screening out the motion region of the same person.
(2) Suppose a frame image PiHas detected the center position of each pedestrian region, Pi+1J pedestrian areas are also arranged in the frame image, the central position of each pedestrian area is detected, the pedestrian areas belonging to the same person in adjacent frames are associated, and then the trend track of the pedestrian can be obtained.
In this embodiment, a minimum distance method is adopted to realize matching of the same pedestrian region in adjacent frames, and the specific process is as follows:
A. known frame PiIs characterized by TijFind frame PiThe k-th pedestrian region and frame Pi+1Degree of difference X of the first pedestrian zoneklThe calculation formula is as follows:
wherein, TikoRepresents a feature TikM represents the total number of components of the feature vector.
B. For frame PiThe k-th pedestrian area is obtained and the k-th pedestrian area and the frame P are obtainedi+1Recording the difference of all pedestrian areas, and recording the frame P corresponding to the difference when the difference obtains the minimum valuei+1The number of the pedestrian area is marked as g. Event frame PiThe k-th pedestrian region and frame Pi+1Middle g pedestrian areaThe two areas correspond to each other one by one and meet matching requirements.
C. Sequentially couple frames PiAnd (C) performing the steps (A) to (B) on the pedestrian areas remained in the image so as to search corresponding matching areas.
(3) And associating the areas of the same pedestrian in the adjacent frames, and connecting the central position of the area corresponding to the same pedestrian in the adjacent frames, namely the output of the fast-RCNN to obtain the moving track of a certain pedestrian.
(4) And acquiring the direction of the pedestrian according to the moving track. The method specifically comprises the following steps: if the track lines are approximately parallel to the horizontal direction, the pedestrian has no intention of going out, the area corresponding to the pedestrian is deleted from the original image, interference is reduced, and a foundation is laid for more accurately judging whether the pedestrian is going out.
As shown in fig. 5, as a further preferable scheme, after the step of judging that the walking direction of the pedestrian is parallel to the horizontal direction, the embodiment further includes the following steps:
(a) acquiring a moving track of a central point position corresponding to an area where a pedestrian is located in continuous multi-frame images to obtain the walking speed of the pedestrian;
(b) judging whether the walking speed of the pedestrian is less than a set threshold value, if so, executing the step (c), otherwise, executing the step (f);
(c) analyzing the orientation of the pedestrian, judging whether the orientation of the pedestrian face is a front face, if so, executing the step (d), and if not, executing the step (f);
(d) analyzing the deflection angle of the face of the pedestrian, and judging whether the deflection angle of the face of the pedestrian is within a set range, if so, executing the step (e), otherwise, executing the step (f);
(e) controlling the intelligent automatic door to open;
(f) the intelligent automatic door does not open.
As a further preferable scheme, the process of analyzing the walking speed of the pedestrian comprises the following steps:
(1) continuously acquiring the moving track of a certain pedestrian in a continuous 10-frame range, and recording the initial point and the end point of the track.
(2) Calculating the average speed V of the pedestrian according to the following formula:
wherein L is the difference between the distances from the end point of the track to the start point.
(3) If the average velocity V is greater than 5 pixels (empirical threshold), the pedestrian is considered to be a walking person in the hallway without opening the door, otherwise, the pedestrian orientation is analyzed.
In practical applications, the speed of a person to be taken out is different from the speed of a person walking straight on the corridor, and thus whether the person is going out can be detected by judging the speed of the pedestrian.
As a further preferred solution, the pedestrian orientation is analyzed as follows:
the difference between the front face image and the side face image of the human face is large, and the orientation of the current human face is judged to be the front face or the side face through content analysis of the human face image. Similarly, we adopt the neural network to perform the following judgment task, which is implemented as follows:
(1) constructing a second neural network: the second neural network is constructed to have 5 layers, wherein the first layer is an input layer and is 300 neurons, the second layer to the fourth layer are hidden layers and respectively contain 200, 400 and 400 neurons, and the last layer is an output layer and is 2 neurons.
(2) Constructing a second training data set: in this data set, there were 6000 data sets including 3000 front face images and 3000 side face images.
(3) The second neural network is trained using the second training data set, and then the weights are updated using the BP algorithm, the principle being the same as the method for training the fast-RCNN in the above steps.
(4) After the network training is finished, the image can be predicted, whether the image belongs to a human side face or a front face is judged, and the specific process is as follows:
A. in the current image frame, the position and the size of the pedestrian are detected, and then the pedestrian area is extracted and recorded as R.
B. The image R is normalized to a size of 30 × 10, and then the normalized image is used as the input of the neural network (each pixel point exactly corresponds to one neuron of the input layer of the neural network, and a total of 300 pixels exactly correspond to 300 neurons of the input layer of the neural network).
C. The output of the neural network is a two-dimensional vector, the first component representing the proportion of the input data belonging to the front face and the second component representing the proportion of the input data belonging to the side face.
(5) The value with the highest ratio is selected as the result of the neural network. If the neural network judges that the current face orientation is the side face, the operation is ended, and the automatic door is informed without opening. And if the neural network judges that the current face orientation is the front face, analyzing the face deflection angle.
It should be noted that after the face orientation image is obtained, the degree of the included angle between the face orientation and the front side (the direction opposite to the camera) needs to be further determined, for a pedestrian who wants to enter or exit the automatic door, the face orientation and the camera direction (the camera is arranged right above the automatic door) meet a certain degree of the included angle, if the degree of the included angle is too large, the automatic door is not opened, and the pedestrian does not intend to exit the door. The included angle degree between the face orientation and the front is judged by adopting a neural network mode, and the specific process is as follows:
(1) constructing a third neural network: the constructed neural network has 5 layers, the first layer is an input layer and is 200 neurons, the second layer to the fourth layer are hidden layers and respectively contain 300, 300 and 400 neurons, and the last layer is an output layer and is 1 neuron.
(2) Constructing a third training data set: the data set contains 5000 images with faces in different orientations.
(3) And training the third neural network, specifically updating the weight and the like by using a BP algorithm, wherein the training process is the same as that of the training fast-RCNN.
(4) After the neural network training is finished, the images are predicted by using the neural network training, and the specific orientation angle of the face is judged, wherein the specific judgment process comprises the following steps.
Cutting out the image region R, leaving only the region above R
Part (the human head approximately occupies the whole body length
) And is denoted as TH. Since the image region R is the entire region information of the human body, the upper part thereof is extracted
And a region where the completed header information can be basically acquired.
Normalizing the image TH to be 20 × 10, and then taking the normalized image as the input of the neural network (each pixel point exactly corresponds to one neuron of the input layer of the neural network, and the total number of the pixels is 200, and exactly corresponds to 200 neurons of the input layer of the neural network).
The output of the neural network is the face orientation angle corresponding to the input image.
If the output result value is within the range of plus or minus 20 ° (positive in the clockwise direction), the automatic door is notified to open, otherwise the automatic door is not opened.
It should be noted that, in the method for controlling an intelligent automatic door based on image analysis provided in this embodiment, the image of the monitored area is acquired by the camera, and the walking direction, the walking speed, the face orientation, and the face deflection angle of the pedestrian in the image are analyzed, so that the departure intention of the pedestrian in the monitored area can be accurately determined, thereby controlling the opening of the intelligent automatic door, and avoiding the damage of the intelligent automatic door due to frequent and incorrect opening.
In practical applications, in order to deal with an unexpected situation, which results in that the door cannot be opened automatically (for example, the above-mentioned judgment incorrectly determines that the person is not coming to the door, and the automatic door is not opened), the embodiment provides an emergency strategy, that is, the door is opened by waving one's hand. As shown in fig. 6, the hand waving detection is implemented as follows:
(1) and aiming at the current difference between the current image and the background, obtaining a foreground area Q of the human body.
(2) And carrying out the pedestrian detection on the foreground region, and obtaining the body trunk region of the human body according to the detection result.
(3) For the foreground region Q in step 1, the torso region of the body obtained in step 2 is deleted from Q, and the deleted region basically only has the arm region of the human body, which is denoted as H.
(4) The hand region H is subjected to the thresholding operation and the generation connected component operation, and the threshold value is 20 (empirical threshold).
(5) And (4) searching the connected domain (the arm region corresponding to the connected domain, and the other connected domains with smaller areas are noise regions) corresponding to the largest connected domain after the operation of the step (4), and recording as Y.
(6) And solving a circumscribed rectangle for the connected domain Y, if the length of the circumscribed rectangle in the horizontal direction is smaller than that in the vertical direction, the meaning represented by the connected domain Y is considered to be waving (when waving, the arm is in the vertical direction), and telling that the automatic door needs to be opened, otherwise, the automatic door does not need to be opened.
It should be noted that, in this embodiment, through increasing and wave hand detection, all wrong in above-mentioned judgement, lead to unable automatic door opening, the accessible waves hand detection to the pedestrian and controls to open the automatically-controlled door, has further improved the accuracy that the automatically-controlled door opened to and improved user experience.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (9)
1. An intelligent automatic door control method based on image analysis is characterized by comprising the following steps:
carrying out background modeling on a monitoring area of a camera installed on the intelligent automatic door to obtain an initial background image;
differentiating the image currently acquired by the camera with the initial background image to obtain a foreground image;
detecting and positioning the pedestrian in the foreground image to obtain a central point position corresponding to the region where the pedestrian is located;
tracking the position of the pedestrian according to the position of the corresponding central point of the area where the pedestrian is located in the continuous multi-frame images to obtain the walking direction of the pedestrian;
judging whether the walking direction of the pedestrian is parallel to the horizontal direction, wherein the horizontal direction is the direction of the pedestrian entering and exiting from the automatic door;
if so, controlling the intelligent automatic door to open the door, otherwise, not opening the door;
the walking direction of the pedestrian is parallel to the horizontal direction, and the horizontal direction is the direction from the automatic door to the people's business turn over after, the pedestrian protection device also comprises:
(a) acquiring a moving track of a central point position corresponding to an area where a pedestrian is located in continuous multi-frame images to obtain the walking speed of the pedestrian;
(b) judging whether the walking speed of the pedestrian is less than a set threshold value, if so, executing the step (c), otherwise, executing the step (f);
(c) analyzing the orientation of the pedestrian, judging whether the orientation of the pedestrian face is a front face, if so, executing the step (d), and if not, executing the step (f);
(d) analyzing the deflection angle of the face of the pedestrian, and judging whether the deflection angle of the face of the pedestrian is within a set range, if so, executing the step (e), otherwise, executing the step (f);
(e) controlling the intelligent automatic door to open;
(f) the intelligent automatic door does not open.
2. The intelligent automatic door control method based on image analysis as claimed in claim 1, wherein after the background modeling is performed on the monitoring area of the camera installed on the intelligent automatic door to obtain the initial background image, the method further comprises:
extracting the current image and the initial background image acquired by the camera according to the set area range to obtain the current image and the initial background image with the reduced range;
correspondingly, the differentiating the image currently shot by the camera with the initial background image to obtain a foreground image includes:
and carrying out difference on the image with the reduced range and the initial background image with the reduced range to obtain a foreground image.
3. The method for controlling an intelligent automatic door based on image analysis according to claim 2, wherein the background modeling is performed on the monitoring area of the camera installed on the intelligent automatic door to obtain a background image, and specifically comprises:
acquiring monitoring area images of a camera to acquire N frames of monitoring area images;
detecting whether a moving target exists in the monitored area image by adopting a frame difference method;
if so, acquiring the monitoring area image of the camera again until N frames of monitoring area images without moving targets are acquired as background images;
averaging pixels at the same pixel point position in the N frames of background images to obtain an initial background image;
analyzing each pixel point in the collected current frame monitoring area image, and judging whether the pixel point belongs to the initial background image or not;
and if a certain pixel point belongs to the background pixel, updating the pixel point into the initial background image in a weighting mode so as to realize the updating of the initial background image.
4. The method for controlling an intelligent automatic door based on image analysis according to claim 2, wherein the detecting and positioning of the pedestrian in the foreground image to obtain the position of the central point corresponding to the area where the pedestrian is located specifically comprises:
training the constructed first neural network by using a first training data set formed by a single pedestrian image sample to obtain a pedestrian detector;
extracting the region part of each pedestrian from the collected image with the reduced range;
and inputting each individual pedestrian area as an individual pedestrian detector, and outputting the central point position of the pedestrian in the current collected image after the prediction of the pedestrian detector.
5. The method for controlling an intelligent automatic door based on image analysis according to claim 4, wherein the tracking of the position of the pedestrian is performed according to the position of the corresponding center point of the area where the pedestrian is located in the continuous multi-frame image, so as to obtain the walking direction of the pedestrian, specifically comprising:
sending all pedestrian areas in continuous multi-frame collected images to a pedestrian detector for prediction to obtain a central point position corresponding to each pedestrian area;
matching pedestrian areas in adjacent frames by adopting a minimum distance method to obtain pedestrian areas belonging to the same pedestrian in the adjacent frames;
associating pedestrian areas belonging to the same person in adjacent frames, and connecting the central positions of the areas corresponding to the same pedestrian in the adjacent frames to obtain the moving track of the pedestrian;
and judging the walking direction of the pedestrian according to the moving track of the pedestrian.
6. The method for controlling an intelligent automatic door based on image analysis according to claim 5, wherein the obtaining of the moving track of the central point position corresponding to the area where the pedestrian is located in the continuous multi-frame image to obtain the walking speed of the pedestrian specifically comprises:
continuously acquiring the moving track of a certain pedestrian in the continuous M-frame image range, and recording the initial point and the end point of the track;
and dividing the difference value of the distances between the track end point and the starting point by the number of the frame images to obtain the average speed of the pedestrian.
7. The method for intelligent automatic door control based on image analysis according to claim 2, wherein the analyzing the pedestrian orientation specifically comprises:
constructing a second neural network and a second training data set, and training the constructed second neural network by using the second training data set to obtain a face orientation detector;
normalizing the input image frame, and taking the normalized image as the input of a human face orientation detector to obtain the proportion that the human face orientation in the input image frame is a front face and the proportion that the human face orientation is a side face;
the face orientation with a higher ratio is selected as the detection result of the face orientation detector from the ratio of the face orientation to the front and the ratio of the face orientation to the side.
8. The method for controlling an intelligent automatic door based on image analysis according to claim 2, wherein the analyzing the deflection angle of the face of the pedestrian specifically comprises:
constructing a third neural network and a third training data set, and training the third neural network by using the third training data set to obtain a human face deflection angle detector;
cutting an input image frame to obtain a local image area with the head of a pedestrian;
normalizing the local image area to obtain a normalized local image area;
and taking the normalized local image area as the input of a human face deflection angle detector to obtain the human face deflection angle.
9. The intelligent automatic door control method based on image analysis as claimed in any one of claims 1 to 8, wherein after detecting and positioning the pedestrian in the foreground image and obtaining the center point position corresponding to the area where the pedestrian is located, the method further comprises:
cutting the pedestrian area, and reserving the arm area of the human body;
carrying out thresholding and connected domain extraction operations on an arm region of a human body to obtain all connected domains of the arm region;
selecting a connected domain corresponding to the largest area of the connected domain, and solving a circumscribed rectangle of the connected domain;
when the length of the horizontal direction of the external rectangle is smaller than the angle of the vertical direction, the intelligent automatic door is controlled to be opened.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711480203.5A CN108222749B (en) | 2017-12-29 | 2017-12-29 | Intelligent automatic door control method based on image analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711480203.5A CN108222749B (en) | 2017-12-29 | 2017-12-29 | Intelligent automatic door control method based on image analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108222749A CN108222749A (en) | 2018-06-29 |
| CN108222749B true CN108222749B (en) | 2020-10-02 |
Family
ID=62647081
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711480203.5A Active CN108222749B (en) | 2017-12-29 | 2017-12-29 | Intelligent automatic door control method based on image analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108222749B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109636956A (en) * | 2018-10-26 | 2019-04-16 | 深圳云天励飞技术有限公司 | A kind of access control system control method, device and electronic equipment |
| CN110316630B (en) * | 2019-06-03 | 2020-12-29 | 浙江新再灵科技股份有限公司 | Deviation early warning method and system for installation angle of elevator camera |
| CN112850436A (en) * | 2019-11-28 | 2021-05-28 | 宁波微科光电股份有限公司 | Pedestrian trend detection method and system of elevator intelligent light curtain |
| CN112861593A (en) * | 2019-11-28 | 2021-05-28 | 宁波微科光电股份有限公司 | Elevator door pedestrian detection method and system, computer storage medium and elevator |
| CN112562139B (en) * | 2020-10-14 | 2023-02-17 | 深圳云天励飞技术股份有限公司 | Access control method and device based on image recognition and electronic equipment |
| CN112560610B (en) * | 2020-12-03 | 2021-09-28 | 西南交通大学 | Video monitoring object analysis method, device, equipment and readable storage medium |
| CN113501398B (en) * | 2021-06-29 | 2022-08-23 | 江西晶浩光学有限公司 | Control method, control device and storage medium |
| CN113668976A (en) * | 2021-07-16 | 2021-11-19 | 广州大学 | Novel intelligence prevents trampling escape door system |
| CN114019835B (en) * | 2021-11-09 | 2023-09-26 | 深圳市雪球科技有限公司 | Automatic door opening method and system, electronic equipment and storage medium |
| CN114333134B (en) * | 2022-03-10 | 2022-05-31 | 深圳灏鹏科技有限公司 | Cabin management method, device, equipment and storage medium |
| CN116591575B (en) * | 2023-07-18 | 2023-09-19 | 山东锐泽自动化科技股份有限公司 | Rotary door safety control method and system based on machine vision |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463903A (en) * | 2014-06-24 | 2015-03-25 | 中海网络科技股份有限公司 | Pedestrian image real-time detection method based on target behavior analysis |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102747919B (en) * | 2012-06-18 | 2014-11-12 | 浙江工业大学 | Omnidirectional computer vision-based safe and energy-saving control device for pedestrian automatic door |
| US10533850B2 (en) * | 2013-07-12 | 2020-01-14 | Magic Leap, Inc. | Method and system for inserting recognized object data into a virtual world |
| CN104751491B (en) * | 2015-04-10 | 2018-01-23 | 中国科学院宁波材料技术与工程研究所 | A kind of crowd's tracking and people flow rate statistical method and device |
| CN105869185A (en) * | 2016-04-15 | 2016-08-17 | 张志华 | Automatic door |
| CN106529442B (en) * | 2016-10-26 | 2019-10-18 | 清华大学 | A kind of pedestrian recognition method and device |
-
2017
- 2017-12-29 CN CN201711480203.5A patent/CN108222749B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104463903A (en) * | 2014-06-24 | 2015-03-25 | 中海网络科技股份有限公司 | Pedestrian image real-time detection method based on target behavior analysis |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108222749A (en) | 2018-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108222749B (en) | Intelligent automatic door control method based on image analysis | |
| CN109344725B (en) | Multi-pedestrian online tracking method based on space-time attention mechanism | |
| CN109522793B (en) | Method for detecting and identifying abnormal behaviors of multiple persons based on machine vision | |
| CN107545582B (en) | Video multi-target tracking method and device based on fuzzy logic | |
| CN108830144B (en) | Lactating sow posture identification method based on improved Faster-R-CNN | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN110399808A (en) | A kind of Human bodys' response method and system based on multiple target tracking | |
| CN110135314A (en) | A kind of multi-object tracking method based on depth Trajectory prediction | |
| CN111310659B (en) | Human body action recognition method based on enhanced graph convolution neural network | |
| Scheutz et al. | Fast, reliable, adaptive, bimodal people tracking for indoor environments | |
| Treptow et al. | Real-time people tracking for mobile robots using thermal vision | |
| CN107977646B (en) | Partition delivery detection method | |
| CN110765906A (en) | Pedestrian detection algorithm based on key points | |
| CN108596087B (en) | Driving fatigue degree detection regression model based on double-network result | |
| CN108320300A (en) | A kind of space-time context visual tracking method of fusion particle filter | |
| CN109993770B (en) | Target tracking method for adaptive space-time learning and state recognition | |
| KR20060018061A (en) | Method and apparatus for tracking human using robot | |
| Zhang et al. | Object detection/tracking toward underwater photographs by remotely operated vehicles (ROVs) | |
| CN111931654A (en) | Intelligent monitoring method, system and device for personnel tracking | |
| JP4978227B2 (en) | Image detection device | |
| CN114916964B (en) | Pharynx swab sampling effectiveness detection method and self-service pharynx swab sampling method | |
| Tuominen et al. | Cloud detection and movement estimation based on sky camera images using neural networks and the Lucas-Kanade method | |
| CN111860297A (en) | SLAM loop detection method applied to indoor fixed space | |
| CN112233145A (en) | Multi-target shielding tracking method based on RGB-D space-time context model | |
| CN116051970A (en) | Identification method for overlapping fish targets based on improved yolov5 model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 230000 Yafu Park, Juchao Economic Development Zone, Chaohu City, Hefei City, Anhui Province Applicant after: ANHUI HUISHI JINTONG TECHNOLOGY Co.,Ltd. Address before: 102, room 602, C District, Hefei National University, Mount Huangshan Road, 230000 Hefei Road, Anhui, China Applicant before: ANHUI HUISHI JINTONG TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |