CN110298278A - A kind of underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence - Google Patents
A kind of underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence Download PDFInfo
- Publication number
- CN110298278A CN110298278A CN201910535807.8A CN201910535807A CN110298278A CN 110298278 A CN110298278 A CN 110298278A CN 201910535807 A CN201910535807 A CN 201910535807A CN 110298278 A CN110298278 A CN 110298278A
- Authority
- CN
- China
- Prior art keywords
- car owner
- vehicle
- pedestrian
- artis
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Abstract
The underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence that the invention discloses a kind of, underground parking garage also lacks the plateform system that effective people's car test is surveyed and is managed collectively at present, and this method in addition to can to car owner and vehicle by face recognition module and vehicle attribute identification module collecting cart advocate peace driven vehicle attribute information and carry out corresponding label connection, and if in a certain moment in parking, identify that face verification is inconsistent, the detection for carrying out human body attitude identifies skeleton key point information, and the detection and identification of abnormal behaviour are carried out by skeleton artis information, it can effectively identify and for example fall, open the car door abnormal behaviours movement such as into the car, it can be with the vehicle of effective guarantee car owner and personal safety.
Description
Technical field
The present invention relates to the technical fields such as computer vision, pattern-recognition, deep learning, database, Android platform exploitation
Management system, especially a kind of underground garage Pedestrians and vehicles monitoring method based on artificial intelligence.
Background technique
As intelligent video monitoring is increasingly becoming an emerging application direction of smart city cell.Its intelligence is not
Human eye only is replaced with video camera, and replaces people, contributor with computer, to complete monitoring or control task and mitigate people
Burden.A large amount of human and material resources and financial resources are not only saved in this way, it is often more important that it can find in monitoring scene in time
Unusual condition avoid the generations of all kinds of anomalous events.
Garage parking in the cell of smart city is the high frequency area that pedestrian's abnormal behaviour occurs.But current smart city parking
Library lacks the platform that pedestrian and vehicle are managed collectively and are monitored and system.
Deep learning is a new field in machine learning research, and motivation is that foundation, simulation human brain are divided
Analysis study.Deep learning is a kind of data driven type model, can simulate human brain vision mechanism automatically learn to data it is each
The abstract characteristics of level, to preferably reflect the substantive characteristics of data, so that the abnormal behaviour and vehicle of detection identification pedestrian
Information becomes more accurate and efficient.
Summary of the invention
The object of the present invention is to provide a kind of the underground garage Pedestrians and vehicles monitoring method based on artificial intelligence, real-time detection
The abnormal behaviour in vehicle periphery pedestrian occurs, allows to preferably carry out vehicle room entry/exit management, vehicle to the region anti-
Steal management, people's vehicle information management.
The present invention is up to foregoing invention purpose, and specific technical solution is as follows:
A kind of underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence, which comprises the steps of:
Step 1, monitoring camera acquire the picture on each parking stall in garage parking in real time, if there is vehicle stops into number to be n
Parking stall, then picture transmission to server can be obtained the vehicle of the parking stall by the Car license recognition model on server by monitoring system
Board number information Vn.When car owner gets off, face recognition algorithms and pedestrian's weight recognizer on server will extract vehicle respectively
Main face characteristic FnWith human external feature P1, and by car owner's face characteristic Fn, pedestrian's feature P1, the license plate driven of car owner
Information VnMark In, it is denoted as record in the server.
Step 2 uses " a key-lock vehicle " function by mobile phone parking management app on the mobile apparatus after car owner has stopped vehicle
The position of vehicle parking can virtually can be locked (i.e. on app), simultaneity factor unlocking vehicle moves detection model to determine to supervise
Whether vehicle has occurred movement in the picture that control camera acquires in real time.If vehicle unlocks " one not on mobile phone app in car owner
Movement has occurred in the case where key positioning " function, the mobile phone that monitoring system will send immediately stolen warning message to car owner stops
On vehicle management system app, to notify the car owner of the vehicle.
Step 3, if still acquisition each stops monitoring camera in real time without using " a key-lock vehicle " function after car owner's parking
The picture of parking stall is simultaneously transferred on server, is extracted in a certain range by pedestrian's weight identification model close to parked vehicles pedestrian
Feature.If the feature of the pedestrian does not meet the surface of the reserved corresponding car owner of the vehicle in the server, lead to simultaneously
It crosses the behavior act identification model based on skeleton to detect and identify that the pedestrian is made that opening car door enters the movement of vehicle, then
It is determined as abnormal operation.Server is sent in stolen warning message to the mobile phone parking management system app of car owner, to notify the vehicle
Car owner.
Step 4, at entrance gate, after car owner has stopped vehicle, it will generate phase in mobile phone parking management system app
The two dimensional code answered.After car owner picks up the car, unlocking virtual lockout vehicle functions and drive that vehicle reaches after entrance gate can whereby two
Dimension code sweeps that sweep can be by entrance gate on the scanner.
Further, the garage parking Pedestrians and vehicles management system for monitoring based on artificial intelligence, it is characterised in that: institute
It states in step 1, comprising:
Step 1 installs the monitoring camera in covering full cut-off parking lot in parking lot, and monitoring camera real-time Transmission is each stopped
The image transmitting of parking stall is to server-side.
Step 2, enters the license plate of the vehicle on parking stall by Recognition Algorithm of License Plate identification first, and vehicle stops into parking stall
Picture is input to convolutional neural networks to extract characteristics of image, these features are then input to long short-term memory and recycle nerve net
In network, license board information then is obtained by CTC loss layer.Wherein, long short-term memory circulation nerve net is obtained from each time step
Network exports and generates the probability value of the letter character of prediction, then removes continuous repeat character (RPT), and deletes the spies such as space
Different character is finally merged into a character string output, that is, our prediction result Vn。
Step 3 then identifies pedestrian's feature by pedestrian's weight recognizer, and the car owner for being transferred to server gets off picture
The characteristic pattern in 1024 channels is generated as the network of convolutional neural networks model by ResNet-50, the feature of generation is desired to make money or profit
Pedestrian's feature is converted with 512 × 3 × 3 convolutional layer, it then will using 9 target frames in each position of characteristic pattern
Original image pixels be converted into a new characteristic pattern and a Soft-max classifier predict each target frame whether include
Including pedestrian, while a linear regressor is further comprised to adjust the position of target frame.Only retain after non-maximum restraining
Then 128 bounding boxes adjusted extract pedestrian's feature P of car owner as final region by ResNet-501。
Step 4, car owner get off picture simultaneously also by Deep ID face recognition algorithms, extract the face characteristic of car owner
F1。
Step 5 after detecting region of the car owner in picture, obtains car owner's skeleton joint point using Open pose algorithm
Characteristic information Jn1。
Step 6, by license plate prediction result V, pedestrian's feature P of car owner1With the characteristic information f of car owner's skeleton joint pointn1,
One-to-one label is carried out according to parking stall numbering n, is denoted as In={ Pn1, Fn1, Jn1,Vn, and saved in server, directly
It is removed after being driven out to vehicle.
Further, the garage parking Pedestrians and vehicles management system for monitoring based on artificial intelligence, it is characterised in that: institute
It states in step 3, further includes:
Every frame image on each parking stall of monitoring camera real-time Transmission is to server-side.When at a time, there is s-th of people
The pedestrian for being n close to number and thering is the picture of the parking stall scene to park cars to extract the pedestrian by pedestrian's weight recognizer
Feature Pns, and pedestrian's feature P with the car owner of the vehiclen1Cosine similarity comparison is carried out, if being less than the threshold value 0.7 of setting, simultaneously
Pass through the face characteristic F of the face recognition algorithms figure area pedestrianns, by the comparison of the two feature cosine similarity, set if being less than
Fixed threshold value 0.7 then determines not being the same person with car owner and detect framework information f by Open pose algorithmns, base
Judge whether the pedestrian has in the behavior act recognizer of human skeleton and the abnormal behaviours movement such as opens car door and fall.
Further, the behavior act recognizer network based on human skeleton, it is characterised in that: the step
In three, further includes:
Step 1, data preparation: collecting pedestrian's tumble video data under underground parking lab environment, and substantially 100 every
A video is that one of 150 frames or so continuously falls action video, and video visual angle is underground parking garage monitoring camera visual angle.It receives
Pedestrian under collection underground parking lab environment opens the sets of video data that car door enters vehicle, and substantially 100 each videos are 150
One continuous action video of frame, video visual angle is underground parking garage monitoring camera visual angle.
Data prediction: ready data are extracted each frame of each video using Open pose algorithm by step 2
The information (x, y, z) of 18 artis of human skeleton (such as head, neck, shoulder, hand etc.), x be artis on the image
Abscissa, y be the ordinate of artis on the image, axle center be the image lower left corner, z be artis the value of the confidence.For one
The video of a batch is indicated with 5 dimensions matrix (N, C, T, V, M).N indicates the quantity of 1 batch training video, and C, which is represented, to close
The feature of section is (x, y, z), and T represents the quantity of a video frame, and V represents the quantity in joint, is 18 joints here, and M is represented
Number in one frame, setting here up to 2.Finally matrix is normalized.
Network structure: step 3 according to spatiality and time domain, is combination with GCN network and TCN network, constructs space
Time-domain diagram convolutional network.It is weight respectively to each section that GCN network, which is to the side A under single frames picture, linked between artis,
Point feature information carries out convolution operation, and the Convolution Formula of artis i is as follows:
N is the artis quantity of adjacent connection, and I is unit matrix,For the degree of artis i,For artis j's
Degree.XjFor the characteristic information (x of j-th of artis under a frame picturej,yj,zj), aggregate (Xi) it is i-th under a frame picture
Artis aggregation information feature.
Formula can simplify are as follows:
Wherein, Λ=∑ (A+I) here, adjacency matrix A and unit matrix I are the link information of single frames bone, then Λ
It is multiframe bone link information model, JinIt is the characteristic information of the skeleton of input, JoutAfter polymerizeing side information for artis
Value.
The operation of TCN is traditional convolution operation, to the J after GCN network polymerization artis informationoutMatrix be (N, C,
T, V, M), the convolution kernel of O × W × 1 is used (C, V, T) therein, completes W row pixel, the convolution of 1 column pixel every time, O is volume
Product core number.
Step 4, model training: after 9 space time-domain diagram convolutional networks, by pond layer and full articulamentum, finally
Model training, mini-batch 16, learning rate 0.01, SGD learning algorithm, training are carried out using soft-max loss function
80 epoch.
Model measurement: step 5 the artis characteristic information of one section of video is input in network, GCN network is passed through
Time convolution operation is carried out to artis information fusion, then by TCN, finally by pond layer and full articulamentum, to artis
Feature is classified, and the result of classification is obtained.
When the result of acquisition, which for example falls, opens car door enters the movement of the abnormal behaviours such as vehicle, then prompting police can be sent
It accuses and reminds car owner in information to the mobile phone app of car owner.When the action behavior for identifying human body has the behavior for tending to illegal operation
Such as open car door and enter vehicle etc. then by issuing early warning warning information to car owner mobile phone app, it reminds and informs car owner.
Compared with prior art, the beneficial effects of the present invention are embodied in:
This method is in addition to that can collect car owner by face recognition module and vehicle attribute identification module to car owner and vehicle
Label connection is corresponded with the information of driven vehicle attribute and carrying out, and if is known in a certain moment in parking
Other face verification is inconsistent, and the detection for carrying out human body attitude identifies skeleton key point information, and is closed by skeleton
Key point information carries out the detection and identification of abnormal behaviour, can effectively identify different into the car such as falling, opening car door
Normal behavior act, the vehicle of effective guarantee car owner and personal safety.
Detailed description of the invention
Fig. 1 is Pedestrians and vehicles monitoring block diagram；
Fig. 2 is parking information collecting system schematic；
Fig. 3 is unusual checking schematic diagram；
Fig. 4 is access management system schematic.
Specific embodiment
The present invention will be further explained below with reference to the attached drawings.
As shown in Figure 1, the underground parking garage Pedestrians and vehicles monitoring method of the invention based on artificial intelligence, including walk as follows
It is rapid:
Step 1, monitoring camera acquire the picture on each parking stall in garage parking in real time, if there is vehicle stops into number to be n
Parking stall, then picture transmission to server can be obtained the vehicle of the parking stall by the Car license recognition model on server by monitoring system
Board number information Vn.When car owner gets off, face recognition algorithms and pedestrian's weight recognizer on server will extract vehicle respectively
Main face characteristic FnWith human external feature P1, and by car owner's face characteristic Fn, pedestrian's feature P1, the license plate driven of car owner
Information VnMark In, it is denoted as record in the server.
Step 2 uses " a key-lock vehicle " function by mobile phone parking management app on the mobile apparatus after car owner has stopped vehicle
The position of vehicle parking can virtually can be locked (i.e. on app), simultaneity factor unlocking vehicle moves detection model to determine to supervise
Whether vehicle has occurred movement in the picture that control camera acquires in real time.If vehicle unlocks " one not on mobile phone app in car owner
Movement has occurred in the case where key positioning " function, the mobile phone that monitoring system will send immediately stolen warning message to car owner stops
On vehicle management system app, to notify the car owner of the vehicle.
Step 3, if still acquisition each stops monitoring camera in real time without using " a key-lock vehicle " function after car owner's parking
The picture of parking stall is simultaneously transferred on server, is extracted in a certain range by pedestrian's weight identification model close to parked vehicles pedestrian
Feature.If the feature of the pedestrian does not meet the surface of the reserved corresponding car owner of the vehicle in the server, lead to simultaneously
It crosses the behavior act identification model based on skeleton to detect and identify that the pedestrian is made that opening car door enters the movement of vehicle, then
It is determined as abnormal operation.Server is sent in stolen warning message to the mobile phone parking management system app of car owner, to notify the vehicle
Car owner.
Step 4, at entrance gate, after car owner has stopped vehicle, it will generate phase in mobile phone parking management system app
The two dimensional code answered.After car owner picks up the car, unlocking virtual lockout vehicle functions and drive that vehicle reaches after entrance gate can whereby two
Dimension code sweeps that sweep can be by entrance gate on the scanner.
In the step 1, comprising:
Step 1.1, the monitoring camera in covering full cut-off parking lot is installed in parking lot, monitoring camera real-time Transmission is each stopped
The image transmitting of parking stall is to server-side.
Step 1.2, enter the license plate of the vehicle on parking stall by Recognition Algorithm of License Plate identification first, vehicle stops into parking stall
Picture is input to convolutional neural networks to extract characteristics of image, these features are then input to long short-term memory and recycle nerve net
In network, license board information then is obtained by CTC loss layer.Wherein, long short-term memory circulation nerve net is obtained from each time step
Network exports and generates the probability value of the letter character of prediction, then removes continuous repeat character (RPT), and deletes the spies such as space
Different character is finally merged into a character string output, that is, our prediction result Vn。
Step 1.3, pedestrian's feature is then identified by pedestrian's weight recognizer, the car owner for being transferred to server gets off picture
The characteristic pattern in 1024 channels is generated as the network of convolutional neural networks model by ResNet-50, the feature of generation is desired to make money or profit
Pedestrian's feature is converted with 512 × 3 × 3 convolutional layer, it then will using 9 target frames in each position of characteristic pattern
Original image pixels be converted into a new characteristic pattern and a Soft-max classifier predict each target frame whether include
Including pedestrian, while a linear regressor is further comprised to adjust the position of target frame.Only retain after non-maximum restraining
Then 128 bounding boxes adjusted extract pedestrian's feature P of car owner as final region by ResNet-501。
Step 1.4, car owner gets off picture simultaneously also by Deep ID face recognition algorithms, extracts the face characteristic of car owner
F1。
Step 1.5, after detecting region of the car owner in picture, car owner's skeleton joint is obtained using Open pose algorithm
The characteristic information J of pointn1。
Step 1.6, by license plate prediction result V, pedestrian's feature P of car owner1With the characteristic information f of car owner's skeleton joint pointn1,
One-to-one label is carried out according to parking stall numbering n, is denoted as In={ Pn1, Fn1, Jn1,Vn, and saved in server, directly
It is removed after being driven out to vehicle.
In the step 3, further includes:
Every frame image on each parking stall of monitoring camera real-time Transmission is to server-side.When at a time, there is s-th of people
The pedestrian for being n close to number and thering is the picture of the parking stall scene to park cars to extract the pedestrian by pedestrian's weight recognizer
Feature Pns, and pedestrian's feature P with the car owner of the vehiclen1Cosine similarity comparison is carried out, if being less than the threshold value 0.7 of setting, simultaneously
Pass through the face characteristic F of the face recognition algorithms figure area pedestrianns, by the comparison of the two feature cosine similarity, set if being less than
Fixed threshold value 0.7 then determines not being the same person with car owner and detect framework information f by Open pose algorithmns, base
Judge whether the pedestrian has in the behavior act recognizer of human skeleton and the abnormal behaviours movement such as opens car door and fall.
In the step 3, further includes:
Step 3.1, data preparation: collecting pedestrian's tumble video data under underground parking lab environment, and substantially 100 every
A video is that one of 150 frames or so continuously falls action video, and video visual angle is underground parking garage monitoring camera visual angle.It receives
Pedestrian under collection underground parking lab environment opens the sets of video data that car door enters vehicle, and substantially 100 each videos are 150
One continuous action video of frame, video visual angle is underground parking garage monitoring camera visual angle.
Step 3.2, ready data data prediction: are extracted into each each frame of video using Open pose algorithm
18 artis of human skeleton (such as head, neck, shoulder, hand etc.) information (x, y, z), x be artis in image
On abscissa, y be the ordinate of artis on the image, axle center be the image lower left corner, z be artis the value of the confidence.For
The video of one batch is indicated with 5 dimensions matrix (N, C, T, V, M).N indicates the quantity of 1 batch training video, and C is represented
The feature in joint is (x, y, z), and T represents the quantity of a video frame, and V represents the quantity in joint, is 18 joints, M generation here
Number in one frame of table, setting here up to 2.Finally matrix is normalized.
Step 3.3, network structure: according to spatiality and time domain, it is combination with GCN network and TCN network, constructs space
Time-domain diagram convolutional network.It is weight respectively to each section that GCN network, which is to the side A under single frames picture, linked between artis,
Point feature information carries out convolution operation, and the Convolution Formula of artis i is as follows:
N is the artis quantity of adjacent connection, and I is unit matrix,For the degree of artis i,For artis j's
Degree.XjFor the characteristic information (x of j-th of artis under a frame picturej,yj,zj), aggregate (Xi) it is i-th under a frame picture
Artis aggregation information feature.
Formula can simplify are as follows:
Wherein, Λ=∑ (A+I) here, adjacency matrix A and unit matrix I are the link information of single frames bone, then Λ
It is multiframe bone link information model, JinIt is the characteristic information of the skeleton of input, JoutAfter polymerizeing side information for artis
Value.
The operation of TCN is traditional convolution operation, to the J after GCN network polymerization artis informationoutMatrix be (N, C,
T, V, M), the convolution kernel of O × W × 1 is used (C, V, T) therein, completes W row pixel, the convolution of 1 column pixel every time, O is volume
Product core number.
Step 3.4, model training: after 9 space time-domain diagram convolutional networks, by pond layer and full articulamentum, most
Model training, mini-batch 16, learning rate 0.01, SGD learning algorithm, instruction are carried out using soft-max loss function afterwards
Practice 80 epoch.
Step 3.5, model measurement: the artis characteristic information of one section of video is input in network, GCN network is passed through
Time convolution operation is carried out to artis information fusion, then by TCN, finally by pond layer and full articulamentum, to artis
Feature is classified, and the result of classification is obtained.
When the result of acquisition, which for example falls, opens car door enters the movement of the abnormal behaviours such as vehicle, then prompting police can be sent
It accuses and reminds car owner in information to the mobile phone app of car owner.When the action behavior for identifying human body has the behavior for tending to illegal operation
Such as open car door and enter vehicle etc. then by issuing early warning warning information to car owner mobile phone app, it reminds and informs car owner.
Claims (4)
1. a kind of underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence, includes the following steps:
Step 1, monitoring camera acquire the picture on each parking stall in garage parking in real time, if there is vehicle stops the vehicle for being n into number
Position, then picture transmission to server can be obtained the license plate number of the parking stall by the Car license recognition model on server by monitoring system
Code information Vn；When car owner gets off, face recognition algorithms and pedestrian's weight recognizer on server will extract car owner respectively
Face feature FnWith human external feature P1, and by car owner's face characteristic Fn, pedestrian's feature P1, the license board information V that drives of car ownern
Mark In, it is denoted as record in the server；
Step 2, after car owner has stopped vehicle, on the mobile apparatus by mobile phone parking management app use " a key-lock vehicle " function be
Can virtual lockout vehicle parking position, simultaneity factor unlocking vehicle mobile detection model determines that monitoring camera acquires in real time
Picture in vehicle whether movement has occurred；If vehicle unlocks the feelings of " key positioning " function in car owner not on mobile phone app
Movement has occurred under condition, monitoring system will be sent immediately in stolen warning message to the mobile phone parking management system app of car owner,
To notify the car owner of the vehicle；
Step 3, if monitoring camera still acquires each parking stall in real time without using " a key-lock vehicle " function after car owner's parking
Picture and be transferred on server, by pedestrian weight identification model extract in a certain range close to parked vehicles pedestrian spy
Sign；If the feature of the pedestrian does not meet the surface of the reserved corresponding car owner of the vehicle in the server, while passing through base
It is detected in the behavior act identification model of skeleton and identifies that the pedestrian is made that the movement opened car door and enter vehicle, then determined
For abnormal operation；Server is sent in stolen warning message to the mobile phone parking management system app of car owner, to notify the vehicle
Car owner；
Step 4, at entrance gate, after car owner has stopped vehicle, it will generated in mobile phone parking management system app corresponding
Two dimensional code；After car owner picks up the car, unlocking virtual lockout vehicle functions and drive can whereby two dimensional code after vehicle reaches entrance gate
Sweep that sweep can be by entrance gate on the scanner.
2. the underground parking garage Pedestrians and vehicles monitoring method according to claim 1 based on artificial intelligence, it is characterised in that:
In the step 1, comprising:
Step 1.1, the monitoring camera in covering full cut-off parking lot, each parking stall of monitoring camera real-time Transmission are installed in parking lot
Image transmitting to server-side；
Step 1.2, enter the license plate of the vehicle on parking stall by Recognition Algorithm of License Plate identification first, vehicle stops the picture into parking stall
Convolutional neural networks are input to extract characteristics of image, these features are then input to long short-term memory Recognition with Recurrent Neural Network
In, then license board information is obtained by CTC loss layer；Wherein, long short-term memory Recognition with Recurrent Neural Network is obtained from each time step
The probability value of the letter character of prediction is exported and generated, then removes continuous repeat character (RPT), and deletion space etc. is special
Character, be finally merged into the output of character string, that is, our prediction result Vn；
Step 1.3, pedestrian's feature is then identified by pedestrian's weight recognizer, the car owner's picture of getting off for being transferred to server passes through
ResNet-50 generates the characteristic pattern in 1024 channels as the network of convolutional neural networks model, and the characteristic pattern of generation utilizes 512
× 3 × 3 convolutional layer converts pedestrian's feature, then will be original using 9 target frames in each position of characteristic pattern
Image pixel is converted into a new characteristic pattern and a Soft-max classifier to predict whether each target frame includes pedestrian
Inside, while a linear regressor is further comprised to adjust the position of target frame；Only retain 128 after non-maximum restraining
Then bounding box adjusted extracts pedestrian's feature P of car owner as final region by ResNet-501；
Step 1.4, car owner gets off picture simultaneously also by Deep ID face recognition algorithms, extracts the face characteristic F of car owner1；
Step 1.5, after detecting region of the car owner in picture, car owner's skeleton joint point is obtained using Open pose algorithm
Characteristic information Jn1；
Step 1.6, by license plate prediction result V, pedestrian's feature P of car owner1With the characteristic information f of car owner's skeleton joint pointn1, according to
Parking stall numbering n carries out one-to-one label, is denoted as In={ Pn1, Fn1, Jn1,Vn, and saved in server, Zhi Daoche
It is removed after being driven out to.
3. the underground parking garage Pedestrians and vehicles monitoring method according to claim 1 based on artificial intelligence, it is characterised in that:
In the step 3, further includes:
Every frame image on each parking stall of monitoring camera real-time Transmission is to server-side；When at a time, there is s-th of people close
Number is n and has the picture of the parking stall scene to park cars to weigh pedestrian's feature that recognizer extracts the pedestrian by pedestrian
Pns, and pedestrian's feature P with the car owner of the vehiclen1Cosine similarity comparison is carried out, if being less than the threshold value 0.7 of setting, is passed through simultaneously
The face characteristic F of the face recognition algorithms figure area pedestrianns, by the comparison of the two feature cosine similarity, if being less than setting
Threshold value 0.7 then determines not being the same person with car owner and detect framework information f by Open pose algorithmns, it is based on people
The behavior act recognizer of body skeleton judges whether the pedestrian has and the abnormal behaviours movement such as opens car door and fall.
4. underground parking garage Pedestrians and vehicles monitoring method of the base according to claim 3 based on artificial intelligence: the step
In three, further includes:
Step 3.1, pedestrian's tumble video data under underground parking lab environment, substantially 100 each views data preparation: are collected
Frequency is that one of 150 frames or so continuously falls action video, and video visual angle is underground parking garage monitoring camera visual angle；Collect ground
Pedestrian under lower parking lab environment opens the sets of video data that car door enters vehicle, and substantially 100 each videos are 150 frames
Continuously the action video, video visual angle are underground parking garage monitoring camera visual angle for one；
Step 3.2, ready data data prediction: are extracted to the people of each frame of each video using Open pose algorithm
Body skeleton 18 artis) information (x, y, z), x is the abscissa of artis on the image, y be artis on the image
Ordinate, axle center are the image lower left corner, and z is the value of the confidence of artis；For the video of a batch, with 5 dimension matrixes (N, C,
T, V, M) it indicates, N indicates the quantity of 1 batch training video, and C represents feature i.e. (x, the y, z) in joint, and T represents a view
The quantity of frequency frame, V represent the quantity in joint, are 18 joints here, and M represents the number in a frame, setting here up to 2；
Finally matrix is normalized；
Step 3.3, network structure: according to spatiality and time domain, it is combination with GCN network and TCN network, constructs space time domain
Figure convolutional network；It is that weight is special to each node respectively that GCN network, which is to the side A under single frames picture, linked between artis,
Reference breath carries out convolution operation, and the Convolution Formula of artis i is as follows:
N is the artis quantity of adjacent connection, and I is unit matrix,For the degree of artis i,For the degree of artis j；XjFor
Characteristic information (the x of j-th of artis under one frame picturej,yj,zj), aggregate (Xi) it is i-th of artis under a frame picture
Aggregation information feature；
Simplified formula are as follows:
Wherein, Λ=∑ (A+I) here, adjacency matrix A and unit matrix I are the link information of single frames bone, then Λ is more
Frame bone links information model, JinIt is the characteristic information of the skeleton of input, JoutValue after polymerizeing side information for artis；
The operation of TCN is traditional convolution operation, to the J after GCN network polymerization artis informationoutMatrix be (N, C, T, V,
M), the convolution kernel of O × W × 1 is used (C, V, T) therein, completes W row pixel every time, the convolution of 1 column pixel, O is convolution kernel
Number；
Step 3.4, model training: after 9 space time-domain diagram convolutional networks, by pond layer and full articulamentum, finally make
Model training, mini-batch 16, learning rate 0.01, SGD learning algorithm, training 80 are carried out with soft-max loss function
A epoch；
Step 3.5, model measurement: the artis characteristic information of one section of video is input in network, by GCN network to pass
Nodal information polymerization, then time convolution operation is carried out by TCN, finally by pond layer and full articulamentum, to joint point feature
Classify, obtains the result of classification；
When the result of acquisition, which for example falls, opens car door enters the movement of the abnormal behaviours such as vehicle, then reminder alerting letter can be sent
It ceases and reminds car owner on the mobile phone app of car owner；Tend to the behavior of illegal operation for example when the action behavior for identifying human body has
It opens car door and enters vehicle etc. then by issuing early warning warning information to car owner mobile phone app, remind and inform car owner.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910535807.8A CN110298278B (en) | 2019-06-19 | 2019-06-19 | Underground parking garage pedestrian and vehicle monitoring method based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910535807.8A CN110298278B (en) | 2019-06-19 | 2019-06-19 | Underground parking garage pedestrian and vehicle monitoring method based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110298278A true CN110298278A (en) | 2019-10-01 |
| CN110298278B CN110298278B (en) | 2021-06-04 |
Family
ID=68028304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910535807.8A Active CN110298278B (en) | 2019-06-19 | 2019-06-19 | Underground parking garage pedestrian and vehicle monitoring method based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110298278B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111277795A (en) * | 2020-01-15 | 2020-06-12 | 重庆特斯联智慧科技股份有限公司 | Parking lot monitoring method and system based on artificial intelligence |
| CN111436944A (en) * | 2020-04-20 | 2020-07-24 | 电子科技大学 | Falling detection method based on intelligent mobile terminal |
| CN111627218A (en) * | 2020-06-02 | 2020-09-04 | 重庆智者炎麒科技有限公司 | Method for recognizing license plate at night through image enhancement |
| CN111695501A (en) * | 2020-06-11 | 2020-09-22 | 青岛大学 | Equipment soft fault detection method based on operating system kernel calling data |
| CN112052850A (en) * | 2020-09-03 | 2020-12-08 | 平安科技（深圳）有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN112183299A (en) * | 2020-09-23 | 2021-01-05 | 成都佳华物链云科技有限公司 | Pedestrian attribute prediction method and device, electronic equipment and storage medium |
| CN112687108A (en) * | 2020-06-28 | 2021-04-20 | 浙江大华技术股份有限公司 | Detection method and detection device for man-vehicle matching, shooting device and storage medium |
| CN112949417A (en) * | 2021-02-05 | 2021-06-11 | 杭州萤石软件有限公司 | Tumble behavior identification method, equipment and system |
| CN113326834A (en) * | 2021-08-04 | 2021-08-31 | 湖南乐泊科技有限公司 | Vehicle moving method and system based on video tracking and storage medium |
| CN114120561A (en) * | 2021-11-09 | 2022-03-01 | 中国第一汽车股份有限公司 | Reminding method, device, equipment and storage medium |
| CN114511870A (en) * | 2020-10-27 | 2022-05-17 | 天津科技大学 | Pedestrian attribute information extraction and re-identification method combined with graph convolution neural network |
| WO2022100107A1 (en) * | 2020-11-13 | 2022-05-19 | Huawei Technologies Co.,Ltd. | Methods and systems for predicting dynamic object behavior |
| US11537852B2 (en) * | 2020-02-13 | 2022-12-27 | International Business Machines Corporation | Evolving graph convolutional networks for dynamic graphs |
| CN115862241A (en) * | 2023-03-03 | 2023-03-28 | 江苏安能科技有限公司 | Charging pile region theft monitoring method |
| CN116704448A (en) * | 2023-08-09 | 2023-09-05 | 山东字节信息科技有限公司 | Pedestrian recognition method and recognition system with multiple cameras |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968873A (en) * | 2012-11-29 | 2013-03-13 | 深圳市捷顺科技实业股份有限公司 | Early warning system in parking lot |
| CN203025884U (en) * | 2012-11-29 | 2013-06-26 | 深圳市捷顺科技实业股份有限公司 | Parking lot early warning system |
| US20140309789A1 (en) * | 2013-04-15 | 2014-10-16 | Flextronics Ap, Llc | Vehicle Location-Based Home Automation Triggers |
| CN106710291A (en) * | 2015-11-17 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Parking stall acquisition method and device of parking lot |
| CN108805008A (en) * | 2018-04-19 | 2018-11-13 | 江苏理工学院 | A kind of community's vehicle security system based on deep learning |
| CN108830213A (en) * | 2018-06-12 | 2018-11-16 | 北京理工大学 | Car plate detection and recognition methods and device based on deep learning |
| CN109101915A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | Face and pedestrian and Attribute Recognition network structure design method based on deep learning |
| CN109686109A (en) * | 2019-01-02 | 2019-04-26 | 江苏警官学院 | A kind of parking lot security monitoring management system based on artificial intelligence |
| CN208969842U (en) * | 2018-09-06 | 2019-06-11 | 恒大智慧科技有限公司 | Parking lot theft management system |
-
2019
- 2019-06-19 CN CN201910535807.8A patent/CN110298278B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102968873A (en) * | 2012-11-29 | 2013-03-13 | 深圳市捷顺科技实业股份有限公司 | Early warning system in parking lot |
| CN203025884U (en) * | 2012-11-29 | 2013-06-26 | 深圳市捷顺科技实业股份有限公司 | Parking lot early warning system |
| US20140309789A1 (en) * | 2013-04-15 | 2014-10-16 | Flextronics Ap, Llc | Vehicle Location-Based Home Automation Triggers |
| CN106710291A (en) * | 2015-11-17 | 2017-05-24 | 腾讯科技（深圳）有限公司 | Parking stall acquisition method and device of parking lot |
| CN108805008A (en) * | 2018-04-19 | 2018-11-13 | 江苏理工学院 | A kind of community's vehicle security system based on deep learning |
| CN108830213A (en) * | 2018-06-12 | 2018-11-16 | 北京理工大学 | Car plate detection and recognition methods and device based on deep learning |
| CN109101915A (en) * | 2018-08-01 | 2018-12-28 | 中国计量大学 | Face and pedestrian and Attribute Recognition network structure design method based on deep learning |
| CN208969842U (en) * | 2018-09-06 | 2019-06-11 | 恒大智慧科技有限公司 | Parking lot theft management system |
| CN109686109A (en) * | 2019-01-02 | 2019-04-26 | 江苏警官学院 | A kind of parking lot security monitoring management system based on artificial intelligence |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111277795A (en) * | 2020-01-15 | 2020-06-12 | 重庆特斯联智慧科技股份有限公司 | Parking lot monitoring method and system based on artificial intelligence |
| US11537852B2 (en) * | 2020-02-13 | 2022-12-27 | International Business Machines Corporation | Evolving graph convolutional networks for dynamic graphs |
| CN111436944A (en) * | 2020-04-20 | 2020-07-24 | 电子科技大学 | Falling detection method based on intelligent mobile terminal |
| CN111627218A (en) * | 2020-06-02 | 2020-09-04 | 重庆智者炎麒科技有限公司 | Method for recognizing license plate at night through image enhancement |
| CN111695501A (en) * | 2020-06-11 | 2020-09-22 | 青岛大学 | Equipment soft fault detection method based on operating system kernel calling data |
| CN112687108A (en) * | 2020-06-28 | 2021-04-20 | 浙江大华技术股份有限公司 | Detection method and detection device for man-vehicle matching, shooting device and storage medium |
| CN112687108B (en) * | 2020-06-28 | 2022-08-09 | 浙江大华技术股份有限公司 | Detection method and detection device for man-vehicle matching, shooting device and storage medium |
| CN112052850A (en) * | 2020-09-03 | 2020-12-08 | 平安科技（深圳）有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN112183299A (en) * | 2020-09-23 | 2021-01-05 | 成都佳华物链云科技有限公司 | Pedestrian attribute prediction method and device, electronic equipment and storage medium |
| CN112183299B (en) * | 2020-09-23 | 2024-02-09 | 成都佳华物链云科技有限公司 | Pedestrian attribute prediction method and device, electronic equipment and storage medium |
| CN114511870A (en) * | 2020-10-27 | 2022-05-17 | 天津科技大学 | Pedestrian attribute information extraction and re-identification method combined with graph convolution neural network |
| WO2022100107A1 (en) * | 2020-11-13 | 2022-05-19 | Huawei Technologies Co.,Ltd. | Methods and systems for predicting dynamic object behavior |
| CN112949417A (en) * | 2021-02-05 | 2021-06-11 | 杭州萤石软件有限公司 | Tumble behavior identification method, equipment and system |
| CN113326834B (en) * | 2021-08-04 | 2021-10-15 | 湖南乐泊科技有限公司 | Vehicle moving method and system based on video tracking and storage medium |
| CN113326834A (en) * | 2021-08-04 | 2021-08-31 | 湖南乐泊科技有限公司 | Vehicle moving method and system based on video tracking and storage medium |
| CN114120561A (en) * | 2021-11-09 | 2022-03-01 | 中国第一汽车股份有限公司 | Reminding method, device, equipment and storage medium |
| CN115862241A (en) * | 2023-03-03 | 2023-03-28 | 江苏安能科技有限公司 | Charging pile region theft monitoring method |
| CN116704448A (en) * | 2023-08-09 | 2023-09-05 | 山东字节信息科技有限公司 | Pedestrian recognition method and recognition system with multiple cameras |
| CN116704448B (en) * | 2023-08-09 | 2023-10-24 | 山东字节信息科技有限公司 | Pedestrian recognition method and recognition system with multiple cameras |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110298278B (en) | 2021-06-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110298278A (en) | A kind of underground parking garage Pedestrians and vehicles monitoring method based on artificial intelligence | |
| CN103839346B (en) | A kind of intelligent door and window anti-intrusion device and system, intelligent access control system | |
| CN103839373B (en) | A kind of unexpected abnormality event Intelligent Recognition alarm device and warning system | |
| CN107730905A (en) | Multitask fake license plate vehicle vision detection system and method based on depth convolutional neural networks | |
| CN110188807A (en) | Tunnel pedestrian target detection method based on cascade super-resolution network and improvement Faster R-CNN | |
| CN103632427B (en) | A kind of gate cracking protection method and gate control system | |
| CN107133569A (en) | The many granularity mask methods of monitor video based on extensive Multi-label learning | |
| CN107220603A (en) | Vehicle checking method and device based on deep learning | |
| CN108805008A (en) | A kind of community's vehicle security system based on deep learning | |
| CN108875754B (en) | Vehicle re-identification method based on multi-depth feature fusion network | |
| CN107644206A (en) | A kind of road abnormal behaviour action detection device | |
| KR102122850B1 (en) | Solution for analysis road and recognition vehicle license plate employing deep-learning | |
| CN106340205A (en) | Traffic monitoring method and traffic monitoring apparatus | |
| CN111145475A (en) | Intrusion alarm system, method and medium based on vibration optical fiber and deep learning | |
| CN112866647A (en) | Intelligent property management system based on smart community | |
| JP6837650B1 (en) | Risk determination program and system | |
| CN109635717A (en) | A kind of mining pedestrian detection method based on deep learning | |
| CN106384089A (en) | Human body reliable detection method based on lifelong learning | |
| Pan et al. | Fake license plate recognition in surveillance videos | |
| CN112633163B (en) | Detection method for realizing illegal operation vehicle detection based on machine learning algorithm | |
| Ho et al. | Public space behavior modeling with video and sensor analytics | |
| CN109727089A (en) | Return a deposit golden automatic prompt method and monitoring method of servicing based on access control system | |
| CN109636155A (en) | A kind of intelligent early-warning method of forced quarantine addict received treatment danger sexual orientation | |
| CN113569709A (en) | Campus security image recognition early warning method based on convolutional neural network | |
| Parab et al. | Automatic parking system using Automatic number plate recognition (ANPR) |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |