WO2018037355A1 - Système et procédé permettant de détecter automatiquement un véhicule et un visage, et de les classer - Google Patents

Système et procédé permettant de détecter automatiquement un véhicule et un visage, et de les classer Download PDF

Info

Publication number
WO2018037355A1
WO2018037355A1 PCT/IB2017/055083 IB2017055083W WO2018037355A1 WO 2018037355 A1 WO2018037355 A1 WO 2018037355A1 IB 2017055083 W IB2017055083 W IB 2017055083W WO 2018037355 A1 WO2018037355 A1 WO 2018037355A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
central module
database
information
details
Prior art date
Application number
PCT/IB2017/055083
Other languages
English (en)
Inventor
Rajkumar BHAGAVANT
Mahesh BHAGYAWANT
Original Assignee
Bhagavant Rajkumar
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bhagavant Rajkumar filed Critical Bhagavant Rajkumar
Publication of WO2018037355A1 publication Critical patent/WO2018037355A1/fr

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/62Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/10Image acquisition
    • G06V10/17Image acquisition using hand-held instruments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161Detection; Localisation; Normalisation
    • G06V40/165Detection; Localisation; Normalisation using facial parts and geometric relationships
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/94Hardware or software architectures specially adapted for image or video understanding
    • G06V10/95Hardware or software architectures specially adapted for image or video understanding structured as a network, e.g. client-server architectures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625License plates

Definitions

  • the present invention relates to a system and method for automatically detecting a vehicle and pedestrian's face from the live streaming of a CCTV surveillance camera and further categorizing information obtained into predefined classes.
  • Unwanted visitor is an individual whose entry in the society is prohibited as he/she is either entering in illegal way or might be a wanted individual as per police records.
  • a security guard at an entry gate of an area such as at a residential/commercial complex, enters such details manually which is a time consuming process and also guard is not able to retrieve the older records or update them.
  • user e.g. resident
  • intercom regarding a visitor's visit, however in case, no one is available in the house or intercom is not working or intercom is not installed, then concurrently communicating the information regarding visitor's arrival to the user is not possible.
  • CCTV cameras with DVR or IP cameras are also installed at different locations for security purpose, however, it only stores the video footage and therefore it is checked only after some incident takes place, for example, robbery, and the concerned person has to check the entire footage to analyze and retrieve the relevant information as incidence time is unknown.
  • WO 2013146461 discloses a vehicle data processing system, method, device, program, and recording medium.
  • the system comprising an image capturing device that captures an image of a roadway over time, a wireless device and a vehicle data processing device which processes vehicle data captured by image capture device and wireless device.
  • the wireless device receives information from a vehicle- mounted device of the vehicle.
  • the vehicle data processing device then binds vehicle data of the same vehicle obtained from image capture device and wireless device.
  • the said system therefore requires additional device both at gate and on vehicle to conclude any information and are dependent on each other.
  • the system also includes character completion unit using vehicle data II information to read any character.
  • 2012/KO/000259 discloses an intelligent automatic traffic enforcement system.
  • the system identifies the traffic violation and alerts traffic personnel by remotely observing the video feeds coming to the control room from the junction through computer monitor.
  • EP 1854297 discloses an entry control point device, system and method which can recognize a vehicle by its undercarriage image only and compares a new vehicle undercarriage image with those stored in a reference database to facilitate the automatic search of a vehicle's undercarriage thereby highlighting any anomalies for example, foreign objects, explosives, drugs, paraphernalia and other illegal and/or harmful substances and devices.
  • this system includes an under vehicle scanning platform that can be employed on the road at an entry point and other devices that are communicating with each other to provide the information. Also system includes RFID tag on vehicle, which is used to identify vehicle.
  • the invention is employing several number of external devices like vehicle-scanning device, AHS and ALS, thereby making the system costly.
  • the proposed invention is a system and method for automated vehicle detection along with its occupant's and pedestrian's face detection that provides security by detecting any suspicious activity, sends concurrent notification to concerned persons in real time, automatically records and updates the details of pedestrians and vehicles and is easy to use.
  • the primary object of the invention is to provide a system and method for auto detection of vehicles from live streaming of CCTV or IP camera and analyzing the image of vehicle for number plate reading and further classification.
  • Another object of the invention is to provide a system and method for auto detection of pedestrians and vehicle occupants from live streaming of CCTV or IP camera and analyzing image of pedestrian for face detection and further classification.
  • Another object of the invention is to provide automated records of details of vehicles along with occupants and pedestrians entering in an area and saving said details in a database on cloud or server. Another object of the invention is to auto-train the database by an application to improve the accuracy of results.
  • Another object of the invention is to concurrently notify the resident or concerned person on arrival of a visitor on user device.
  • a further object of the invention is to detect any suspicious activity being captured from live streaming of CCTV or IP camera and concurrently notify the respective or concerned person(s) on user device.
  • Another object of the invention is to detect entry of any unwanted individuals in an area either through the entry gate or plunging over the wall or gate.
  • Yet another object of the invention is to provide concurrent updates and information of vehicles and pedestrians entering in an area to a user on user device.
  • FIG. 1 illustrates architecture of the present invention
  • Figure 2 illustrates flow chart of detection process
  • FIG. 3 illustrates flow chart of vehicle detection process
  • Figure 4 illustrates flow chart of vehicle number plate detection mechanism
  • Figure 5 illustrates the process of displaying the results on user device and saving the record
  • Figure 6 illustrates system architecture of vehicle detection process
  • Figure 7 illustrates flow chart of face detection process
  • Figure 8 illustrates flow chart of gender and age classification process
  • Figure 9 illustrates system architecture of face detection process.
  • the proposed invention facilitates an automation of records of details of vehicles along with occupants and pedestrians entering in an area, for example, in a residential/commercial complex/premises.
  • the term 'automation' is used to indicate that the proposed invention automatically detects pedestrian and vehicle from live streaming of CCTV surveillance camera.
  • Cloud- It is a virtual storage space where data is maintained, managed, backed up remotely and is available to users for access over a network.
  • the user herein can be anybody such as guard, society chairman, resident, security personnel and so on.
  • a user device can be any device that allows user to communicate with central module.
  • User devices include but not limited to mobile communication devices such as cellular phones, smart phones, tablets.
  • the central module is a complete processing system consisting of a central server or multiple servers for secured exchange of information among themselves and/or with other platforms, applications or devices, including detection application and further processing.
  • Platform on user device It is interface that enables interaction or exchange of information between user device and central module.
  • Unwanted visitor A visitor whose entry in an area is prohibited as he/she is either entering in illegal way or might be a wanted individual as per police records.
  • One aspect of the invention is a system for automated vehicle detection along with its occupant's and pedestrian's face detection, the system comprising: CCTV surveillance camera or IP camera, central module in cloud and user device.
  • CCTV surveillance camera records the events at a place and simultaneously sends live footage to the central module in the cloud.
  • the central module analyzes and extracts each frame from live footage for further processing such as verification, matching the details from existing database and other such activities as shown in Figures 2, 3, 4, 5, 7 and 8. After processing, central module sends the information obtained to the user device. In case any change is required in the obtained information, the same can be done from the user device manually.
  • the retrieved or modified information is then sent to the central module and is saved in its database for future reference
  • Another aspect of the invention is a method for automated vehicle detection along with its occupant's and pedestrian's face detection.
  • the live footage recorded by a CCTV surveillance camera or IP camera is concurrently sent to central module in the cloud for further analysis and processing.
  • each frame from live footage is extracted and analyzed by central module.
  • obtained information is sent to the user device where it is verified and if required, modified manually by the user. The verified and modified information is then saved on central module in its database for future reference.
  • frame to frame comparison is performed while processing the footage and to obtain further information.
  • the difference between the motions of two frames is detected.
  • the object detection process is initiated to detect the size of object to take decision.
  • vehicle detection is performed as illustrated in Figure 3.
  • the location of detected vehicle is marked and extracted out from original frame.
  • the detected vehicle is then analyzed for vehicle classification, its occupant's images if two-wheeled vehicle and number plate detection. If required, the frame from the footage is split and zoomed out to detect the number plate.
  • the application trained database and detected height-width information of the vehicle is used for vehicle classification based on detected image from the footage.
  • the location of number plate and their format varies with different vehicles and other reasons. Hence, along with number plate detection, the approximate location on vehicle is fixed for number plate reading. The approximate location of number plate is fixed and each and every characters and digits present on focused area are recognized.
  • the OCR tool (tesseract) is used for reading characters on detected object(s). If OCR fails to read any character then the application trained character database is used to find out the characters in the detected object.
  • the detected characters and digits are analyzed against vehicle registration number formats to form the proper number of vehicle.
  • the numbers formed are presented to user (for example- guard). The user can verify or modify the detected formed number and also take the picture of vehicle occupant(s) with their permission for records in case vehicle occupant(s) face is not captured by the CCTV camera.
  • the said number alongwith other information are saved in the database on central server for future use. If same vehicle arrives at the premises again, then all saved information for vehicle is displayed to user on user device.
  • face detection is performed as shown in Figure 7.
  • the image frame is split and zoomed out for face detection. If any facial attributes are detectable from the detected face, then only detected face is used for further processing and location of the person along with time of entrance and exit are noted. In the detected facial image, location of facial attributes are marked and extracted out for further processing. At least two facial attributes are compared with application-trained database. The application- trained database is used for gender and age classification based on detected face and facial attributes. The detected face and age-gender information is then presented to user (for example- guard) for verification and modification (if required).
  • the user may assign a name/tag to the face and then the face and other information is saved in the database for future use. If same face is detected again then the previously stored information is displayed onto the user device. In case image of the face retrieved is not correct, the user can take the picture of pedestrian in order to correct and train the database.
  • full body of a person along with face and facial attributes are detected. In case, detected moving object is not recognized as either human being or vehicle, then detected object is considered as a foreign object. In case of a person is not detected (for example, face is covered), then also the person is categorized as unknown or an intruder.
  • the application trained database is used for vehicle type, basic face and facial attributes detection like eyes, nose and mouth detection. In order to train the application database for such detection, the modifications or verifications done by user is noted and saved in the database thereby improving the results.
  • the human face is very important for identification.
  • the human age is divided in three categories: child, adult and old.
  • Gender and age detection from human face is a very challenging task.
  • Human's shape, size and skin textures varies with age. As a human grows older, the facial appearance also changes. These features become difficult to detect due to makeup, hair style, accessories, color, size, increase in weight, emotional stress, smoking and such similar factors. Therefore, to get the most relevant results, database of variety of face images is required as features of a person varies region wise as well.
  • face images of a region are referred where the invention is used.
  • the invention is used in India
  • Indian facial images are used to train the database.
  • These images contain all possible categories of people with different age-group and gender.
  • gender and age detection factors such as position of eyes, nose and mouth are considered.
  • the training of an application database is a continuous process for improvement and accuracy of detection.
  • the training of application database for different class type vehicles is carried out. Multiple vehicle images from real time of different class are used for training.
  • Number plate carries text other than number
  • the basic information like name, house address, contact details, photograph and other such details are required to be saved on the central module.
  • the central module On successful registration, the central module generates a unique user-id and password for the said resident.
  • the resident or user using login details is able to login onto the platform on user device which is linked or connected with central module and concurrently access the information of vehicles and pedestrians entering in an area.
  • the platform facilitates features like communication with society members, send premises notices and events details, provide list of emergency contacts in premises and nearby places with calling facility, list of service providers with calling facility, profile update with vehicle detail entries, and such other features.
  • the resident or user can update the respective profile, upload photograph, add family members and other such information.
  • the details of society staff and service provider along with their photographs and vehicle details like vehicle image and vehicle registration number are also required to be maintained in the database in order to improve the results.
  • For example- Guard using the platform on mobile device maintains and updates visitor's entry and exit records at premises entry and/or exit gates.
  • the information such as vehicle image, vehicle number, vehicle class, entry time, entry gate number from where the vehicle entered and other such details are auto -retrieved from live CCTV surveillance camera footage.
  • the guard is present, the aforesaid information is collected with the host or resident name to whom visitor is going to meet and visitor's photograph of the face with permission is entered by the guard.
  • the entire visitor's information is then sent on resident's user device by the central module and also saved in the database. All such information is gathered in absence of guard as well but then visitor is considered as uninformed and information is not communicated to the host or resident.
  • a pedestrian enters the premises, the face of pedestrian is captured from CCTV surveillance camera. The image of the pedestrian is then analyzed to determine the approximate age group and gender of the pedestrian. This information is then displayed to the guard on mobile device and if required, guard may update the retrieved details and enter the purpose of visit and resident's name or flat number to whom or where visitor is visiting and save all such information in the database. Therefore, the resident is notified about the arrival of a visitor via the platform on user device or registered mobile number or other contact information.
  • the database of central module stores and maintains information and using the stored information, central module responds accordingly via platform on user device.
  • Database maintains the data regarding house owners, registers or updates information of staff members, service providers, emergency contacts from nearby place, records of security guards, updates regarding notices and events and other such information.
  • the central module facilitates, via platform on user device, sending reminders, generating defaulter list, providing access to the list of pedestrians in a premises, sending notifications about uninformed pedestrians, foreign object detection, missed visitors, facilitating communication with premises residents, accessing events or notices details, providing search for neighboring residents, setting resident IN/OUT status for visitor, updating profile, setting meeting requests and receiving response regarding the same, updating the results.
  • the proposed invention also solves the problem of suspicious or unwanted entry in a premise by generating an auto-alert. For instance, any suspicious movement is detected in CCTV surveillance camera, such movement is detected and respective person is immediately sent an emergency alert along with respective footage so that appropriate action can be taken.
  • the criteria for suspicious movements may be the case where the face of an entrant is not detected or if entrant did not enter through an authorized gate or if the detected object is neither human being nor vehicle and other such suspicious cases.
  • the entry of a person enlisted in wanted database of police is also considered as suspicious movement.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • Alarm Systems (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

L'invention concerne un système et un procédé permettant de détecter automatiquement un véhicule ainsi que les visages d'un occupant et d'un piéton à partir de la séquence de caméras de surveillance CCTV en direct, ainsi que d'avertir simultanément l'utilisateur ou la personne concernée en conséquence. Selon l'invention, chaque trame de la séquence est analysée dans le module central afin d'obtenir des informations telles que le type de véhicule, la plaque d'immatriculation, les lieux d'entrée et de sortie, les heures d'entrée et de sortie, le visage d'un occupant, le visage d'un piéton, l'âge et le sexe ainsi que d'autres détails relatifs au visiteur (véhicule ou piéton). Les informations obtenues sont ensuite affichées à l'attention de l'utilisateur à des fins de vérification ou de modification (si nécessaire). Les informations vérifiées ou modifiées sont ensuite envoyées à la personne ou aux personnes concernées, puis stockées dans la base de données formée par application sur un module central afin de servir de référence à l'avenir.
PCT/IB2017/055083 2016-08-24 2017-08-23 Système et procédé permettant de détecter automatiquement un véhicule et un visage, et de les classer WO2018037355A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN201621028900 2016-08-24
IN201621028900 2016-08-24

Publications (1)

Publication Number Publication Date
WO2018037355A1 true WO2018037355A1 (fr) 2018-03-01

Family

ID=61246372

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2017/055083 WO2018037355A1 (fr) 2016-08-24 2017-08-23 Système et procédé permettant de détecter automatiquement un véhicule et un visage, et de les classer

Country Status (1)

Country Link
WO (1) WO2018037355A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110674332A (zh) * 2019-08-01 2020-01-10 南昌市微轲联信息技术有限公司 基于ocr与文本挖掘的机动车数字电子档案分类方法
CN112200953A (zh) * 2020-10-10 2021-01-08 合肥远康信息技术有限公司 一种智慧平安校园访客预约平台
CN112633163A (zh) * 2020-12-22 2021-04-09 合肥品恩智能科技有限公司 一种基于机器学习算法实现非法运营车辆检测的检测方法
US20210142268A1 (en) * 2019-04-30 2021-05-13 Transportation Ip Holdings, Llc Asset identification and tracking system
CN114743298A (zh) * 2022-04-26 2022-07-12 广东申立信息工程股份有限公司 一种一站式车辆通关系统

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110302130A1 (en) * 2010-06-03 2011-12-08 Electronics And Telecommunications Research Institute Intelligent video information retrieval apparatus and method capable of multi-dimensional video indexing and retrieval

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110302130A1 (en) * 2010-06-03 2011-12-08 Electronics And Telecommunications Research Institute Intelligent video information retrieval apparatus and method capable of multi-dimensional video indexing and retrieval

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210142268A1 (en) * 2019-04-30 2021-05-13 Transportation Ip Holdings, Llc Asset identification and tracking system
US11875284B2 (en) * 2019-04-30 2024-01-16 Transportation Ip Holdings, Llc Asset identification and tracking system
CN110674332A (zh) * 2019-08-01 2020-01-10 南昌市微轲联信息技术有限公司 基于ocr与文本挖掘的机动车数字电子档案分类方法
CN110674332B (zh) * 2019-08-01 2022-11-15 南昌市微轲联信息技术有限公司 基于ocr与文本挖掘的机动车数字电子档案分类方法
CN112200953A (zh) * 2020-10-10 2021-01-08 合肥远康信息技术有限公司 一种智慧平安校园访客预约平台
CN112633163A (zh) * 2020-12-22 2021-04-09 合肥品恩智能科技有限公司 一种基于机器学习算法实现非法运营车辆检测的检测方法
CN114743298A (zh) * 2022-04-26 2022-07-12 广东申立信息工程股份有限公司 一种一站式车辆通关系统
CN114743298B (zh) * 2022-04-26 2023-11-10 广东申立信息工程股份有限公司 一种一站式车辆通关系统

Similar Documents

Publication Publication Date Title
US10582163B2 (en) Monitoring an area using multiple networked video cameras
WO2018037355A1 (fr) Système et procédé permettant de détecter automatiquement un véhicule et un visage, et de les classer
Norris From personal to digital: CCTV, the panopticon, and the technological mediation of suspicion and social control
US10152858B2 (en) Systems, apparatuses and methods for triggering actions based on data capture and characterization
US20140334684A1 (en) System and method for neighborhood-scale vehicle monitoring
KR102152318B1 (ko) 객체의 이동경로를 추적할 수 있는 추적시스템
KR101419794B1 (ko) 학교 방문자의 얼굴인식을 통한 학교 출입자 관리 방법
US10896321B2 (en) Monitoring inmate movement with facial recognition
US20140369566A1 (en) Perimeter Image Capture and Recognition System
US20040001142A1 (en) Method for suspect identification using scanning of surveillance media
US20140369567A1 (en) Authorized Access Using Image Capture and Recognition System
WO2019090096A1 (fr) Procédé et système pour surveiller et évaluer l'humeur d'employés
CN111597999A (zh) 一种基于视频检测的4s店销售服务管理方法和系统
US11308792B2 (en) Security systems integration
KR101084914B1 (ko) 차량번호 및 사람 이미지의 인덱싱 관리시스템
JPWO2020115890A1 (ja) 情報処理装置、情報処理方法、およびプログラム
CN114358980A (zh) 一种基于物联网的智慧社区物业管理系统及方法
CN112132048A (zh) 一种基于计算机视觉的社区巡更分析方法及系统
Sai et al. Low cost automated facial recognition system
Saranya et al. Computer Vision on Identifying Persons under Real Time Surveillance using IOT
CN210983509U (zh) 入境重点人群布控系统
Khot Harish et al. Smart video surveillance
Musthafa et al. Smart Authentication System Using Deep Learning Techniques Based on Face and License Plate Recognition
JP2006163788A (ja) 入場者管理方法及び管理システム
WO2017048148A1 (fr) Surveillance de flux d'objets au moyen d'un détecteur de cartes sim

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17843035

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17843035

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 14/08/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 17843035

Country of ref document: EP

Kind code of ref document: A1