WO2020096111A1 - Dispositif et procédé de reconnaissance de comportement anormal basés sur un apprentissage profond - Google Patents

Dispositif et procédé de reconnaissance de comportement anormal basés sur un apprentissage profond Download PDF

Info

Publication number
WO2020096111A1
WO2020096111A1 PCT/KR2018/013948 KR2018013948W WO2020096111A1 WO 2020096111 A1 WO2020096111 A1 WO 2020096111A1 KR 2018013948 W KR2018013948 W KR 2018013948W WO 2020096111 A1 WO2020096111 A1 WO 2020096111A1
Authority
WO
WIPO (PCT)
Prior art keywords
abnormal behavior
deep learning
image
present
recognition model
Prior art date
Application number
PCT/KR2018/013948
Other languages
English (en)
Korean (ko)
Inventor
김동칠
박성주
Original Assignee
전자부품연구원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 전자부품연구원 filed Critical 전자부품연구원
Publication of WO2020096111A1 publication Critical patent/WO2020096111A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602—Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19613—Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms

Definitions

  • the present invention relates to an apparatus and method for recognizing an abnormal behavior.
  • the 2D image-based image security system has a problem in that reliability of abnormal behavior recognition is greatly deteriorated due to the influence of environmental factors.
  • an anomaly recognition technique based on a 3D depth image is proposed, but it has a limitation in that it cannot process an image input in real time due to low recognition accuracy and slow processing speed.
  • the present invention has been proposed to solve the above-mentioned problems, and an object thereof is to provide an apparatus and method for recognizing anomalous behavior through analysis of spatiotemporal image data using deep learning based on 3D depth image input.
  • the apparatus for recognizing anomalous behavior based on deep learning includes an image receiving unit for receiving a 3D image, a processor for storing a program for determining whether an abnormal behavior is analyzed by analyzing the 3D image, and a processor for executing the program, wherein the processor is a 3D image It is characterized by performing spatiotemporal image data analysis using deep learning to generate an abnormal behavior recognition model and estimating the abnormal behavior.
  • the method for recognizing anomalous behavior based on deep learning includes receiving 3D depth image information, performing spatio-temporal analysis based on deep learning using 3D depth image information, performing learning, and generating an abnormal behavior recognition model And analyzing the actual depth image data and determining whether the behavior is abnormal using the step and the behavior recognition model.
  • the present invention is applied to a video security system to protect personal safety and maximize a crime prevention effect. It is robust to various environmental changes and can process images in real time, greatly improving the reliability of inference of abnormal behavior. It has the effect.
  • FIG. 1 is a block diagram illustrating a deep learning-based abnormal behavior recognition apparatus according to an embodiment of the present invention.
  • FIG. 2A to 2D illustrate a pre-processing process for depth image data according to an embodiment of the present invention.
  • FIG. 3 shows a network structure for spatiotemporal analysis using deep learning according to an embodiment of the present invention.
  • FIG. 4 is a flowchart illustrating a deep learning-based abnormal behavior recognition method according to an embodiment of the present invention.
  • the 2D image-based image security system detects and classifies objects such as people and vehicles, and tracks the objects, thereby recognizing simple abnormal behaviors such as passage, intrusion, and loitering of a specific point.
  • the 2D image-based image security system detects objects and events, and is affected by environmental factors such as lighting, weather, color, etc. In particular, identification of objects is not possible in night situations when sufficient light is not secured. It is difficult to acquire the image as much as possible.
  • 3D Depth-based abnormal behavior recognition technology has been developed to solve the problems caused by external factors, but it is difficult to process an image input in real time due to low recognition accuracy and slow processing speed. .
  • the present invention has been proposed to solve the above-mentioned problems, and proposes an apparatus and method for analyzing anomalous behavior by analyzing spatiotemporal image data using CNN as a 3D depth image input, and using an end-to-end structure.
  • the present invention recognizes anomalous behavior based on 3D depth image, overcomes various environmental factors, and enables real-time image processing and anomaly recognition using 3D depth image and deep learning, and an image security system It is applied to ensure personal safety and prevent crime.
  • FIG. 1 is a block diagram illustrating a deep learning-based abnormal behavior recognition apparatus according to an embodiment of the present invention.
  • spatiotemporal image data analysis using CNN is performed on 3D Depth image information that is robust to various environmental changes such as lighting, weather, and color to analyze abnormal behavior.
  • the apparatus for recognizing anomalous behavior based on deep learning includes an image receiving unit 100 for receiving a 3D image, a memory 200 storing a program for determining whether an abnormal behavior is analyzed by analyzing the 3D image, and a program to be executed.
  • a processor 300 is included, and the processor 300 performs space-time image data analysis using deep learning on the 3D image to generate an abnormal behavior recognition model and estimates the abnormal behavior.
  • the processor 300 receives a 3D image for deep learning-based image learning, calculates a depth value difference, separates and tracks an object from the background, generates ground truth data for cognitive behavior, and creates a convolutional 3D network structure Based on this, we implement a network model for analyzing anomalies and generate an anomaly recognition model.
  • the processor 300 analyzes the actual depth image data using the abnormal behavior recognition model, and provides an alert to the security system when the abnormal behavior is estimated.
  • the processor 300 performs depth image data acquisition and preprocessing steps for deep learning-based image learning, and at this time, acquires depth image data for image data learning and backgrounds from the acquired image
  • the object is extracted through the modeling method.
  • the process of extracting objects includes the process of modeling the depth image background for object extraction, the process of calculating the difference in depth values between the object and the background, the process of extracting the object after separating the blob corresponding to the object from the background, and comparing it with the previous frame. And tracking the blob with the minimum L2-norm around the blob as the same object.
  • FIGS. 2A to 2D illustrate a pre-processing process for depth image data according to an embodiment of the present invention, each showing a background image modeling, object / background depth value difference calculation, object extraction and object tracking process.
  • the processor 300 performs the generation of learning depth image data GT, and at this time, generates an action start time, action end time, and action name GT for recognition of an abnormal action.
  • cognitive behavior includes roaming, punching, kicking, collapsing, clustering, group beating, passage of virtual paths, vandalism, appearance, disappearance, camouflage, and discarding.
  • the processor 300 performs space-time analysis based on deep learning on image learning data, implements a network model for analyzing abnormal behavior based on a convolutional 3D network structure, and end-to-end It is possible to recognize abnormal behavior in real time by processing image data using a structure and configuring relatively few layers.
  • 3 shows a network structure for spatiotemporal analysis using deep learning according to an embodiment of the present invention.
  • a deep learning based abnormal behavior recognition model is generated, and at this time, an abnormal behavior recognition model is generated by learning about 50 epochs of the learning image data in the deep learning based abnormal behavior recognition model.
  • the processor receives the actual depth image data, estimates the abnormal behavior, performs event notification, receives the actual depth image data in the abnormal behavior recognition model, estimates the abnormal behavior, and determines whether the abnormal behavior Inform the user.
  • FIG. 4 is a flowchart illustrating a deep learning-based abnormal behavior recognition method according to an embodiment of the present invention.
  • the method for recognizing an abnormal behavior based on deep learning includes receiving 3D depth image information (S410), and performing learning by performing spatio-temporal analysis based on deep learning using 3D depth image information. It includes a step of generating a behavior recognition model (S420) and a step of analyzing the actual depth image data and determining whether the behavior is abnormal using the abnormal behavior recognition model (S430).
  • step S420 the object is separated from the background, extracted and tracked, and ground truth data for the action start and end times and the action name is generated.
  • step S420 the depth image background modeling process for object extraction, the process of calculating the difference in depth values between the object and the background, the process of separating the blob corresponding to the object from the background, and extracting the object, centers on the blob compared to the previous frame. It includes the process of tracking the blob with the minimum L2-norm as the same object.
  • step S420 in generating ground truth data, a GT for an action start time, an action end time, and an action name is generated.
  • step S420 spatiotemporal analysis is performed based on deep learning on the image learning data, and a network model for analyzing anomalous behavior is implemented based on a convolutional 3D network structure.
  • step S430 the actual depth image data is input to estimate the abnormal behavior, the event notification is performed, the actual depth image data is input to the abnormal behavior recognition model to estimate the abnormal behavior, and the user is informed of the abnormal behavior.
  • the deep learning-based abnormal behavior recognition method may be implemented in a computer system or recorded on a recording medium.
  • the computer system may include at least one processor, memory, a user input device, a data communication bus, a user output device, and storage. Each of the above-described components communicates through a data communication bus.
  • the computer system can further include a network interface coupled to the network.
  • the processor may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory and / or storage.
  • the memory and storage may include various types of volatile or nonvolatile storage media.
  • the memory may include ROM and RAM.
  • the deep learning-based abnormal behavior recognition method according to the embodiment of the present invention may be implemented in a method executable on a computer.
  • computer-readable instructions may perform the recognition method according to the present invention.
  • Computer-readable recording media includes all kinds of recording media storing data that can be read by a computer system.
  • ROM read only memory
  • RAM random access memory
  • magnetic tape magnetic tape
  • magnetic disk magnetic disk
  • flash memory an optical data storage device
  • the computer-readable recording medium may be distributed over computer systems connected through a computer communication network, and stored and executed as code readable in a distributed manner.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Image Analysis (AREA)
  • Alarm Systems (AREA)

Abstract

La présente invention concerne un dispositif de reconnaissance de comportement anormal sur la base d'un apprentissage profond et un procédé pour cela. Un dispositif de reconnaissance de comportement anormal basé sur un apprentissage profond selon la présente invention comprend : une unité de réception d'image pour recevoir une image 3D ; une mémoire dans laquelle un programme pour analyser l'image 3D de façon à déterminer la présence d'un comportement anormal est mémorisé ; et un processeur pour exécuter le programme, le processeur pour effectuer une analyse de données d'image spatio-temporelle, à l'aide d'un apprentissage profond, sur l'image 3D, générant ainsi un modèle de reconnaissance de comportement anormal, et estimant un comportement anormal.
PCT/KR2018/013948 2018-11-08 2018-11-15 Dispositif et procédé de reconnaissance de comportement anormal basés sur un apprentissage profond WO2020096111A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020180136243A KR20200055812A (ko) 2018-11-08 2018-11-08 딥러닝 기반 이상 행위 인지 장치 및 방법
KR10-2018-0136243 2018-11-08

Publications (1)

Publication Number Publication Date
WO2020096111A1 true WO2020096111A1 (fr) 2020-05-14

Family

ID=70611022

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2018/013948 WO2020096111A1 (fr) 2018-11-08 2018-11-15 Dispositif et procédé de reconnaissance de comportement anormal basés sur un apprentissage profond

Country Status (2)

Country Link
KR (1) KR20200055812A (fr)
WO (1) WO2020096111A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114938300A (zh) * 2022-05-17 2022-08-23 浙江木链物联网科技有限公司 基于设备行为分析的工控系统态势感知方法及其系统

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102397837B1 (ko) * 2020-06-25 2022-05-16 주식회사 자비스넷 엣지 컴퓨팅 기반 보안 감시 서비스 제공 장치, 시스템 및 그 동작 방법
KR102484198B1 (ko) * 2020-11-04 2023-01-04 한국전자기술연구원 이상 이벤트 탐지 방법, 장치 및 시스템

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110110585A1 (en) * 2009-11-10 2011-05-12 Samsung Electronics Co., Ltd. Image processing apparatus, method and computer-readable medium
WO2014081687A1 (fr) * 2012-11-21 2014-05-30 Pelco, Inc. Procédé et système pour compter des personnes à l'aide d'un capteur de profondeur
KR20150100141A (ko) * 2014-02-24 2015-09-02 주식회사 케이티 행동패턴 분석 장치 및 방법
KR20160011523A (ko) * 2014-07-22 2016-02-01 주식회사 에스원 3차원 영상 정보를 이용한 이상 행동 감시 장치 및 방법
KR20180028198A (ko) * 2016-09-08 2018-03-16 연세대학교 산학협력단 실시간 영상을 이용하여 위험 상황을 예측하기 위한 영상 처리 방법, 장치 및 그를 이용하여 위험 상황을 예측하는 방법, 서버

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110110585A1 (en) * 2009-11-10 2011-05-12 Samsung Electronics Co., Ltd. Image processing apparatus, method and computer-readable medium
WO2014081687A1 (fr) * 2012-11-21 2014-05-30 Pelco, Inc. Procédé et système pour compter des personnes à l'aide d'un capteur de profondeur
KR20150100141A (ko) * 2014-02-24 2015-09-02 주식회사 케이티 행동패턴 분석 장치 및 방법
KR20160011523A (ko) * 2014-07-22 2016-02-01 주식회사 에스원 3차원 영상 정보를 이용한 이상 행동 감시 장치 및 방법
KR20180028198A (ko) * 2016-09-08 2018-03-16 연세대학교 산학협력단 실시간 영상을 이용하여 위험 상황을 예측하기 위한 영상 처리 방법, 장치 및 그를 이용하여 위험 상황을 예측하는 방법, 서버

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114938300A (zh) * 2022-05-17 2022-08-23 浙江木链物联网科技有限公司 基于设备行为分析的工控系统态势感知方法及其系统

Also Published As

Publication number Publication date
KR20200055812A (ko) 2020-05-22

Similar Documents

Publication Publication Date Title
US10059569B2 (en) Monitoring system of a passenger conveyor, a passenger conveyor, and a monitoring method thereof
WO2020096111A1 (fr) Dispositif et procédé de reconnaissance de comportement anormal basés sur un apprentissage profond
KR101394242B1 (ko) 영상 감시 장치 및 영상 감시 방법
KR101748121B1 (ko) 객체 인지 기반의 실시간 영상 검지 시스템 및 그 방법
KR101764845B1 (ko) 다중 이동 물체의 겹침 제거 및 추적을 위한 영상 감시 장치 및 방법
KR102217253B1 (ko) 행동패턴 분석 장치 및 방법
CN110516623B (zh) 一种人脸识别方法、装置及电子设备
CN111783853B (zh) 一种基于可解释性的检测并恢复神经网络对抗样本方法
Zhang et al. Moving objects detection and segmentation in dynamic video backgrounds
KR101454644B1 (ko) 보행자 추적기를 이용한 서성거림을 탐지하는 방법
KR101879444B1 (ko) Cctv 구동 방법 및 시스템
Beghdadi et al. Towards the design of smart video-surveillance system
KR102113489B1 (ko) 카메라 입력 영상을 딥러닝 활용한 행동 감지 시스템 및 방법
KR101547255B1 (ko) 지능형 감시 시스템의 객체기반 검색방법
CN108108696B (zh) 一种安全防护方法、装置及系统
Suresh et al. An efficient low cost background subtraction method to extract foreground object during human tracking
Zang et al. Evaluation of an adaptive composite Gaussian model in video surveillance
WO2022267266A1 (fr) Procédé de commande de véhicule à base de reconnaissance visuelle, et dispositif
CN112287808B (zh) 运动轨迹分析告警方法、装置、系统及存储介质
WO2019124636A1 (fr) Procédé basé sur la syntaxe pour détecter une circulation routière à contresens dans une vidéo compressée
KR102518061B1 (ko) 얼굴 윤곽 추정을 통한 마스크 착용 여부 확인 방법 및 장치
CN115661735A (zh) 一种目标物检测方法、装置和计算机可读存储介质
KR102544492B1 (ko) 수영장 안전관리 장치 및 방법
KR102481215B1 (ko) 통계적 인구 관제를 위한 공간 내 객체감지시스템
WO2023120986A1 (fr) Dispositif de prédiction de défaillance de capteur de caméra utilisant l'apprentissage profond et procédé associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18939317

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18939317

Country of ref document: EP

Kind code of ref document: A1