EP3746938A4 - System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering - Google Patents

System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering Download PDF

Info

Publication number
EP3746938A4
EP3746938A4 EP19748018.9A EP19748018A EP3746938A4 EP 3746938 A4 EP3746938 A4 EP 3746938A4 EP 19748018 A EP19748018 A EP 19748018A EP 3746938 A4 EP3746938 A4 EP 3746938A4
Authority
EP
European Patent Office
Prior art keywords
foveated
detection
classification based
activity classification
visual activity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP19748018.9A
Other languages
German (de)
French (fr)
Other versions
EP3746938A2 (en
Inventor
Deepak Khosla
Ryan M. UHLENBROCK
Yang Chen
Huapeng SU
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HRL Laboratories LLC
Original Assignee
HRL Laboratories LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US15/883,822 external-priority patent/US11055872B1/en
Priority claimed from US15/947,032 external-priority patent/US10997421B2/en
Application filed by HRL Laboratories LLC filed Critical HRL Laboratories LLC
Publication of EP3746938A2 publication Critical patent/EP3746938A2/en
Publication of EP3746938A4 publication Critical patent/EP3746938A4/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
EP19748018.9A 2018-01-30 2019-01-14 System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering Pending EP3746938A4 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US15/883,822 US11055872B1 (en) 2017-03-30 2018-01-30 Real-time object recognition using cascaded features, deep learning and multi-target tracking
US201862642959P 2018-03-14 2018-03-14
US15/947,032 US10997421B2 (en) 2017-03-30 2018-04-06 Neuromorphic system for real-time visual activity recognition
PCT/US2019/013513 WO2019152177A2 (en) 2018-01-30 2019-01-14 System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering

Publications (2)

Publication Number Publication Date
EP3746938A2 EP3746938A2 (en) 2020-12-09
EP3746938A4 true EP3746938A4 (en) 2021-10-06

Family

ID=67479383

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19748018.9A Pending EP3746938A4 (en) 2018-01-30 2019-01-14 System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering

Country Status (3)

Country Link
EP (1) EP3746938A4 (en)
CN (1) CN111566661B (en)
WO (1) WO2019152177A2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111652128B (en) * 2020-06-02 2023-09-01 浙江大华技术股份有限公司 High-altitude power operation safety monitoring method, system and storage device
US20230206615A1 (en) * 2021-12-29 2023-06-29 Halliburton Energy Services, Inc. Systems and methods to determine an activity associated with an object of interest
US11776247B2 (en) * 2022-01-07 2023-10-03 Tomahawk Robotics Classification parallelization architecture
KR20230134846A (en) * 2022-03-15 2023-09-22 연세대학교 산학협력단 Multiscale object detection device and method

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008098188A2 (en) * 2007-02-08 2008-08-14 Behavioral Recognition Systems, Inc. Behavioral recognition system

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8116527B2 (en) * 2009-10-07 2012-02-14 The United States Of America As Represented By The Secretary Of The Army Using video-based imagery for automated detection, tracking, and counting of moving objects, in particular those objects having image characteristics similar to background
US9008366B1 (en) 2012-01-23 2015-04-14 Hrl Laboratories, Llc Bio-inspired method of ground object cueing in airborne motion imagery
KR102028732B1 (en) * 2012-04-05 2019-10-04 매직 립, 인코포레이티드 Wide-field of view (fov) imaging devices with active foveation capability
US9230302B1 (en) * 2013-03-13 2016-01-05 Hrl Laboratories, Llc Foveated compressive sensing system
US9147255B1 (en) 2013-03-14 2015-09-29 Hrl Laboratories, Llc Rapid object detection by combining structural information from image segmentation with bio-inspired attentional mechanisms
US9959468B2 (en) * 2015-11-06 2018-05-01 The Boeing Company Systems and methods for object tracking and classification

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008098188A2 (en) * 2007-02-08 2008-08-14 Behavioral Recognition Systems, Inc. Behavioral recognition system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ARUN HAMPAPUR ET AL: "Smart Video Surveillance [Exploring the concept of multiscale spatiotemporal tracking]", IEEE SIGNAL PROCESSING MAGAZINE, 1 March 2005 (2005-03-01), XP055347119, Retrieved from the Internet <URL:http://www.andrewsenior.com/papers/HampapurIEEESP05.pdf> [retrieved on 20170217], DOI: 10.1109/MSP.2005.1406476 *
PEIXOTO P ET AL: "A surveillance system combining peripheral and foveated motion tracking", PATTERN RECOGNITION, 1998. PROCEEDINGS. FOURTEENTH INTERNATIONAL CONFE RENCE ON BRISBANE, QLD., AUSTRALIA 16-20 AUG. 1998, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, vol. 1, 16 August 1998 (1998-08-16), pages 574 - 577, XP010297420, ISBN: 978-0-8186-8512-5, DOI: 10.1109/ICPR.1998.711208 *
See also references of WO2019152177A2 *

Also Published As

Publication number Publication date
EP3746938A2 (en) 2020-12-09
WO2019152177A2 (en) 2019-08-08
WO2019152177A3 (en) 2019-10-10
CN111566661A (en) 2020-08-21
CN111566661B (en) 2023-11-17

Similar Documents

Publication Publication Date Title
EP3779786A4 (en) Diabetic retina image classification method and system based on deep learning
EP3746938A4 (en) System and method for neuromorphic visual activity classification based on foveated detection and contextual filtering
EP3662453A4 (en) Method and system for accident detection using contextual data
EP3619681A4 (en) System and method for detecting moving obstacles based on sensory prediction from ego-motion
EP3862991A4 (en) Information processing method and information processing system
EP3819863A4 (en) Learning data collection device, learning data collection system, and learning data collection method
EP3771171A4 (en) Website detection method and system
EP3961423A4 (en) Information processing method, information processing device, and information processing system
EP3744484A4 (en) Information processing device, information processing method, and information processing system
EP3848889A4 (en) Information processing method and information processing system
EP3848860A4 (en) Qubit detection system and detection method
EP3805914A4 (en) Information processing device, information processing method, and information processing system
EP3747727A4 (en) Data processing system and method
EP3879524A4 (en) Information processing method and information processing system
EP3862995A4 (en) Information processing system, information processing device, and information processing method
EP4062369A4 (en) Systems and methods for object detection and recognition
EP3862990A4 (en) Information processing method, and information processing system
EP3813003A4 (en) Data processing method and system based on blockchain smart contract
EP3933383A4 (en) Information processing device, information processing method, and information processing system
EP3805048A4 (en) Information processing device, information processing method, and information processing system
EP3819788A4 (en) Data processing system and data processing method
EP4083844A4 (en) Information processing device, information processing system, and information processing method
EP4044149A4 (en) Information processing device, information processing system, and information processing method
EP3998390A4 (en) Anti-pinch detection method and system
EP3825970A4 (en) Cash processing system and cash processing method

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200403

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G06K0009000000

Ipc: G06N0003020000

A4 Supplementary search report drawn up and despatched

Effective date: 20210903

RIC1 Information provided on ipc code assigned before grant

Ipc: G06K 9/00 20060101ALI20210830BHEP

Ipc: G06N 3/02 20060101AFI20210830BHEP

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230525

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20230828