US10489660B2 - Video processing with object identification - Google Patents
Video processing with object identification Download PDFInfo
- Publication number
- US10489660B2 US10489660B2 US15/413,329 US201715413329A US10489660B2 US 10489660 B2 US10489660 B2 US 10489660B2 US 201715413329 A US201715413329 A US 201715413329A US 10489660 B2 US10489660 B2 US 10489660B2
- Authority
- US
- United States
- Prior art keywords
- video
- human
- regions
- user
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000012545 processing Methods 0.000 title description 14
- 238000000034 method Methods 0.000 claims abstract description 55
- 241000282414 Homo sapiens Species 0.000 claims abstract description 44
- 239000000203 mixture Substances 0.000 claims description 5
- 230000004044 response Effects 0.000 claims description 5
- 238000005286 illumination Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims 1
- 238000013500 data storage Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 13
- 238000003860 storage Methods 0.000 description 13
- 238000004422 calculation algorithm Methods 0.000 description 11
- 238000001514 detection method Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 230000015654 memory Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000003936 working memory Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000001143 conditioned effect Effects 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G06K9/00771—
-
- G06K9/00664—
-
- G06K9/2081—
-
- G06K9/3233—
-
- G06K9/342—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/235—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on user input or interaction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V10/7747—Organisation of the process, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Definitions
- APCF features in the whole image can be extracted.
- a sliding window can scan the whole frame.
- a detector M in each sliding window, a chain of weak classifiers will be used to determine if this area is a human or not.
- each sliding window needs to pass every weak classifier in order to decide this is the human.
- feature detection may occur within a user specified region of interest.
- a user may draw a bounding box in one or more frames of a scene recorded by a camera.
- the bounding box may define the region of interest within which events may be identified and/or tracked.
- a user may specify two or more points within a frame and a polygonal shaped bounding box may be created based on these points.
- the user may have the option to specify whether the bounding box bounds the region of interest or should be excluded from the region of interest.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (17)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/413,329 US10489660B2 (en) | 2016-01-21 | 2017-01-23 | Video processing with object identification |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662281285P | 2016-01-21 | 2016-01-21 | |
US15/413,329 US10489660B2 (en) | 2016-01-21 | 2017-01-23 | Video processing with object identification |
Publications (2)
Publication Number | Publication Date |
---|---|
US20170213091A1 US20170213091A1 (en) | 2017-07-27 |
US10489660B2 true US10489660B2 (en) | 2019-11-26 |
Family
ID=59360695
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/413,329 Expired - Fee Related US10489660B2 (en) | 2016-01-21 | 2017-01-23 | Video processing with object identification |
Country Status (2)
Country | Link |
---|---|
US (1) | US10489660B2 (en) |
WO (1) | WO2017151241A2 (en) |
Families Citing this family (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10839203B1 (en) | 2016-12-27 | 2020-11-17 | Amazon Technologies, Inc. | Recognizing and tracking poses using digital imagery captured from multiple fields of view |
GB2560177A (en) | 2017-03-01 | 2018-09-05 | Thirdeye Labs Ltd | Training a computational neural network |
GB2560387B (en) | 2017-03-10 | 2022-03-09 | Standard Cognition Corp | Action identification using neural networks |
US10691950B2 (en) * | 2017-03-10 | 2020-06-23 | Turing Video, Inc. | Activity recognition method and system |
US10699421B1 (en) | 2017-03-29 | 2020-06-30 | Amazon Technologies, Inc. | Tracking objects in three-dimensional space using calibrated visual cameras and depth cameras |
US10127438B1 (en) | 2017-08-07 | 2018-11-13 | Standard Cognition, Corp | Predicting inventory events using semantic diffing |
US10474991B2 (en) | 2017-08-07 | 2019-11-12 | Standard Cognition, Corp. | Deep learning-based store realograms |
US11023850B2 (en) | 2017-08-07 | 2021-06-01 | Standard Cognition, Corp. | Realtime inventory location management using deep learning |
US10853965B2 (en) | 2017-08-07 | 2020-12-01 | Standard Cognition, Corp | Directional impression analysis using deep learning |
US10445694B2 (en) | 2017-08-07 | 2019-10-15 | Standard Cognition, Corp. | Realtime inventory tracking using deep learning |
US11250376B2 (en) | 2017-08-07 | 2022-02-15 | Standard Cognition, Corp | Product correlation analysis using deep learning |
US11200692B2 (en) | 2017-08-07 | 2021-12-14 | Standard Cognition, Corp | Systems and methods to check-in shoppers in a cashier-less store |
US10650545B2 (en) | 2017-08-07 | 2020-05-12 | Standard Cognition, Corp. | Systems and methods to check-in shoppers in a cashier-less store |
US11232687B2 (en) | 2017-08-07 | 2022-01-25 | Standard Cognition, Corp | Deep learning-based shopper statuses in a cashier-less store |
US10133933B1 (en) | 2017-08-07 | 2018-11-20 | Standard Cognition, Corp | Item put and take detection using image recognition |
US10055853B1 (en) | 2017-08-07 | 2018-08-21 | Standard Cognition, Corp | Subject identification and tracking using image recognition |
US10474988B2 (en) | 2017-08-07 | 2019-11-12 | Standard Cognition, Corp. | Predicting inventory events using foreground/background processing |
US11232294B1 (en) | 2017-09-27 | 2022-01-25 | Amazon Technologies, Inc. | Generating tracklets from digital imagery |
US11030442B1 (en) | 2017-12-13 | 2021-06-08 | Amazon Technologies, Inc. | Associating events with actors based on digital imagery |
US11284041B1 (en) | 2017-12-13 | 2022-03-22 | Amazon Technologies, Inc. | Associating items with actors based on digital imagery |
US11468698B1 (en) | 2018-06-28 | 2022-10-11 | Amazon Technologies, Inc. | Associating events with actors using digital imagery and machine learning |
US11482045B1 (en) | 2018-06-28 | 2022-10-25 | Amazon Technologies, Inc. | Associating events with actors using digital imagery and machine learning |
US11468681B1 (en) | 2018-06-28 | 2022-10-11 | Amazon Technologies, Inc. | Associating events with actors using digital imagery and machine learning |
CN113228626B (en) * | 2018-12-29 | 2023-04-07 | 浙江大华技术股份有限公司 | Video monitoring system and method |
US11232575B2 (en) | 2019-04-18 | 2022-01-25 | Standard Cognition, Corp | Systems and methods for deep learning-based subject persistence |
US11398094B1 (en) | 2020-04-06 | 2022-07-26 | Amazon Technologies, Inc. | Locally and globally locating actors by digital cameras and machine learning |
US11443516B1 (en) | 2020-04-06 | 2022-09-13 | Amazon Technologies, Inc. | Locally and globally locating actors by digital cameras and machine learning |
US11361468B2 (en) | 2020-06-26 | 2022-06-14 | Standard Cognition, Corp. | Systems and methods for automated recalibration of sensors for autonomous checkout |
US11303853B2 (en) | 2020-06-26 | 2022-04-12 | Standard Cognition, Corp. | Systems and methods for automated design of camera placement and cameras arrangements for autonomous checkout |
CN111986233B (en) * | 2020-08-20 | 2023-02-10 | 西安电子科技大学 | Large-scene minimum target remote sensing video tracking method based on feature self-learning |
CN112733650B (en) * | 2020-12-29 | 2024-05-07 | 深圳云天励飞技术股份有限公司 | Target face detection method and device, terminal equipment and storage medium |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040120581A1 (en) | 2002-08-27 | 2004-06-24 | Ozer I. Burak | Method and apparatus for automated video activity analysis |
US20080170751A1 (en) | 2005-02-04 | 2008-07-17 | Bangjun Lei | Identifying Spurious Regions In A Video Frame |
US20080263012A1 (en) * | 2005-09-01 | 2008-10-23 | Astragroup As | Post-Recording Data Analysis and Retrieval |
US20090041297A1 (en) | 2005-05-31 | 2009-02-12 | Objectvideo, Inc. | Human detection and tracking for security applications |
US20090231433A1 (en) | 2008-03-17 | 2009-09-17 | International Business Machines Corporation | Scene selection in a vehicle-to-vehicle network |
US20090307261A1 (en) | 2008-06-05 | 2009-12-10 | Apple Inc. | Categorization of Digital Media Based on Media Characteristics |
US20130170760A1 (en) * | 2011-12-29 | 2013-07-04 | Pelco, Inc. | Method and System for Video Composition |
US8665333B1 (en) | 2007-01-30 | 2014-03-04 | Videomining Corporation | Method and system for optimizing the observation and annotation of complex human behavior from video sources |
US20140333775A1 (en) | 2013-05-10 | 2014-11-13 | Robert Bosch Gmbh | System And Method For Object And Event Identification Using Multiple Cameras |
-
2017
- 2017-01-23 US US15/413,329 patent/US10489660B2/en not_active Expired - Fee Related
- 2017-01-23 WO PCT/US2017/014629 patent/WO2017151241A2/en active Application Filing
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040120581A1 (en) | 2002-08-27 | 2004-06-24 | Ozer I. Burak | Method and apparatus for automated video activity analysis |
US20080170751A1 (en) | 2005-02-04 | 2008-07-17 | Bangjun Lei | Identifying Spurious Regions In A Video Frame |
US20090041297A1 (en) | 2005-05-31 | 2009-02-12 | Objectvideo, Inc. | Human detection and tracking for security applications |
US20080263012A1 (en) * | 2005-09-01 | 2008-10-23 | Astragroup As | Post-Recording Data Analysis and Retrieval |
US8665333B1 (en) | 2007-01-30 | 2014-03-04 | Videomining Corporation | Method and system for optimizing the observation and annotation of complex human behavior from video sources |
US20090231433A1 (en) | 2008-03-17 | 2009-09-17 | International Business Machines Corporation | Scene selection in a vehicle-to-vehicle network |
US20090307261A1 (en) | 2008-06-05 | 2009-12-10 | Apple Inc. | Categorization of Digital Media Based on Media Characteristics |
US20130170760A1 (en) * | 2011-12-29 | 2013-07-04 | Pelco, Inc. | Method and System for Video Composition |
US20140333775A1 (en) | 2013-05-10 | 2014-11-13 | Robert Bosch Gmbh | System And Method For Object And Event Identification Using Multiple Cameras |
Non-Patent Citations (5)
Title |
---|
European Partial Search Report dated Apr. 2, 2019 as received in Application No. 15895256.4. |
Extended Search Report dated Jul. 10, 2019 as received in Application No. 15895256.4. |
International Search Report dated Sep. 27, 2017 as received in Application No. PCT/US2017/014629. |
Kruegle, H., "Smart Camera" In: "CCTV Surveillance: Analog and Digital Video Practices and Technology," pp. 122-123, (Jan. 1, 2006). |
Written Opinion of the International Searching Authority dated Sep. 27, 2017 as received in Application No. PCT/US2017/014629. |
Also Published As
Publication number | Publication date |
---|---|
WO2017151241A3 (en) | 2017-11-09 |
WO2017151241A2 (en) | 2017-09-08 |
US20170213091A1 (en) | 2017-07-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10489660B2 (en) | Video processing with object identification | |
KR101942808B1 (en) | Apparatus for CCTV Video Analytics Based on Object-Image Recognition DCNN | |
WO2016201683A1 (en) | Cloud platform with multi camera synchronization | |
IL261696A (en) | System and method for training object classifier by machine learning | |
US10140508B2 (en) | Method and apparatus for annotating a video stream comprising a sequence of frames | |
US20170053192A1 (en) | Systems and methods for detecting parking occupancy status | |
US20180253596A1 (en) | System and method for person re-identification using overhead view images | |
US20170193810A1 (en) | Video event detection and notification | |
US10140554B2 (en) | Video processing | |
WO2016126885A1 (en) | Method and apparatus for target acquisition | |
US10997469B2 (en) | Method and system for facilitating improved training of a supervised machine learning process | |
US10410059B2 (en) | Cloud platform with multi camera synchronization | |
CN113228626B (en) | Video monitoring system and method | |
KR20140095333A (en) | Method and apparratus of tracing object on image | |
KR102127276B1 (en) | The System and Method for Panoramic Video Surveillance with Multiple High-Resolution Video Cameras | |
US11954880B2 (en) | Video processing | |
US20190370553A1 (en) | Filtering of false positives using an object size model | |
US20210287051A1 (en) | Methods and systems for recognizing object using machine learning model | |
US20220180102A1 (en) | Reducing false negatives and finding new classes in object detectors | |
CN113515978B (en) | Data processing method, device and storage medium | |
KR102375541B1 (en) | Apparatus for Providing Artificial Intelligence Service with structured consistency loss and Driving Method Thereof | |
CN111062337B (en) | People stream direction detection method and device, storage medium and electronic equipment | |
US20190370560A1 (en) | Detecting changes in object size over long time scales | |
WO2017012123A1 (en) | Video processing | |
CN113836979A (en) | Action recognition method and device, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: WIZR LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CAO, SONG;DUAN, GENQUAN;REEL/FRAME:041212/0630 Effective date: 20170207 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
STCC | Information on status: application revival |
Free format text: WITHDRAWN ABANDONMENT, AWAITING EXAMINER ACTION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20231126 |