WO2019055112A1 - Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples - Google Patents

Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples Download PDF

Info

Publication number
WO2019055112A1
WO2019055112A1 PCT/US2018/041714 US2018041714W WO2019055112A1 WO 2019055112 A1 WO2019055112 A1 WO 2019055112A1 US 2018041714 W US2018041714 W US 2018041714W WO 2019055112 A1 WO2019055112 A1 WO 2019055112A1
Authority
WO
WIPO (PCT)
Prior art keywords
interest
region
zones
set forth
cluster
Prior art date
Application number
PCT/US2018/041714
Other languages
English (en)
Inventor
Kang-Yu NI
Tsai-Ching Lu
Original Assignee
Hrl Laboratories, Llc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hrl Laboratories, Llc filed Critical Hrl Laboratories, Llc
Priority to CN201880052092.4A priority Critical patent/CN111033411A/zh
Priority to EP18855282.2A priority patent/EP3682304A4/fr
Publication of WO2019055112A1 publication Critical patent/WO2019055112A1/fr

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
    • G05B19/0426—Programming the control sequence
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B15/00—Systems controlled by a computer
    • G05B15/02—Systems controlled by a computer electric
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B23/00—Testing or monitoring of control systems or parts thereof
    • G05B23/02—Electric testing or monitoring
    • G05B23/0205—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
    • G05B23/0208—Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the configuration of the monitoring system
    • G05B23/0216—Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2323—Non-hierarchical techniques based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
    • G06V10/426—Graphical representations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/457—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the computer system 100 is configured to utilize one or more data
  • the computer system 100 presented herein is an example computing environment in accordance with an aspect.
  • the non-limiting example of the computer system 100 is not strictly limited to being a computer system.
  • the computer system 100 represents a type of data processing analysis that may be used in accordance with various aspects described herein.
  • other computing systems may also be implemented.
  • the spirit and scope of the present technology is not limited to any single data processing environment.
  • one or more operations of various aspects of the present technology are controlled or implemented using computer- executable instructions, such as program modules, being executed by a computer.
  • program modules include routines, programs, objects, components and/or data structures that are configured to perform particular tasks or implement particular abstract data types.
  • the multi-scale multilayer graph representation for information dynamics can be used to detect and infer their dependencies that cannot be directly observed (or measured).
  • the multiple spatial scale formulation of this framework allows the construction of the multilayer graph to adapt to the activities and dynamics to reduce measurement requirements while maintaining the analysis performance.
  • a key aspect that enables this multiple spatial scale within the information dynamic framework is a flow-rate optimization method that merges graph nodes into clusters. The activities can then be
  • the system described herein can be deployed as embedded decision support modules in the cloud computing infrastructures or a stand-alone system for the application areas of complex systems, such as intelligence surveillance and reconnaissance (ISR) for posturing maritime activities (as demonstrated), crisis management, social unrests, and financial markets.
  • ISR intelligence surveillance and reconnaissance
  • the successful deployment of this technology is expected to result in detection and inference of system behaviors, activities, and dependency. Further details are provided below.
  • FIG. 5 provides a schematic illustration of a mixed, coarse-scale multilayer network.
  • the original scale 500 of multilayer network at the left is processed by a novel flow- clustering algorithm to maximize observed flow rate (middle) and to generate clusters 502 (rectangular boxes) which in turns enables the cross-layer dependency computation among the flow dependency of clustered entities (simplified edges at the right).
  • Flow ⁇ 3 is the collection of all links from nodes in cluster to nodes in cluster n d ⁇ s
  • the flow rate of flow ⁇ 3 is defined as:
  • the flow clustering problem is posed as finding k clusters that maximizes the sum of the flow rate in £ largest intra-cluster or inter-cluster flows.
  • the numbers of clusters k and flows £ are pre-defined.
  • the flow rate maximization problem is optimization problem is as follows:
  • the symmetric NMF aims to find an N X k matrix H (where k ⁇ N) with nonnegative entries H i; - > 0 that minimizes
  • the flows from region R t to region Rj are denoted as: V R . ⁇ R . (t) and C R . ⁇ R .(t) for vessels and communication, respectively.
  • the present method is directed to capturing the dependency of these flows (edges) and their changes across different types of flows.
  • Sensor data e.g., from a plane, satellite, etc.
  • the time series will be density of vessels and communications in each region for the layers: V R . ⁇ R .(t) and C R . ⁇ R . (t), respectively.
  • FIG. 7 provides a schematic illustration of the discovery of inter4ayer dependency relations: the communication flow between node 1 and node 12 in the upper panel 700 influences the vessel flows on the path of node 1 ⁇ 4 ⁇ 8 ⁇ 12 in the bottom panel 702.
  • Such flow dependency (edges) between layers are inferred automatically by the ATE methods.
  • the flow clustering process 300 receives inputs as a vessel flow tensor and, based on that, generates cluster membership. The process is provided below and further depicted in FIG. 3 :
  • Output, d An N x 1 vector d that indicates the cluster membership with entries from ⁇ 1,2, k ⁇ .
  • the vessel flow clustering process was performed with a set of data to validate the system and process.
  • Provided below is an example to illustrate that flow clustering summarizes vessel flow and reduces the number of flows.
  • the example graph in FIG. 8 is a l0 x l0 regular grid (therefore 100 nodes) 800 with three major communications from node 3 to node 77, node 35 to node 77, and node 59 to node 77.
  • the communication frequency of these are 5%, i.e., if the sampling rate is per minute, in average 5 times out of 100 minutes are active.
  • There is also noise communication with a 2% frequency with a pair of nodes randomly picked at each time.
  • the vessel flow is simulated with the partial differential equation described above with random initialization for the vessel density on each node.
  • the grid 800 shows the corresponding vessel flows for the major communications.
  • FIG. 8 also depicts the vessel flow clustering results 802 with 10 clusters, where each cluster is color-coded (nodes with the same color is a cluster).
  • Vessel flow summarization 804 is also depicted, showing the summarized version of the vessel flows in the grid 800.
  • the vessel flow summarization 804 is indicated by the directed edges, showing that the number of flows is reduced.
  • the system described herein detects communication and vessel flow dependency with low resolution and cue regions of interest with TE for multiscale monitoring (depicted as element 302 in FIG. 3).
  • the process is provided in further detail below:
  • Inputs. V, C and k An N X N X T vessel flow tensor V where entry i;t indicates the amount of vessels flowing from node i to node j at time t.
  • An N X N X T communication tensor C where entry C i;t indicates the amount of communication from node i to node j at time t.
  • the number of clusters is k.
  • decision making tools e.g. situation awareness tool in monitoring vessel movements in/out of contested water
  • exploratory analysis e.g. drill down to high-flow entropy zones based on dependent clusters
  • refine units of analysis for tracking purpose e.g., use corresponding multiscale flow and corresponding dependency links.
  • a multi-scale vessel flow graph 904 is generated.
  • regions of interest can be cued for multiscale vessel flow monitoring.
  • the system can zoom-into regions of interest while maintaining sufficient monitoring of low interest regions.
  • FIG. 9C provides a snapshot of the multiscale vessel flow 904 where the dependent clusters (dependent on communications) have the original resolution and the rest have the low resolution.
  • a mobile device display e.g., a mobile device display, a virtual reality display, an augmented reality display, a computer monitor, a motor, a machine, a drone, a camera, etc.
  • a drone or other autonomous vehicle may be controlled to move to an area within the multi-scale vessel flow graph based on identified dependent flows/clusters or their changes over time.
  • the system can generate the multi-scale vessel graph by applying the algorithm on data collected via satellites, determine regions of interest with thresholds (e.g., a significant deviation/changes in flow-dependency within a priori-determined time window), and send drones to the regions of interest to collect finer-grained data, or perform monitoring and tracking with desired level of coverage (e.g., zone size) for given constraints (e.g., # of drones available, processing powers, etc.).
  • a camera may be controlled to orient towards the region of interest and zoom in as needed. In other words, actuators or motors are activated to cause the camera (or sensor) to move or zoom in on the region of interest.
  • any recitation of "means for” is intended to evoke a means-plus-function reading of an element and a claim, whereas, any elements that do not specifically use the recitation "means for”, are not intended to be read as means-plus-function elements, even if the claim otherwise includes the word "means”.
  • any recitation of "means for” is intended to evoke a means-plus-function reading of an element and a claim, whereas, any elements that do not specifically use the recitation "means for”, are not intended to be read as means-plus-function elements, even if the claim otherwise includes the word "means”.
  • particular method steps have been recited in a particular order, the method steps may occur in any desired order and fall within the scope of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Discrete Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Complex Calculations (AREA)
  • Traffic Control Systems (AREA)

Abstract

La présente invention concerne un système de surveillance à échelles multiples. Pendant l'exploitation, le système reçoit des données de surveillance d'une scène comprenant une pluralité de zones. Les données de surveillance comprennent un tenseur de flux d'objet (V) indiquant un nombre d'objets circulant d'une zone à une autre zone à un instant t et un tenseur de communication d'objet (C) indiquant un nombre de communications qui sont envoyées d'une zone à une autre zone à l'instant t. Le système détermine ensuite une appartenance à une grappe de la pluralité de zones. Des liens de dépendance entre des communications et des flux sont ensuite déterminés. Au moins une grappe d'une ou de plusieurs zones est désignée comme région d'intérêt sur la base des liens de dépendance, ce qui permet au système de commander un dispositif en fonction de ladite région désignée d'intérêt.
PCT/US2018/041714 2017-09-12 2018-07-11 Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples WO2019055112A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880052092.4A CN111033411A (zh) 2017-09-12 2018-07-11 用于多尺度监测的粗粒度多层流信息动态
EP18855282.2A EP3682304A4 (fr) 2017-09-12 2018-07-11 Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762557733P 2017-09-12 2017-09-12
US62/557,733 2017-09-12

Publications (1)

Publication Number Publication Date
WO2019055112A1 true WO2019055112A1 (fr) 2019-03-21

Family

ID=65723837

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/041714 WO2019055112A1 (fr) 2017-09-12 2018-07-11 Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples

Country Status (3)

Country Link
EP (1) EP3682304A4 (fr)
CN (1) CN111033411A (fr)
WO (1) WO2019055112A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150310721A1 (en) * 2012-08-06 2015-10-29 Superior Edge, Inc. Determining in-season crop status in an agricultural crop and alerting users
US20150379408A1 (en) * 2014-06-30 2015-12-31 Microsoft Corporation Using Sensor Information for Inferring and Forecasting Large-Scale Phenomena

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235143A1 (en) * 2003-12-30 2015-08-20 Kantrack Llc Transfer Learning For Predictive Model Development
US8107677B2 (en) * 2008-02-20 2012-01-31 International Business Machines Corporation Measuring a cohort'S velocity, acceleration and direction using digital video
US9049464B2 (en) * 2011-06-07 2015-06-02 Qualcomm Incorporated Multiple description coding with plural combined diversity
US8860602B2 (en) * 2012-10-09 2014-10-14 Accipiter Radar Technologies Inc. Device and method for cognitive radar information network
US9135826B2 (en) * 2012-12-26 2015-09-15 Sap Se Complex event processing for moving objects
US9380275B2 (en) * 2013-01-30 2016-06-28 Insitu, Inc. Augmented video system providing enhanced situational awareness
US20150301208A1 (en) * 2014-04-22 2015-10-22 Westerngeco L.L.C. Seismic data processing
US9963215B2 (en) * 2014-12-15 2018-05-08 Leidos, Inc. System and method for fusion of sensor data to support autonomous maritime vessels
KR101703906B1 (ko) * 2015-01-22 2017-02-08 한국전자통신연구원 선박 관제 시스템 및 그것의 선박 관제 방법
US20170178524A1 (en) * 2015-05-06 2017-06-22 Ocula Corporation Swim Lap Counting and Timing System and Methods for Event Detection from Noisy Source Data
US9727976B1 (en) * 2015-09-08 2017-08-08 Sandia Corporation Geospatial-temporal semantic graph representations of trajectories from remote sensing and geolocation data
US10002471B2 (en) * 2015-09-30 2018-06-19 Ants Technology (Hk) Limited Systems and methods for autonomous vehicle navigation
ES2613310B1 (es) * 2015-10-23 2018-03-02 Consejo Superior De Investigaciones Científicas (Csic) Vehículo aéreo no tripulado biomimético y zoosemiótico dirigido por piloto automático para vuelos de precisión y/o persecución

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150310721A1 (en) * 2012-08-06 2015-10-29 Superior Edge, Inc. Determining in-season crop status in an agricultural crop and alerting users
US20150379408A1 (en) * 2014-06-30 2015-12-31 Microsoft Corporation Using Sensor Information for Inferring and Forecasting Large-Scale Phenomena

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
FILIZ BUNYAK ET AL.: "Flux Tensor Constrained Geodesic Active Contours with Sensor Fusion for Persistent Object Tracking", JOURNAL OF MULTIMEDIA, vol. 2, 2007, pages 4, XP055582642 *
HUIRU CAO ET AL.: "Cloud-Assisted UAV Data Collection for Multiple Emerging Events in Distributed WSNs", SENSORS, 7 August 2017 (2017-08-07), XP055582640 *
SANGIL JWA ET AL.: "MULTI-UAV SENSING OVER URBAN AREAS VIA LAYERED DATA FUSION", 2007 IEEE /SP 14TH WORKSHOP ON STATISTICAL SIGNAL PROCESSING, 2007, pages 576 - 580, XP031134135 *
See also references of EP3682304A4 *

Also Published As

Publication number Publication date
EP3682304A4 (fr) 2021-07-14
EP3682304A1 (fr) 2020-07-22
CN111033411A (zh) 2020-04-17

Similar Documents

Publication Publication Date Title
Wang et al. Earthquake prediction based on spatio-temporal data mining: an LSTM network approach
US11954317B2 (en) Systems and method for a customizable layered map for visualizing and analyzing geospatial data
Wang et al. A spatial-adaptive sampling procedure for online monitoring of big data streams
US20180332256A1 (en) Coarse-grained multilayer flow information dynamics for multiscale monitoring
US11386507B2 (en) Tensor-based predictions from analysis of time-varying graphs
US11225337B2 (en) Parallel calculation of satellite access windows and native program implementation framework
US10528600B1 (en) System to identify unknown communication behavior relationships from time series
Armstrong et al. The Internet of Things and fast data streams: prospects for geospatial data science in emerging information ecosystems
US10089549B1 (en) Valley search method for estimating ego-motion of a camera from videos
Pu et al. Constrained least squares algorithms for nonlinear unmixing of hyperspectral imagery
Huang et al. Implementation of the parallel mean shift-based image segmentation algorithm on a GPU cluster
Shi et al. Parallelizing maximum likelihood classification on computer cluster and graphics processing unit for supervised image classification
Xing et al. A land use/land cover change geospatial cyberinfrastructure to integrate big data and temporal topology
Muyskens et al. Star–galaxy image separation with computationally efficient gaussian process classification
Leykam et al. Topological data analysis and machine learning
Antunes et al. Proof of concept of a novel cloud computing approach for object-based remote sensing data analysis and classification
Thomopoulos et al. Automated real-time risk assessment for airport passengers using a deep learning architecture
Vrins et al. On the entropy minimization of a linear mixture of variables for source separation
EP3682304A1 (fr) Dynamique d'informations de flux multicouche à gros grain servant à une surveillance à échelles multiples
Lim et al. Detection of salient regions in crowded scenes
ALDabbas et al. Complex event processing based analysis of cassini–huygens interplanetary dataset
Kalra et al. COVID-based edge-drone application approach
Feder Drones Move From" Nice To Have" to Strategic Resources for Projects
Ji et al. A new non-negative matrix factorization method based on barycentric coordinates for endmember extraction in hyperspectral remote sensing
CN114581836A (zh) 异常行为检测方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18855282

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2018855282

Country of ref document: EP

Effective date: 20200414