EP3682304A1 - Grobkörnige, mehrschichtige strömungsinformationsdynamik für mehrskalige überwachung - Google Patents

Grobkörnige, mehrschichtige strömungsinformationsdynamik für mehrskalige überwachung

Info

Publication number
EP3682304A1
EP3682304A1 EP18855282.2A EP18855282A EP3682304A1 EP 3682304 A1 EP3682304 A1 EP 3682304A1 EP 18855282 A EP18855282 A EP 18855282A EP 3682304 A1 EP3682304 A1 EP 3682304A1
Authority
EP
European Patent Office
Prior art keywords
interest
region
zones
set forth
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP18855282.2A
Other languages
English (en)
French (fr)
Other versions
EP3682304A4 (de
Inventor
Kang-Yu NI
Tsai-Ching Lu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HRL Laboratories LLC
Original Assignee
HRL Laboratories LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by HRL Laboratories LLC filed Critical HRL Laboratories LLC
Publication of EP3682304A1 publication Critical patent/EP3682304A1/de
Publication of EP3682304A4 publication Critical patent/EP3682304A4/de
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/04Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/042Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
    • G05B19/0426Programming the control sequence
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B15/00Systems controlled by a computer
    • G05B15/02Systems controlled by a computer electric
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B23/00Testing or monitoring of control systems or parts thereof
    • G05B23/02Electric testing or monitoring
    • G05B23/0205Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults
    • G05B23/0208Electric testing or monitoring by means of a monitoring system capable of detecting and responding to faults characterized by the configuration of the monitoring system
    • G05B23/0216Human interface functionality, e.g. monitoring system providing help to the user in the selection of tests or in its configuration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • G06F18/232Non-hierarchical techniques
    • G06F18/2323Non-hierarchical techniques based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00Machine learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/42Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/422Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation for representing the structure of the pattern or shape of an object therefor
    • G06V10/426Graphical representations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/457Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • the present invention relates to monitoring system and, more specifically, to a system for multiscale monitoring using coarse grained, multilayer flow information dynamics.
  • the system comprises one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform several operations, such as receiving surveillance data of a scene having a plurality of zones, the surveillance data having an object flow tensor V indicating a number of objects flowing from one zone to another zone at time t and an object communication tensor C indicating a number of communications sending from one zone to another zone at time t; determining a cluster membership of the plurality of zones; determining dependency links between communications and flows; designating at least one cluster of one or more zones as a region of interest based on the dependency links; and controlling a device based on the region of interest.
  • determining a cluster membership of the plurality of zones further comprises operations of constructing an adjacency matrix ⁇ based on the object flow tensor V; symmetrizing the adjacency matrix A; solving nonnegative matrix factorization of the symmetrized adjacency matrix; and assigning cluster membership of the objects in each of the plurality of zones to generate the cluster membership.
  • communications and flows further comprises operations of constructing a low- resolution flow tensor based on the cluster membership by merging vessel flows V within each cluster; determining flow transfer entropy; and identifying dependency links and dependent clusters by thresholding.
  • designating at least one cluster of one or more zones as a region of interest based on the dependency links includes designating the dependent clusters as regions of interest.
  • controlling a device based on the region of interest further comprises causing an unmanned aerial vehicle to move to the region of interest.
  • controlling a device based on the region of interest furthermore, controlling a device based on the region of interest further
  • the present invention also includes a computer program product and a computer implemented method.
  • the computer program product includes computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having one or more processors, such that upon execution of the instructions, the one or more processors perform the operations listed herein.
  • the computer implemented method includes an act of causing a computer to execute such instructions and perform the resulting operations.
  • the patent or application file contains at least one drawing executed in color.
  • FIG. 1 is a block diagram depicting the components of a system
  • FIG. 2 is an illustration of a computer program product embodying an aspect of the present invention
  • FIG. 3 is a flowchart depicting a process for multiscale monitoring
  • FIG. 4 is a multilayer information dynamic model for finding activity dependencies across layers according to various embodiments of the present invention.
  • FIG. 5 is a schematic illustration of a mixed, coarse-scale multilayer network
  • FIG. 6 is an illustration depicting how the multiple spatial scales of the multilayer information dynamic framework offers the ability to zoom in to a region of interest;
  • FIG. 7 is a schematic illustration of the discovery of inter-layer dependency relations;
  • FIG. 8 is an illustration showing that flow clustering summarizes vessel flow and reduces the number of flows
  • FIG. 9A is an example of a vessel flow graph at full resolution
  • FIG. 9B is an example of a vessel flow graph at low-resolution
  • FIG. 9C is an example of a vessel flow graph, depicting a multi-scale version.
  • FIG. 10 is a block diagram depicting control of a device according to various embodiments.
  • the present invention relates to monitoring system and, more
  • N-K. Ni and T-C. Lu Information Dynamic Spectrum Characterizes System Instability toward Critical Transitions, EPJ Data Science, 3:28, 2014
  • the first is a system for multiscale monitoring.
  • the system is typically in the form of a computer system operating software or in the form of a "hard-coded" instruction set. This system may be incorporated into a wide variety of devices that provide different functionalities.
  • the second principal aspect is a method, typically in the form of software, operated using a data processing system (computer).
  • the third principal aspect is a computer program product.
  • the computer program product generally represents computer-readable instructions stored on a non- transitory computer-readable medium such as an optical storage device, e.g., a compact disc (CD) or digital versatile disc (DVD), or a magnetic storage device such as a floppy disk or magnetic tape.
  • Other, non-limiting examples of computer- readable media include hard disks, read-only memory (ROM), and flash-type memories.
  • the computer system 100 is configured to perform calculations, processes, operations, and/or functions associated with a program or algorithm.
  • certain processes and steps discussed herein are realized as a series of instructions (e.g., software program) that reside within computer readable memory units and are executed by one or more processors of the computer system 100. When executed, the instructions cause the computer system 100 to perform specific actions and exhibit specific behavior, such as described herein.
  • the computer system 100 may include an address/data bus 102 that is configured to communicate information. Additionally, one or more data processing units, such as a processor 104 (or processors), are coupled with the address/data bus 102.
  • the processor 104 is configured to process information and instructions.
  • the processor 104 is a microprocessor.
  • the processor 104 may be a different type of processor such as a parallel processor, application-specific integrated circuit (ASIC), programmable logic array (PLA), complex programmable logic device (CPLD), or a field programmable gate array (FPGA).
  • ASIC application-specific integrated circuit
  • PLA programmable logic array
  • CPLD complex programmable logic device
  • FPGA field programmable gate array
  • the computer system 100 is configured to utilize one or more data
  • the computer system 100 may include a volatile memory unit 106 (e.g., random access memory (“RAM”), static RAM, dynamic RAM, etc.) coupled with the address/data bus 102, wherein a volatile memory unit 106 is configured to store information and instructions for the processor 104.
  • the computer system 100 further may include a non-volatile memory unit 108 (e.g., read-only memory (“ROM”), programmable ROM (“PROM”), erasable programmable ROM
  • EPROM electrically erasable programmable ROM
  • EEPROM electrically erasable programmable ROM
  • flash memory etc.
  • the computer system 100 may execute instructions retrieved from an online data storage unit such as in "Cloud” computing.
  • the computer system 100 also may include one or more interfaces, such as an interface 110, coupled with the address/data bus 102.
  • the one or more interfaces are configured to enable the computer system 100 to interface with other electronic devices and computer systems.
  • the communication interfaces implemented by the one or more interfaces may include wireline (e.g., serial cables, modems, network adaptors, etc.) and/or wireless (e.g., wireless modems, wireless network adaptors, etc.)
  • the computer system 100 may include an input device 112 coupled with the address/data bus 102, wherein the input device 112 is configured to communicate information and command selections to the processor 100.
  • the input device 112 is an alphanumeric input device, such as a keyboard, that may include alphanumeric and/or function keys.
  • the input device 112 may be an input device other than an
  • the computer system 100 may include a cursor control device 114 coupled with the address/data bus 102, wherein the cursor control device 114 is configured to communicate user input information and/or command selections to the processor 100.
  • the cursor control device 114 is implemented using a device such as a mouse, a track-ball, a track-pad, an optical tracking device, or a touch screen.
  • the cursor control device 114 is directed and/or activated via input from the input device 112, such as in response to the use of special keys and key sequence commands associated with the input device 112.
  • the cursor control device 114 is configured to be directed or guided by voice commands.
  • the computer system 100 further may include one or more optional computer usable data storage devices, such as a storage device 116, coupled with the address/data bus 102.
  • the storage device 116 is configured to store information and/or computer executable instructions.
  • the storage device 116 is a storage device such as a magnetic or optical disk drive (e.g., hard disk drive (“HDD”), floppy diskette, compact disk read only memory (“CD- ROM”), digital versatile disk (“DVD”)).
  • a display device e.g., a display device
  • the display device 118 is coupled with the address/data bus 102, wherein the display device 118 is configured to display video and/or graphics.
  • the display device 118 may include a cathode ray tube ("CRT"), liquid crystal display (“LCD”), field emission display (“FED”), plasma display, or any other display device suitable for displaying video and/or graphic images and alphanumeric characters recognizable to a user.
  • CTR cathode ray tube
  • LCD liquid crystal display
  • FED field emission display
  • plasma display or any other display device suitable for displaying video and/or graphic images and alphanumeric characters recognizable to a user.
  • the computer system 100 presented herein is an example computing environment in accordance with an aspect.
  • the non-limiting example of the computer system 100 is not strictly limited to being a computer system.
  • the computer system 100 represents a type of data processing analysis that may be used in accordance with various aspects described herein.
  • other computing systems may also be implemented.
  • the spirit and scope of the present technology is not limited to any single data processing environment.
  • one or more operations of various aspects of the present technology are controlled or implemented using computer- executable instructions, such as program modules, being executed by a computer.
  • program modules include routines, programs, objects, components and/or data structures that are configured to perform particular tasks or implement particular abstract data types.
  • an aspect provides that one or more aspects of the present technology are implemented by utilizing one or more distributed computing environments, such as where tasks are performed by remote processing devices that are linked through a communications network, or such as where various program modules are located in both local and remote computer- storage media including memory-storage devices.
  • FIG. 2 An illustrative diagram of a computer program product (i.e., storage device) embodying the present invention is depicted in FIG. 2.
  • the computer program product is depicted as floppy disk 200 or an optical disk 202 such as a CD or DVD.
  • the computer program product generally represents computer-readable instructions stored on any compatible non- transitory computer-readable medium.
  • the term "instructions” as used with respect to this invention generally indicates a set of operations to be performed on a computer, and may represent pieces of a whole program or individual, separable, software modules.
  • Non-limiting examples of "instruction” include computer program code (source or object code) and "hard-coded" electronics (i.e. computer operations coded into a computer chip).
  • the "instruction" is stored on any non- transitory computer-readable medium, such as in the memory of a computer or on a floppy disk, a CD-ROM, and a flash drive. In either event, the instructions are encoded on a non-transitory computer-readable medium.
  • This disclosure provides a unique multi-scale multilayer graph
  • the multi-scale multilayer graph representation for information dynamics can be used to detect and infer their dependencies that cannot be directly observed (or measured).
  • the multiple spatial scale formulation of this framework allows the construction of the multilayer graph to adapt to the activities and dynamics to reduce measurement requirements while maintaining the analysis performance.
  • a key aspect that enables this multiple spatial scale within the information dynamic framework is a flow-rate optimization method that merges graph nodes into clusters. The activities can then be
  • a purpose of this invention is to efficiently direct computing resources to monitor and analyze emerging activities from multiple sources at multiple scales.
  • the advantages of the new feature of the multiscale, multilayer dynamic information are two-fold: 1) It reduces computations without losing the ability to find activity dependencies.
  • the coarse resolution corresponds to sparse activity dependency; therefore, it provides better abstraction and enables coverage of larger graphs. 2) It has the ability to zoom-in or zoom-out of regions of interest in order to provide better actionable insights to an analyst or other system operations.
  • the system described herein can be deployed as embedded decision support modules in the cloud computing infrastructures or a stand-alone system for the application areas of complex systems, such as intelligence surveillance and reconnaissance (ISR) for posturing maritime activities (as demonstrated), crisis management, social unrests, and financial markets.
  • ISR intelligence surveillance and reconnaissance
  • the successful deployment of this technology is expected to result in detection and inference of system behaviors, activities, and dependency. Further details are provided below.
  • FIG. 3 illustrates a flowchart depicting a process for multiscale monitoring, including the flow clustering process 300 that receives inputs as a vessel flow tensor and generates cluster membership, following by the multi-scale multilayer information dynamics process 302 that provides dependency links, dependent clusters and corresponding multiscale flow. Further details regarding these processes are provided below.
  • the signals are communication activities between nodes 402 (or zones).
  • the nodes 402 indicate zones or area of certain locations, such as exclusive economic zones, ports, etc. These are time series for each pair of nodes 402, which measure the amount of collective communication activities between those zones (all the vessels within each zone).
  • the signals are vessel flows between nodes, indicating the quantity of vessels flowing from one zone to another over a period of time.
  • G t denotes vessel graph
  • G 2 denotes communication graph
  • t denotes time
  • denotes reaction time delay
  • J 7 denotes vessel density
  • a denotes diffusion constant
  • denotes coefficient of weighting communication information
  • This semi-discrete (continuous in time and discrete in space) partial differential equation describes that the change of vessel density (left-hand side) depends on 1) diffusion of the vessels with the graph Laplacian operator with the vessel graph G t and 2) advection of vessels with the graph gradient operator coming from the communication graph G 2 with a small reaction time delay ⁇ .
  • This model generates data in a way that the vessel flows between certain zones depend on the communication activities.
  • a goal of this multilayer information framework is to discover the
  • dependencies i.e. identification of the vessel flows that depend on certain communication activities.
  • FIG. 5 provides a schematic illustration of a mixed, coarse-scale multilayer network.
  • the original scale 500 of multilayer network at the left is processed by a novel flow- clustering algorithm to maximize observed flow rate (middle) and to generate clusters 502 (rectangular boxes) which in turns enables the cross-layer dependency computation among the flow dependency of clustered entities (simplified edges at the right).
  • Region of interest 504 are identified by the across-layer dependency links 506 (directed edges). As shown on the right of FIG. 5, all the zones that are adjacent of the across-layer dependency links 506 are identified as regions of interest 504. In this simplified illustration, the only zone that is not region of interest is the right bottom zone 508.
  • FIG. 6 provides a schematic illustration of zoom-in and zoom-out capability.
  • the coarse-scale, multilayer network 504 further enables zoom-in to a designated node, such as the selected square node 600 which allows for a zoomed-in selected node 602.
  • the zoomed-in selected node 602 in this example encompasses 7 nodes.
  • the flow clustering process 604 can be continued to provide another level of finer-grained clusters 606.
  • Flow ⁇ 3 is the collection of all links from nodes in cluster to nodes in cluster n d ⁇ s
  • the flow rate of flow ⁇ 3 is defined as:
  • the flow clustering problem is posed as finding k clusters that maximizes the sum of the flow rate in £ largest intra-cluster or inter-cluster flows.
  • the numbers of clusters k and flows £ are pre-defined.
  • the flow rate maximization problem is optimization problem is as follows:
  • the solution to this can be approximated with kernel k-mean clustering (see Literature Reference No. 8), because both aim to maximize the weighted sum of the graph adjacency matrix entries.
  • the kernel k-mean clustering is equivalent to the symmetric nonnegative matrix factorization (NMF) (see Literature Reference No. 3) and can be efficiently solved by coordinate descent methods (see Literature
  • the symmetric NMF aims to find an N X k matrix H (where k ⁇ N) with nonnegative entries H i; - > 0 that minimizes
  • the flows from region R t to region Rj are denoted as: V R . ⁇ R . (t) and C R . ⁇ R .(t) for vessels and communication, respectively.
  • the present method is directed to capturing the dependency of these flows (edges) and their changes across different types of flows.
  • Sensor data e.g., from a plane, satellite, etc.
  • the time series will be density of vessels and communications in each region for the layers: V R . ⁇ R .(t) and C R . ⁇ R . (t), respectively.
  • FIG. 7 provides a schematic illustration of the discovery of inter4ayer dependency relations: the communication flow between node 1 and node 12 in the upper panel 700 influences the vessel flows on the path of node 1 ⁇ 4 ⁇ 8 ⁇ 12 in the bottom panel 702.
  • Such flow dependency (edges) between layers are inferred automatically by the ATE methods.
  • the flow clustering process 300 receives inputs as a vessel flow tensor and, based on that, generates cluster membership. The process is provided below and further depicted in FIG. 3 :
  • V and k An N X N X T vessel flow tensor V where each entry i;t indicates the amount of vessels flowing from node i to node j at time t. The number of clusters k.
  • Output, d An N x 1 vector d that indicates the cluster membership with entries from ⁇ 1,2, k ⁇ .
  • the vessel flow clustering process was performed with a set of data to validate the system and process.
  • Provided below is an example to illustrate that flow clustering summarizes vessel flow and reduces the number of flows.
  • the example graph in FIG. 8 is a l0 x l0 regular grid (therefore 100 nodes) 800 with three major communications from node 3 to node 77, node 35 to node 77, and node 59 to node 77.
  • the communication frequency of these are 5%, i.e., if the sampling rate is per minute, in average 5 times out of 100 minutes are active.
  • There is also noise communication with a 2% frequency with a pair of nodes randomly picked at each time.
  • the vessel flow is simulated with the partial differential equation described above with random initialization for the vessel density on each node.
  • the grid 800 shows the corresponding vessel flows for the major communications.
  • FIG. 8 also depicts the vessel flow clustering results 802 with 10 clusters, where each cluster is color-coded (nodes with the same color is a cluster).
  • Vessel flow summarization 804 is also depicted, showing the summarized version of the vessel flows in the grid 800.
  • the vessel flow summarization 804 is indicated by the directed edges, showing that the number of flows is reduced.
  • the system described herein detects communication and vessel flow dependency with low resolution and cue regions of interest with TE for multiscale monitoring (depicted as element 302 in FIG. 3).
  • the process is provided in further detail below:
  • Inputs. V, C and k An N X N X T vessel flow tensor V where entry i;t indicates the amount of vessels flowing from node i to node j at time t.
  • An N X N X T communication tensor C where entry C i;t indicates the amount of communication from node i to node j at time t.
  • the number of clusters is k.
  • decision making tools e.g. situation awareness tool in monitoring vessel movements in/out of contested water
  • exploratory analysis e.g. drill down to high-flow entropy zones based on dependent clusters
  • refine units of analysis for tracking purpose e.g., use corresponding multiscale flow and corresponding dependency links.
  • FIGs. 9A through 9C depict an example of the multi-scale multilayer information dynamics framework.
  • FIG. 9A is a snapshot of the vessel flow 900 with a full resolution 10 x 10 grid (100 nodes), where the thickness of the link represents the amount of vessels flowing from one node to another.
  • FIG. 9B is a snapshot of the summarized vessel flow 902 on the flow cluster graph, where 100 nodes are reduced to 10 clusters, providing a low-resolution flow (after clustering). As shown in FIG.
  • a multi-scale vessel flow graph 904 is generated.
  • regions of interest can be cued for multiscale vessel flow monitoring.
  • the system can zoom-into regions of interest while maintaining sufficient monitoring of low interest regions.
  • FIG. 9C provides a snapshot of the multiscale vessel flow 904 where the dependent clusters (dependent on communications) have the original resolution and the rest have the low resolution.
  • a processor 104 may be used to control a device
  • a mobile device display e.g., a mobile device display, a virtual reality display, an augmented reality display, a computer monitor, a motor, a machine, a drone, a camera, etc.
  • a drone or other autonomous vehicle may be controlled to move to an area within the multi-scale vessel flow graph based on identified dependent flows/clusters or their changes over time.
  • the system can generate the multi-scale vessel graph by applying the algorithm on data collected via satellites, determine regions of interest with thresholds (e.g., a significant deviation/changes in flow-dependency within a priori-determined time window), and send drones to the regions of interest to collect finer-grained data, or perform monitoring and tracking with desired level of coverage (e.g., zone size) for given constraints (e.g., # of drones available, processing powers, etc.).
  • a camera may be controlled to orient towards the region of interest and zoom in as needed. In other words, actuators or motors are activated to cause the camera (or sensor) to move or zoom in on the region of interest.
  • the system can be connected with or otherwise incorporated into a satellite as one- level of monitoring (more holistic, passive), such that upon identifying a region of interest, the surveillance apparatus (cameras, sensors, etc.) can be caused to focus on or otherwise zoom into the region of interest.
  • the surveillance apparatus cameras, sensors, etc.
  • drones as more active and provide fine-grained monitoring (especially under occluded conditions per adversarial intent and actions).
  • a drone or unmanned aerial vehicle can be caused to drive or otherwise move to the region of interest for further
  • any recitation of "means for” is intended to evoke a means-plus-function reading of an element and a claim, whereas, any elements that do not specifically use the recitation "means for”, are not intended to be read as means-plus-function elements, even if the claim otherwise includes the word "means”.
  • any recitation of "means for” is intended to evoke a means-plus-function reading of an element and a claim, whereas, any elements that do not specifically use the recitation "means for”, are not intended to be read as means-plus-function elements, even if the claim otherwise includes the word "means”.
  • particular method steps have been recited in a particular order, the method steps may occur in any desired order and fall within the scope of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Discrete Mathematics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Complex Calculations (AREA)
  • Traffic Control Systems (AREA)
EP18855282.2A 2017-09-12 2018-07-11 Grobkörnige, mehrschichtige strömungsinformationsdynamik für mehrskalige überwachung Pending EP3682304A4 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762557733P 2017-09-12 2017-09-12
PCT/US2018/041714 WO2019055112A1 (en) 2017-09-12 2018-07-11 INFORMATION DYNAMIC OF LARGE GRAIN MULTILAYER FLOW FOR MULTIPLE SCALE MONITORING

Publications (2)

Publication Number Publication Date
EP3682304A1 true EP3682304A1 (de) 2020-07-22
EP3682304A4 EP3682304A4 (de) 2021-07-14

Family

ID=65723837

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18855282.2A Pending EP3682304A4 (de) 2017-09-12 2018-07-11 Grobkörnige, mehrschichtige strömungsinformationsdynamik für mehrskalige überwachung

Country Status (3)

Country Link
EP (1) EP3682304A4 (de)
CN (1) CN111033411A (de)
WO (1) WO2019055112A1 (de)

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150235143A1 (en) * 2003-12-30 2015-08-20 Kantrack Llc Transfer Learning For Predictive Model Development
US8107677B2 (en) * 2008-02-20 2012-01-31 International Business Machines Corporation Measuring a cohort'S velocity, acceleration and direction using digital video
US9049464B2 (en) * 2011-06-07 2015-06-02 Qualcomm Incorporated Multiple description coding with plural combined diversity
US9113590B2 (en) * 2012-08-06 2015-08-25 Superior Edge, Inc. Methods, apparatus, and systems for determining in-season crop status in an agricultural crop and alerting users
US8860602B2 (en) * 2012-10-09 2014-10-14 Accipiter Radar Technologies Inc. Device and method for cognitive radar information network
US9135826B2 (en) * 2012-12-26 2015-09-15 Sap Se Complex event processing for moving objects
US9380275B2 (en) * 2013-01-30 2016-06-28 Insitu, Inc. Augmented video system providing enhanced situational awareness
US20150301208A1 (en) * 2014-04-22 2015-10-22 Westerngeco L.L.C. Seismic data processing
US20150379408A1 (en) * 2014-06-30 2015-12-31 Microsoft Corporation Using Sensor Information for Inferring and Forecasting Large-Scale Phenomena
US9963215B2 (en) * 2014-12-15 2018-05-08 Leidos, Inc. System and method for fusion of sensor data to support autonomous maritime vessels
KR101703906B1 (ko) * 2015-01-22 2017-02-08 한국전자통신연구원 선박 관제 시스템 및 그것의 선박 관제 방법
US20170178524A1 (en) * 2015-05-06 2017-06-22 Ocula Corporation Swim Lap Counting and Timing System and Methods for Event Detection from Noisy Source Data
US9727976B1 (en) * 2015-09-08 2017-08-08 Sandia Corporation Geospatial-temporal semantic graph representations of trajectories from remote sensing and geolocation data
US10002471B2 (en) * 2015-09-30 2018-06-19 Ants Technology (Hk) Limited Systems and methods for autonomous vehicle navigation
ES2613310B1 (es) * 2015-10-23 2018-03-02 Consejo Superior De Investigaciones Científicas (Csic) Vehículo aéreo no tripulado biomimético y zoosemiótico dirigido por piloto automático para vuelos de precisión y/o persecución

Also Published As

Publication number Publication date
WO2019055112A1 (en) 2019-03-21
EP3682304A4 (de) 2021-07-14
CN111033411A (zh) 2020-04-17

Similar Documents

Publication Publication Date Title
Wang et al. Earthquake prediction based on spatio-temporal data mining: an LSTM network approach
Wang et al. A spatial-adaptive sampling procedure for online monitoring of big data streams
US11954317B2 (en) Systems and method for a customizable layered map for visualizing and analyzing geospatial data
US20180332256A1 (en) Coarse-grained multilayer flow information dynamics for multiscale monitoring
US11386507B2 (en) Tensor-based predictions from analysis of time-varying graphs
US10528600B1 (en) System to identify unknown communication behavior relationships from time series
US10089549B1 (en) Valley search method for estimating ego-motion of a camera from videos
Armstrong et al. The Internet of Things and fast data streams: prospects for geospatial data science in emerging information ecosystems
Butenuth et al. Infrastructure assessment for disaster management using multi-sensor and multi-temporal remote sensing imagery
Leykam et al. Topological data analysis and machine learning
Shi et al. Parallelizing maximum likelihood classification on computer cluster and graphics processing unit for supervised image classification
Xing et al. A land use/land cover change geospatial cyberinfrastructure to integrate big data and temporal topology
Muyskens et al. Star–galaxy image separation with computationally efficient gaussian process classification
Antunes et al. Proof of concept of a novel cloud computing approach for object-based remote sensing data analysis and classification
Bhuiyan et al. Hajj pilgrimage video analytics using CNN
Ciancarelli et al. A gan approach for anomaly detection in spacecraft telemetries
Thomopoulos et al. Automated real-time risk assessment for airport passengers using a deep learning architecture
US11227162B1 (en) Multilayer information dynamics for activity and behavior detection
Vrins et al. On the entropy minimization of a linear mixture of variables for source separation
WO2019055112A1 (en) INFORMATION DYNAMIC OF LARGE GRAIN MULTILAYER FLOW FOR MULTIPLE SCALE MONITORING
Lim et al. Detection of salient regions in crowded scenes
Shulepov et al. Approach to Anomaly Detection in Cyber-Physical Object Behavior
ALDabbas et al. Complex event processing based analysis of cassini–huygens interplanetary dataset
Feder Drones Move From" Nice To Have" to Strategic Resources for Projects
Singh et al. Real‐Time Implementation of Change Detection for Automated Video Surveillance System

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200311

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20210611

RIC1 Information provided on ipc code assigned before grant

Ipc: G05B 23/02 20060101AFI20210607BHEP

Ipc: G06N 20/00 20190101ALI20210607BHEP

Ipc: G05B 15/02 20060101ALI20210607BHEP

Ipc: G05B 19/042 20060101ALI20210607BHEP

Ipc: G06K 9/46 20060101ALI20210607BHEP

Ipc: G06K 9/62 20060101ALI20210607BHEP

Ipc: G08B 25/00 20060101ALI20210607BHEP