Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/24—Querying
  • G06F16/245—Query processing
  • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
  • G06F16/2471—Distributed queries
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/50—Network services
  • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
  • H04L67/63—Routing a service request depending on the request content or context
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L67/00—Network arrangements or protocols for supporting network services or applications
  • H04L67/50—Network services
  • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Definitions

• IP Internet Protocol
• Other devices on the network would then be able to access the data provided by the data sources, either individually or in aggregate depending on the application.
• IP Internet Protocol
• wireless networks of data sources define their topologies dynamically as they are deployed, and continuously redefine their links and routing schemes to account for new and failing nodes and optimal power management. Rudimentary forms of networks of data sources are already being used in some industrial process control systems, and future applications for networks of data sources are widely predicted in many domains.
• Prototype networks of data sources use protocols from the computer network world, based on TCP and UDP, which have been developed for communication rather than data management.
• Industrial systems use different but similar protocols such as Modbus. These protocols offer mechanisms to deliver messages in a point-to-point manner or to broadcast messages to all members of a network.
• Modbus These protocols offer mechanisms to deliver messages in a point-to-point manner or to broadcast messages to all members of a network.
• these protocols often can be inadequate and have limited value in providing efficient data management in the network of data sources world. Accordingly, these types of systems developed for communication are problematic in that more complicated, extensive computer programs and systems used for data management in networks of data sources must often be created from scratch by programmers and system managers in order to implement more sophisticated data management techniques that have been available in IT technologies but by-and-large have not been available for networks of data sources.
• Boasson One approach to limited data management, such as described for the field of industrial process control in U.S. Patent No. 5,301,339 issued to Boasson, is to provide a system for communication among subsystems in which requests are fulfilled based on the type of the data.
• data of a certain type may be requested, there is no mechanism for constraining the desired values of the requested data.
• Such a mechanism is critical to implement traditional database queries such as, for example, asking for all temperature sensors reading temperatures higher than 100 degrees F.
• the system described by Boasson lacks any database view of the data to the data processing subsystems, any relational or object schema describing the data, and any facility to request data in traditional query language.
• the system of Boasson provides only limited data management capabilities far below the capabilities of traditional databases.
• Another distributed database approach such as described in U.S. Patent No. 4,635,189 issued to Kendall and in U.S. Patent No. 5,179,660 issued to Devany et al.. provides mechanisms for dividing traditional database queries among distributed databases to which the data has not been carefully fragmented ahead of time.
• these types of systems are not suitable for networks of data sources for several reasons.
• the present invention allows traditional information technology data management techniques to be applied directly within networks of data sources. More specifically, the present invention allows a program, running on a device logically connected to a network which also logically connects the networked data sources, to issue a traditional database query onto the network and to receive back from the network the result of that query as it applies to the data produced by those data sources.
• the invention provides a method for information management of a distributed data sources network database that includes multiple nodes.
• the nodes include a querying node and multiple data sources.
• the method includes steps of providing a schema for the distributed data sources network database, entering a query in a database language at the querying node in the network, decomposing the query into at least one network message, and transmitting the network message only to data sources relevant to the query.
• the method further includes steps of receiving the network message at the data sources relevant to the query, sending a reply message to the network message when the query is met, and providing a query result in the database language at the querying node from the reply message.
• the present invention provides a system for information management of a distributed data sources network database.
• the system includes a network, multiple data sources coupled to the network, and at least one querying node coupled to the network.
• the data sources are capable of providing information according to a schema for the distributed data sources network database.
• the querying node is capable of receiving a query in a database language and decomposing the query into at least one network message that is transmitted over the network only to data sources relevant to the query.
• the data sources relevant to the query send a reply message over the network in response to the network message when the query is met, and the querying node provides a query result in the database language from the reply message.
• FIG. 1 shows an example of a network architecture in which the present invention may be employed.
• FIG. 2 shows a general functional description of a specific embodiment of the invention.
• FIG. 3 shows a functional description for translating a database query into network messages, in accordance with specific embodiments of the invention.
• FIG. 4 shows a functional description of a network interface processing network messages received, in accordance with a specific embodiment of the invention.
• FIGs. 5A-5C show functional descriptions for collecting network messages and interpreting query results, in accordance with specific embodiments of the invention.
• the present invention includes a system that describes a network of data sources in terms of a traditional database schema, converts traditional database queries into network messages, and routes those messages to those data sources which have relevant data.
• the network interface of the data source accepts the message, filters the data source's output according to the instructions in the message, and then sends reply messages to the originator of the query. The system then collects these reply messages at the query originator and produces query results as a traditional database.
• the present invention provides a system and methods which allow a network of data sources to be managed by multiple distributed clients as if the network of data sources were a single database. More specifically, the invention allows a program running on a networked device to issue a database query to its network interface, and for the network infrastructure to calculate the results of the query and return these results to the querying device. Specific embodiments of the present invention will be useful for information management in many different applications. A specific application is in industrial automation such as factory equipment control and management, as described below for exemplary purposes. However, other specific embodiments will be useful for logistics management such as for package delivery services, crisis management for toxin tracking or fire tracking, highway traffic management, security management for smart building or smart battlefield applications, and many other applications.
• some of the advantages offered by this invention include: allowing users or programs to access and process data from networked data sources according to well known information technology standards, such as Structured Query Language (SQL); allowing multiple users and programs to access and process data source data from any point in the network; significantly reducing network traffic compared to polling or continuous-refresh systems; having no central point of failure for data access; having minimal latency, as data always travels the direct path from the data source to the requesting node; and not being necessary for the querying node to know the physical locations of the responding data sources.
• SQL Structured Query Language
• FIG. 1 shows an example of a network architecture in which the present invention may be employed.
• an internetwork 10 that is. a collection of networks connected bv network routers 15. which mav be interconnected to each other.
• the term router is used herein in a general sense and may include traditional routers, network switches, gateways, bridges, or controllers bridging separate networks.
• the present invention may also be used on a single network, but its value is higher on an internetwork.
• Each network may connect an arbitrary number of nodes.
• the lines 35 connecting various nodes and routers in this network architecture are wired connections, in this specific embodiment.
• Another architecture on which the present invention may be employed is a wireless network. This differs from the network described above for FIG.
• lines 35 can be viewed as logical connections for a wireless internetwork. Further, in embodiments where the internetwork includes a combination of wireless and wired networks, lines 35 are logical connections and wired connections respectively.
• the internetwork includes a combination of wireless and wired networks
• lines 35 are logical connections and wired connections respectively.
• each node on the network has a network interface and queries may originate from any data consumer node in the network.
• a network node may be a data producer 20, a data consumer 25, or both (a data producer/consumer 30).
• data producers 20 include a data source (for example, a sensor) or a data source bank (often called a distributed I/O).
• data consumers 25 include controllers and monitoring systems.
• An example of a node that is a data producer/consumer 30 is a user operator panel.
• a controller acting as the interface to the network for one or more data sources is considered a single node which is another example of a data producer/consumer 30.
• any node in the network can be equipped with the functionality of a data consumer node, a data producer node, or a data producer/consumer node by embedding the appropriate software, in accordance with the present invention, in the node.
• each node (including each data source) on the network has a network interface, appropriate for the type of network protocol used on the network to which that node is logically connected.
• This network interface includes the relevant traditional network interface protocols, of which examples include but are not limited to: Ethernet, IP, TCP, UDP, Profibus, DeviceNet, and wireless protocols such as IEEE 802.1 1, Ricochet, GSM, CDMA, etc.
• each nodes' network interface includes software, in addition to the typical network protocol interface, that provides the functionalities of the present invention, as will be described in more detail below.
• this additional software preferably is embedded (stored in memory such as ROM) in the network interface of the node (e.g., data consumer node, data producer node, or data producer/consumer node) or in the application resident on the node (e.g., data consumer node, or data producer/consumer node).
• the present invention provides for a description of the network of data sources in terms of a traditional database schema.
• the nodes on a network view the data sources (e.g., data producer 20 or data producer/consumer 30) on the network as a "database.”
• a schema is understood to mean tables, each of which has columns (each column corresponding to attributes of relations) and rows (each row corresponding to a relation grouping called a tuple).
• a schema is traditionally understood to mean a set of object classes, which may form a hierarchy from general to specific. Either a relational or object-oriented philosophy with a schema may be followed within the framework of the present invention.
• a table is made in the schema for each data source type.
• the attributes of this table include (1) each of the output types which the data source can provide, (2) attributes for descriptive information about the data source (e.g. the ID of a component which it is connected to, ID of the subsystem to which it belongs, etc.) and (3) an ID. This last ID is unique within the table for each data source in the listed in the table. If some data source types are similar but slightly different, they may be merged into a single table with extra attributes to distinguish between the types.
• an object class is defined for each data source type.
• data source types are similar but slightly different, they may be represented as subclasses of a common, more general class.
• Methods are included within each class to allow retrieval of the data source data, such as one method for each output type of the data source. Additional methods are included to retrieve descriptive information about the data source (e.g. the ID of a component which it is connected to, ID of the subsystem to which it belongs, etc.). Additional methods may be included to access special functions of the data source, such as reset, calibration, setting the dynamic range, etc.
• the present invention may view the network of data sources with a schema from either a relational or object-oriented philosophy.
• a relational database philosophy For clarity in understanding the invention, the following description will be describe the invention from a relational database philosophy, in accordance with a specific embodiment. It is understood that other specific embodiments of the present invention from an object- oriented philosophy are also within the scope of the present invention. Further, some databases have a schema using a combination of relational and object-oriented philosophies, and these types of databases also are within the scope of the invention.
• the database schema need not be explicitly stored at any node.
• Each querying node need only know the table and attribute names of the data that it requires, and not the entire schema.
• the schema of the database is implicit from the behavior of the system, as is further described below.
• Any data consumer node 25 or 30 in the network may issue a traditional database query in a step 100.
• queries may specify a "refresh rate" which indicates that the query is to persist and be continually evaluated against the current network status at a given rate.
• that query is decomposed into the relevant parts for each data source type by the network interface of the querying node into network messages. Each network message is then routed over the network only to the data sources of the appropriate type by the routing system, in a step 104.
• the network routers 15 may also route the network messages based on constraints from the query in addition to based on data source types.
• the network messages are received by each of the appropriate data sources' network interfaces.
• Each data source ' s network interface checks the constraints of the query periodically according to the refresh rate of the query, as indicated by a feedback line 107.
• the data source's network interface replies to the query, in a step 108, and the reply is routed back to the querying node.
• the network interface of the querying node collects the replies and continually checks them for query satisfaction. Each time the query is satisfied, the network interface passes the relevant data to the querying program or user in a step 112.
• the present invention provides a system to convert traditional database queries into network messages that are appropriate for a network of data sources in which each data source is viewed as a database record (relational model) or object instance (object-oriented model) or some combination thereof, and in which the schema described above is used.
• This system extracts the relevant parts of the query for each data source, so that it may be sent to the data source.
• each data consuming node 25 or 30 includes either in its network interface or in the application program resident on that data consuming node the necessary software/firmware that converts traditional database queries into network messages containing the relevant parts of the query to be sent to the appropriate data producing node 20 or 30.
• This network messaging software includes the functionality of extracting the relevant parts of the query and then including these parts into a message encapsulated in the data payload of a typical network protocol packet (e.g., within an Ethernet packet payload, etc.) transmitted over the network.
• Queries may specify a "refresh rate" which indicates that the query should be continuous and should be updated at the rate given. Note that even if a refresh rate is given, queries are only answered when the query constraints are satisfied, as is described in detail later.
• the relevant parts of the query for each data source are: (1) a list of constraints, possibly empty, based on which the data source should decide to send information, (2) a list of return values which the data source should return if the constraints are satisfied, (3) optionally, a refresh rate at which the data source should reconsider sending the information, (4) a unique message ID, and (5) the address of the querying node.
• the address of the querying node may be omitted if it is automatically provided as part of the underlying network service.
• the exact structure (e.g., order and/or size of the fields containing the above relevant parts of the query) of the network message is not crucial to the invention.
• the network message may be sent using one network protocol packet, or the network message may be broken into segments that are sent using multiple network protocol packets.
• FIG. 3 describes a system for decomposing SQL queries into the network messages described above, in accordance with a specific embodiment.
• the present invention is not limited to SQL as the query language.
• SQL is practically the standard query language for relational databases and is also being used increasingly with object- oriented databases.
• this description of decomposing traditional database queries into network messages in accordance to a specific embodiment of the invention is described in the context of relational database approach, but should not be so limited.
• constraint predicates is a significant portion of most query languages, and extracting the predicates based on relational tables referenced (or referenced classes, in an object-oriented case) can be performed for these other query languages in accordance with the present invention.
• Most other query languages also allow OR expressions or subqueries, and they are handled similarly as described below for SQL.
• the system for converting the traditional database query into network messages that are sent by the querying node over the network begins by creating the necessary messages.
• a step 150 one message is created for each table which is referenced within each operand of an OR expression, in the WHERE clause of the query or any subquery expression.
• each predicate that refers to a column of the table is included in the message as a constraint, in a step 152.
• a message is created for each table which is referenced outside an OR expression, but which is within the WHERE clause of a subquery expression, in a step 154. All references to columns of that table which are within the WHERE clause of the subquery, but not already included in other messages, are then included in the new message as constraints, in a step 156.
• a message is created for each table that is referenced in the WHERE clause outside any OR expression and outside a subquery expression, in a step 158. All predicates that reference columns of this table, but have not yet been included in other messages, are then included as constraints in this message, in a step 160.
• this constraint is identified as either "local” to one data source or “distributed” over many data sources. This is achieved by counting the number of tables which are referenced in the constraint. If it is 1, then the constraint is "local.” If it is 2 or more, than the constraint is "distributed.”
• a step 164 the system collects all of the columns in the SELECT expression which reference the table for which the message has been created, and adds to this list each column that references this table and occurs in a "distributed" constraint of the message. This list is added to the message as the "return values" for the message. The “distributed” constraints are then removed from the message's constraint list.
• the refresh rate is included in the message, in a step 166.
• the system includes a unique message ID and the network address of the local querying node to the message, in a step 168.
• the system then sends each message over the network in a step 170.
• predicate 1 where predicate 1 could be "Temp > 100 degrees", would be sent translated into a network message with predicate 1 with a unique message ID and the network address of the querying node.
• WHERE (predicate 1 AND predicate2) where predicate2 could be "Pressure > 100 psi", would be sent translated into a network message with predicate 1 and another network message with predicate2, with both network messages having the same message ID and the network address of the querying node.
• predicate 1 OR predicate3 where predicate3 could be "Volume ⁇ 250 cubic cm" would be sent translated into a network message with predicate 1 and another network message with predicate3, with both network messages having the same message ID and the network address of the querying node.
• Characteristic routing is a routing protocol that allows data to be transported multi-hop through a network from a sender node to a set of nodes using an arbitrary mix of "characteristics" (characteristics are descriptions of the nodes in the form of multiple arbitrary identifying descriptive names). Nodes can have multiple, dynamic characteristics. Characteristic routing is optimized to make routing using multiple characteristics fast and efficient by creating a routing table index using bit vectors and compression techniques.
• characteristic routing as the index for the networked objects (e.g., data sources) being queried provides an efficient indexing means for fast and efficient information retrieval from a distributed network of data sources database.
• characteristic routing eliminates the need to individually contact data sources or to create predefined multicast groups in order to query the data sources relevant to a query.
• Characteristic routing which provides a bit vector representing the particular characteristics for each node, where each bit location in the bit vector represents the existence of a particular characteristic out of a set of total characteristics for a node. Network messages sent using characteristic routing can be directed to data sources that have the information requested in the query.
• IP- multicast Internet Protocol multicast
• Routers 15 in the network will be equipped appropriately to perform the particular type-based message routing that might be utilized in specific embodiments. Accordingly, the network messages are routed only to those data producers 20 or 30 which meet the defined type relevant to the query.
• each data producing node 20 or 30 includes either in its network interface or in the application program resident on that node the necessary software/firmware that processes received network messages and transmits response messages back to the appropriate querying node when the query constraints are met.
• the response messages are encapsulated in the data payload of a typical network protocol packet (e.g., within an Ethernet packet payload, etc.) that is transmitted by the data producing node 20 or 30 over the network.
• a typical network protocol packet e.g., within an Ethernet packet payload, etc.
• the data source's network interface adds the message to its list of outstanding queries (for example in a buffer).
• the data source's network interface characterizes each constraint in the message as either "static” or “dynamic,” in a step 202. This characterization is achieved by considering all of the column references in the constraint. If all of the column references are for "descriptive" attributes, then the constraint is considered “static.” Otherwise, the constraint is considered “dynamic.”
• the network interface determines in a step 204 if the previously- characterized constraint is the last constraint in that message. If not, then the system returns to step 202 to characterize the next constraint in the message.
• the data source's network interface determines in a step 206 whether all the constraints in the message are "static.” If all the constraints in the message are determined to be "static,” then the network interface in a step 208 performs a one-time comparison of the current readings of the data source to the query constraints. If the current readings meet the query constraints, as determined in a step 210, then the network interface issues back to the querying node a reply message that includes: the current values of the return values specified in the processed network message, an indication that the constraints were "static,” and the unique message ID of the processed network message.
• the reply message which includes the address of the replying node and is addressed to the querying node, gets transmitted by the data source's network interface over the network for routing back to the querying node.
• step 206 determines in a step 214 whether a refresh rate was specified in the network message. If a refresh rate is not specified, then the network interface proceeds to step 208 and performs a onetime comparison of the data source's current readings to the query constraints. The system then executes the remaining steps 210 and 212, in a similar manner as already described above.
• the network interface compares the current data source readings to the query constraints in a step 216. If the constraints are determined in a step 218 not to be met, then the network interface returns (as indicated by line 220) at the specified refresh rate to step 216 to compare the current data source readings to the query constraints. If the constraints are determined in step 218 to be met, then the network interface in a step 222 issues back to the querying node a reply message that includes: the current values of the return values specified in the processed network message, and the unique message ID of the processed network message.
• the reply message which is addressed to the querying node, gets transmitted by the data source's network interface over the network for routing back to the querying node. If the value of a predetermined lifetime parameter that optionally may be specified in the network message has been exceeded, as determined in a step 224, then the network interface ends its processing of the message. However, if this value has been determined in step 224 not to be exceeded, then the network interface returns to step 216 to make another comparison at the specified refresh rate. The system then continues on from step 216, as already described above.
• FIGs. 5A-5C describe the functionality of the present invention for collecting reply messages and producing the query results at the querying node.
• the system which can be software running in a client application resident on the querying node or embedded in memory in the network interface of the querying node, has three logical threads, which may be implemented as actual separate threads or as a single monolithic process.
• the first thread is responsible for collecting the reply messages from the network.
• each reply message is received from the network in a step 300.
• Each reply message is then placed into the appropriate buffer in a step 302.
• Separate buffers, indexed by the different message IDs, are maintained for each of the original network messages (each having its own message ID) sent by the querying node.
• the reply is added to the relevant buffer.
• a timestamp is added to the reply message to indicate the time at which it was received. Note that multiple queries may originate from the same node, so this thread accepts reply messages relevant to different queries from this node.
• the purpose of the second thread shown in FIG. 5B is to enforce the timing constraints of the system.
• This thread includes a timing interval, called ReplyLifetime, after which any reply message is to be removed from its buffer.
• the value of the ReplyLifetime is to be determined on a case-by-case basis, but a reasonable default value can be three times the refresh rate of the relevant query.
• This thread continuously checks though all of the buffers looking for any reply message which was received at a time greater than ReplyLifetime units ago. If any such message is found, it is deleted from the buffer, unless it is marked as "static" in which case it is unchanged. In particular, for each buffer, each reply message is scanned in a step 330. A determination whether a reply message is marked "static" is made in step 332.
• reply message is not "static,” then the system continues on to scan the next reply message in step 330. If the reply message is marked “static,” then a determination is made whether the message timestamp is older than the ReplyLifetime value in a step 334. If not, then the system continues to scan the next reply message in the buffer in step 330. However, if the timestamp message is older than the ReplyLifetime value, then that reply message is removed from that buffer. Accordingly, those reply messages that are marked "static" and are older than some predetermined time that exceeds a desired threshold are deleted.
• queries for which a refresh rate has been specified continue until terminated.
• Any node may terminate a query by sending a special "terminate_query" message to each data source meeting the data source- type referenced in the query.
• This termination message includes the message IDs of the network messages to be terminated.
• the data source's network interface then removes the network message from its list of outstanding queries.
• a lifetime may be assigned to a query, so that the network interface of each data source will automatically delete any network messages after their lifetime has elapsed.
• the querying node also erases any reply messages relating to that query which are still in its buffers.
• the reply messages to be erased would also include any reply messages marked as "static.”
• Databases provide the ability to cross-reference information (i.e., perform "joins").
• the embodiments described above primarily discuss the processing, including joining, of returned information sent in the various reply messages at the querying node.
• the processing of returned information can be performed as the information returns through the network through progressive joins, or the task of performing joins can be delegated to specific nodes, called designated joiners, which lie along the upstream path from the queried data producing nodes to the querying node and which possess greater computational ability or memory than the data producing nodes.
• Designated join nodes perform computations in a distributed manner on returning information in reply messages as they arrive at the designated join nodes.
• designated join nodes can perform averages, etc.
• the processing reply messages at the querying node will be reduced or simplified, as much of the processing will have occurred such that the returned information sent to the querying node has been previously processed.
• Designated join nodes are particularly useful when the system uses a mix of lower-cost, lower-functionality data sources (i.e., data producing nodes) and higher-cost, higher-functionality active components (e.g., data producing/consuming nodes, or data producing nodes with higher processing power and/or more memory)- This mix of devices can occur, for example, when upgrading an existing network of data sources or when overall system cost savings are desired.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computational Linguistics (AREA)
• Databases & Information Systems (AREA)
• Mathematical Physics (AREA)
• Probability & Statistics with Applications (AREA)
• Software Systems (AREA)
• Signal Processing (AREA)
• Data Mining & Analysis (AREA)
• Fuzzy Systems (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Computer Networks & Wireless Communication (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
• Computer And Data Communications (AREA)
• Data Exchanges In Wide-Area Networks (AREA)

Applications Claiming Priority (5)

Publications (1)

Family

ID=26864110

Family Applications (1)

Country Status (6)

Families Citing this family (3)

Family Cites Families (4)

• 2000
  • 2000-11-30 IL IL14991500A patent/IL149915A0/xx unknown
  • 2000-11-30 AU AU18073/01A patent/AU781476B2/en not_active Ceased
  • 2000-11-30 CA CA002392826A patent/CA2392826A1/en not_active Abandoned
  • 2000-11-30 EP EP00980866A patent/EP1250653A2/en not_active Withdrawn
  • 2000-11-30 JP JP2001541964A patent/JP2004500631A/ja active Pending
  • 2000-11-30 WO PCT/US2000/032515 patent/WO2001040973A2/en not_active Application Discontinuation

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events