AUSTRALIA
Patents Act 1990
DESCRIPTION
REFERENCES TO RELATED APPLICATIONS
SafeEShare separate innovations patent #2020100425
SafeXshare separateprovisional patent #2021900281
Detpat : separate standard patent#2021202215
DiagML : separate innovations patent #2020901672
Intellex : separate provisional patent #2021900208
DigikeyP : separate provisional patent #2021901862
FunjMakBrake : separate provisional patent #2021901953
FIELD OF THE INVENTION
Embodiments of the present invention are in the field of health, mining, processing and manufacturing and pertain particularly to extracting and using patterns in data from previous experience to operate more efficiently.
BACKGROUND OF THE INVENTION
The statements in the background of the invention are provided to assist with understanding the invention and its applications and uses, and may not constitute prior art.
The costs and suffering associated with systemic failures are likely to be reduced when custodians of a system are able to predict its failure and apply preventative remediation. The systemisation for such prediction is based on condition monitoring and a systemic mechanism for assessing monitored points or items of information against previous experience would be quite similar regardless of the system being monitored, be that system an organism's health, a process of manufacture or the individual items of equipment within such a system where systems containing such items and the effects of failures of them are coupled or sequenced. The costs of unexpected failure can be enormous and while the opportunity for using available data has been growing with use of increased technology, the uncertainties of predictability of failures have also grown because, as the amount of data increases, the effort required to amass and access it in appropriate forms has increased and where that data has been perceived as irrelevant to predictions, the cost collecting it risks being seen as unjustifiable or disproportionate to any possible savings in downtime or loss of production or resources. The determination of exactly how and which items are most directly relevant from within the swathes of tantalizing data available for collection, has extended past the likely scope of normal human comprehension and motivation.
Limitations in the structure and existence of sufficient bodies of data as well as skill overcoming shortcomings in data structure have restricted any ability to predict system failures as fully effective. With increased availability of measurement devices it would be an advancement in the state of the art to provide a system and method for applying Al to this situation.
It is against this background that the present invention is developed.
BRIEF SUMMARY OF THE INVENTION
The present invention relates to methods and systems for organizing monitored equipment and process data for automatic application in appropriate dimensions for discovering patterns in diverse points of data relating to current situations that also appear in bodies of data from previous experiences of failures or previous experience of exposure to similar events in similar (and occasionally apparently dissimilar) systems, processes and equipment items.
Monitored signals for the purpose of failure prevention necessarily include any of images, sounds, vibrations, size, temperature, pressure and calculated quantities eg trends (difference over time), financial ratios, credit levels, frequency profiles.
Outcomes essentially are failures and any number of different statuses of NOT failures. These are associated with occurrences and circumstances pertaining to some measured or unmeasured contribution or disturbance to a system of operation.
Events are also associated measured or unmeasured contribution or disturbance to a system of operation and a monitored system's response to an applied measured event can also be indicative of an imminent failure.
The invention attempts to systemize the application of these techniques.
More specifically, in various embodiments, the present invention is a computer-implemented method for adding to the body of knowledge about failures, system responses and condition states while also using available data to predict and hopefully avert new failures, the several computer-implemented methods executable by hardware processors, the various methods comprising actions including : receiving one or more monitored parameters from an operating systems device; receiving at least one record from the device, the at least one record including at least one measured data item including identification; a data extraction component communicatively coupled with the processor that extracts data associated with the identification, and that performs parsing on the record to extract further items of information relating to that identification; and a data analysis component that performs data detection based on standard measures to generate measurements or classifications related to the identification referred to in the record and potentially also related to other such measures or previous stored similar or other measures for the purpose of understanding trends in the measure or classification or system; utilizing a plurality of deep-learning networks that have been separately trained on each measure and or classification to achieve consistency of data use; generating measurements or classifications as parameters with consistency from the monitored data record; systemizing them for the purpose of comparisons of current and previous situations; utilizing a machine-learning module based on measurements and records including known outcomes related to failures and disturbances and the one or more monitored parameters; and generating an operational status by aggregating the available measurements.
In an embodiment, the standard measure deep-learning networks utilize training data comprising one or more records for one or more systems of operation or existence (health, process, equipment) with records about outcomes of- and or responses to certain events by- the one or more sample operating systems available for assessment as past experience.
In an embodiment, the machine-learning module comprises a random forest algorithm, and the machine-learning module is trained on status assessments of measurements comprising one or more measurements for one or more sample operating systems related to the outcome data.
In an embodiment, records about systems and outcomes for the one or more sample operating systems available for assessment as past experience are processed for assessment of the datapoints including their ranges of values for which the datapoint is allocated a type and thereafter records are classified ands formatted into tolerance subranges based on assumptions about source representation completion, sample rates, periodicity, change inertia and value variation.
In an embodiment, the monitored parameters are selected from operational data associated with an operating system.
In an embodiment, the receiving the one or more measurements from the operating system device comprises receiving measurement input of the monitored parameters through the user device.
In an embodiment, the receiving the one or more monitored parameters from the user device comprises receiving a measurement performed by the user device.
Yet another embodiment of the present invention is a computer-implemented method for adding to the body of knowledge about failures or monitored conditions while also prioritizing further assessments or measurements, executable from a non-transitory computer readable storage medium having program instructions embodied therein, the program instructions executable by a processor to cause the processor to receive one or more user parameters from a user device; receiving at least one data record from the user device, the at least one record including at least one measured data item including identification; a data extraction component communicatively coupled with the processor that extracts data associated with the identification, and that performs parsing on the record to extract further items of information relating to that identification; and a data analysis component that performs data detection based on standard measures to generate measurements related to the identification referred to in the record; utilizing a plurality of deep learning networks that have been separately trained on each measure to achieve consistency of data use; generating measurements as user parameters with consistency from the data; utilizing a diagnostic machine-learning module based on measurements and records including known outcomes and the one or more user parameters; and generating a status by aggregating the available measurements and outcomes.
In various embodiment, a system is described, including a memory that stores computer-executable components; a hardware processor, operably coupled to the memory, and that executes the computer-executable components stored in the memory, wherein the computer-executable components may include a components communicatively coupled with the processor that execute the aforementioned steps.
In another embodiment, the present invention is a non-transitory, computer-readable storage medium storing executable instructions, which when executed by a processor, causes the processor to perform a process for generating measurements, the instructions causing the processor to perform the aforementioned steps.
In another embodiment, the present invention is a system for making measures using a 2D camera, the system comprising a user device having a 2D camera, a processor, a display, a first memory; a server comprising a second memory and a data repository; a telecommunications-link between said user device and said server; and a plurality of computer codes embodied on said first and second memory of said user-device and said server, said plurality of computer codes which when executed causes said server and said user-device to execute a process comprising the aforementioned steps.
In another embodiment, the present invention is a system for making measures (eg status) using processing of data from specific measuring devices, the system comprising a measuring device, a processor, a display, a first memory; a server comprising a second memory and a data repository; a telecommunications-link between said user device and said server; and a plurality of computer codes embodied on said first and second memory of said device and said server, said plurality of computer codes which when executed causes said server and said device to execute a process comprising the aforementioned steps.
In another embodiment, the present invention is a system for making measures (eg ratios) using assessments from one or more results of processing of specific values obtained from specific measuring devices, the system comprising a measuring device, a processor, a display, a first memory; a server comprising a second memory and a data repository; a telecommunications-link between said user device and said server; and a plurality of computer codes embodied on said first and second memory of said device and said server, said plurality of computer codes which when executed causes said server and said device to execute a process comprising the aforementioned steps.
In another embodiment, the present invention is a system for vector-systemizing diverse measures using assessments from one or more results of processing of specific values obtained from specific measuring devices, the system comprising a measuring device, a processor, a display, a first memory; a server comprising a second memory and a data repository; a telecommunications-link between said user device and said server; and a plurality of computer codes embodied on said first and second memory of said device and said server, said plurality of computer codes which when executed causes said server and said device to execute a process comprising the aforementioned steps.
In another embodiment, the present invention is a system for matrix-systemizing diverse measures related to specific outcomes using assessments from one or more results of processing of specific values obtained from specific measuring devices, the system comprising a measuring device, a processor, a display, a first memory; a server comprising a second memory and a data repository; a telecommunications-link between said user device and said server; and a plurality of computer codes embodied on said first and second memory of said device and said server, said plurality of computer codes which when executed causes said server and said device to execute a process comprising the aforementioned steps.
In yet another embodiment, the present invention is a computerized server comprising at least one processor, memory, and a plurality of computer codes embodied on said memory, said plurality of computer codes which when executed causes said processor to execute a process comprising the aforementioned steps.
Other aspects and embodiments of the present invention include the methods, processes, and algorithms comprising the steps described herein, and also include the processes and modes of operation of the systems and servers described herein.
Yet other aspects and embodiments of the present invention will become apparent from the detailed description of the invention when read in conjunction with any attached drawings.
DETAILED DESCRIPTION OF THE INVENTION
Overview
With reference to the figures provided, embodiments of the present invention are now described in detail.
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures, devices, activities, and methods are shown using schematics, use cases, and/or flow diagrams in order to avoid obscuring the invention. Although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to suggested details are within the scope of the present invention. Similarly, although many of the features of the present invention are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the invention is set forth without any loss of generality to, and without imposing limitations upon, the invention.
Multiple Deep Learning Networks
Illustrative Deep Learning Network and Machine Learning Architectures
In one embodiment, the diagnostic determinants are found using a deep learning network (DLN) using training data as described above. In one embodiment, this is performed using a convolutional neural network (CNN) combined with a pyramid scene parsing network (PSPNet) for improved global and local context information. In a PSPNet, the process may utilize "global and local context information" from different regions that are aggregated through a "pyramid pooling module."
In one embodiment, the PSPNet algorithm is implementation as described in Hengshuang Zhao, et al., "Pyramid Scene Parsing Network," CVPR 2017, Dec. 4, 2016, available at arXiv:1612.01105, which is hereby incorporated by reference in its entirety herein as if fully set forth herein. PSPNet is only one illustrative deep learning network algorithm that is within the scope of the present invention, and the present invention is not limited to the use of PSPNet. Other deep learning algorithms are also within the scope of the present invention.
In one embodiment, the diagnostic measures are determined using a random forest algorithm, a specialized machine learning algorithm. Random forests use a multitude of decision tree predictors, such that each decision tree depends on the values of a random subset of the training data, which minimizes the chances of "overfitting". In one embodiment, the random forest algorithm is implementation as described in Leo Breiman, "Random Forests," Machine Learning, 45, -32, 2001, Kluwer Academic Publishers, Netherlands, Available at doi.org/10.1023/A:1010933404324, which is hereby incorporated by reference in its entirety herein as if fully set forth herein. Random forest is only one illustrative machine learning algorithm that is within the scope of the present invention, and the present invention is not limited to the use of random forest. Other machine learning algorithms, including but not limited to, nearest neighbor, decision trees, support vector machines (SVM), Adaboost, Bayesian networks, various neural networks including deep learning networks, evolutionary algorithms, and so forth, are within the scope of the present invention.
The input to the machine learning algorithm are the determining measures obtained from the deep-learning networks. The output of the machine learning algorithm are the predicted values.
As noted, embodiments of devices and systems (and their various components) described herein can employ artificial intelligence (Al) to facilitate automating one or more features described herein. The components can employ various Al-based schemes for interpretation. To provide for or aid in the numerous determinations (e.g., determine, ascertain, infer, calculate, predict, prognose, estimate, derive, forecast, detect, compute) described herein, components described herein can examine the entirety or a subset of the data to which it is granted access and can provide for reasoning about or determine states of the system, environment, etc. from a set of observations as captured via events and/or data. Determinations can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The determinations can be probabilistic-that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Determinations can also refer to techniques employed for composing higher-level events from a set of events and/or data.
Such determinations can result in the construction of new events or actions from a set of observed events and/or stored event data, whether the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources. Components disclosed herein can employ various classification (explicitly trained (e.g., via training data) as well as implicitly trained (e.g., via observing behavior, preferences, historical information, receiving extrinsic information, etc.)) schemes and/or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, etc.) in connection with performing automatic and/or determined action in connection with the claimed subject matter. Thus, classification schemes and/or systems can be used to automatically learn and perform a number of functions, actions, and/or determinations.
A classifier may map an input attribute vector, z=(zl, z2, z3, z4,.. . , zn), to a confidence that the input belongs to a class, as by f(z)=confidence(class). Such classification may employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to determinate an action to be automatically performed.
Training the Deep Learning Networks and Machine Learning Modules
Data Model Embodiments
Some embodiments of the present invention include a computer-implemented method for adding to the body of knowledge about failures, system responses and condition states while also using available data to predict new failures, the several computer-implemented methods executable by hardware processors, the various methods comprising actions including : receiving one or more monitored parameters from an operating systems device; receiving at least one record from the device, the at least one record including at least one measured data item including identification; a data extraction component communicatively coupled with the processor that extracts data associated with the identification, and that performs parsing on the record to extract further items of information relating to that identification; and a data analysis component that performs data detection based on standard measures to generate measurements or classifications related to the identification referred to in the record and potentially also related to other such measures or previous stored similar or other measures for the purpose of understanding trends in the measure or classification or system; utilizing a plurality of deep-learning networks that have been separately trained on each measure and or classification to achieve consistency of data use; generating measurements or classifications as parameters with consistency from the monitored data record; systemizing them for the purpose of comparisons of current and previous situations; utilizing a machine-learning module based on measurements and records including known outcomes related to failures and disturbances and the one or more monitored parameters; and generating an operational status by aggregating the available measurements.
A single value device record would refer to a measure name, units, tolerance type, value occurring for a particular identified situation related to a system and a time with further data clarifying types of systems and other information pertinent to the status of the device with that value at that time.
A context status record includes a set of device measures and or assessments representing a particular system at a point in time or in some cases a particular event or disturbance relative to different points of position at a time relevant to that position. Matching structures contain value changes for trend purposes.
A single value assessment record would be information derived from one or more stages of processing including Al assessment of text, audio, video signals.
Systemisations for processing single value, derived value and more complex inputs including text, audio, video through intermediate stages include structures and processing to enable process according to taxonomised specifications.
Systemisations for making classifications of previous experience data including for assessment into equivalent tolerance bands includes data structures for statistical and advanced mathematical analysis as well as frequency domain translations as well as structures describing attached failure-contributing or other impacting events and the particular outcomes.
Accordingly, other embodiments of the present invention include a computer system, comprising a memory that stores computer-executable components; a processor, operably coupled to the memory, and that executes the computer executable components stored in the memory, wherein the computer-executable components comprise a data collection component communicatively coupled with the processor that receives user parameters from the user or a user device and receives at least one diagnostic record from the user device, the at least one record including suitable data items including identification; a data extraction component communicatively coupled with the processor that extracts data associated with the identification, and that performs parsing on the record to extract further items of information concerning the identification; and a data analysis component that performs data detection based on standard measures and generates the required assessment and or measurements using the record.
Hardware, Software, and Cloud Implementation of the Present Invention
As discussed, the data (e.g., photos, textual descriptions, and the like) described throughout the disclosure can include data that is stored on a database stored or hosted on a cloud computing platform. It is to be understood that although this disclosure includes a detailed description on cloud computing, below, implementation of the teachings recited herein are not limited to a cloud computing environment. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment now known or later developed.
Cloud computing can refer to a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.
Characteristics may include one or more of the following. On-demand self-service: a cloud consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider. Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). Resource pooling: the provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. There is a sense of location independence in that the consumer generally has no control or knowledge over the exact location of the provided resources but can be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be purchased in any quantity at any time. Measured service: cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.
In another embodiment, Service Models may include the one or more of the following.
Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.
Infrastructure as a Service (IaaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).
Deployment Models may include one or more of the following.
Private cloud: the cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.
Community cloud: the cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It can be managed by the organizations or a third party and can exist on-premises or off-premises.
Public cloud: the cloud infrastructure is made available to the general public or a large industry group and is owned by an organization selling cloud services.
Hybrid cloud: the cloud infrastructure is a composition of two or more clouds (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).
A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.
The cloud computing environment may include one or more cloud computing nodes with which local computing devices used by cloud consumers, such as, for example, personal digital assistant (PDA) or cellular telephone, desktop computer, laptop computer, and/or automobile computer system can communicate. Nodes can communicate with one another. They can be group physically or virtually, in one or more networks, such as Private, Community, Public, or Hybrid clouds as described hereinabove, or a combination thereof This allows cloud computing environment to offer infrastructure, platforms and/or software as services for which a cloud consumer does not need to maintain resources on a local computing device. It is understood that the types of computing devices are intended to be exemplary only and that computing nodes and cloud computing environment can communicate with any type of computerized device over any type of network and/or network addressable connection (e.g., using a web browser).
The present invention may be implemented using server-based hardware and software.
For interface with a user, the hardware may include one or more user input devices (e.g., a tablet, a keyboard, a mouse, a scanner, a microphone, a web camera, etc.) and a display (e.g., a Liquid Crystal Display (LCD) panel). For additional storage, the hardware my also include one or more mass storage devices, e.g., a floppy or other removable disk drive, a hard disk drive, a Direct Access Storage Device (DASD), an optical drive (e.g. a Compact Disk (CD) drive, a Digital Versatile Disk (DVD) drive, etc.) and/or a tape drive, among others. Furthermore, the hardware may include an interface one or more external SQL databases, as well as one or more networks (e.g., a local area network (LAN), a wide area network (WAN), a wireless network, and/or the Internet among others) to permit the communication of information with other computers coupled to the networks. It should be appreciated that the hardware typically includes suitable analog and/or digital interfaces to communicate with each other. The hardware operates under the control of an operating system, and executes various computer software applications, components, programs, codes, libraries, objects, modules, etc. indicated collectively by reference numerals to perform the methods, processes, and techniques described above.
The present invention may be implemented in a client server environment. In some embodiments of the present invention, the entire system can be implemented and offered to the end-users and operators over the Internet, in a so called cloud implementation. No local installation of software or hardware would be needed, and the end-users and operators would be allowed access to the systems of the present invention directly over the Internet, using either a web browser or similar software on a client, which client could be a desktop, laptop, mobile device, and so on. This eliminates any need for custom software installation on the client side and increases the flexibility of delivery of the service (software-as-a-service), and increases user satisfaction and ease of use. Various business models, revenue models, and delivery mechanisms for the present invention are envisioned, and are all to be considered within the scope of the present invention.
In general, the method executed to implement the embodiments of the invention, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as "computer program(s)" or "computer code(s)." The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations necessary to execute elements involving the various aspects of the invention. Moreover, while the invention has been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments of the invention are capable of being distributed as a program product in a variety of forms, and that the invention applies equally regardless of the particular type of machine or computer-readable media used to actually effect the distribution. Examples of computer-readable media include but are not limited to recordable type media such as volatile and non volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.), and digital and analog communication media.