WO2021244799A1 - Classification d'événements d'écoulement - Google Patents

Classification d'événements d'écoulement Download PDF

Info

Publication number
WO2021244799A1
WO2021244799A1 PCT/EP2021/059679 EP2021059679W WO2021244799A1 WO 2021244799 A1 WO2021244799 A1 WO 2021244799A1 EP 2021059679 W EP2021059679 W EP 2021059679W WO 2021244799 A1 WO2021244799 A1 WO 2021244799A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
pipe
flow
pipe system
model
Prior art date
Application number
PCT/EP2021/059679
Other languages
English (en)
Inventor
Saed HUSSAIN
Original Assignee
Homeserve Plc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Homeserve Plc filed Critical Homeserve Plc
Publication of WO2021244799A1 publication Critical patent/WO2021244799A1/fr

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M3/00—Investigating fluid-tightness of structures
    • G01M3/007—Leak detector calibration, standard leaks
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M3/00—Investigating fluid-tightness of structures
    • G01M3/002—Investigating fluid-tightness of structures by using thermal means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M3/00—Investigating fluid-tightness of structures
    • G01M3/02—Investigating fluid-tightness of structures by using fluid or vacuum
    • G01M3/04—Investigating fluid-tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point
    • G01M3/24—Investigating fluid-tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using infrasonic, sonic, or ultrasonic vibrations
    • G01M3/243—Investigating fluid-tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using infrasonic, sonic, or ultrasonic vibrations for pipes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
    • G01M3/00—Investigating fluid-tightness of structures
    • G01M3/38—Investigating fluid-tightness of structures by using light
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Definitions

  • the present disclosure is in the field of pipe systems, for example, water plumbing systems in a property.
  • the present disclosure is in the field of flow event detection in such pipe systems.
  • a pipe system in a given installation can be defined as a system of systems, which may consist of several subsystems (e.g. header tank, connecting pipes, bath, kitchen sink, etc.). These subsystems can generate events, which will leave event-signatures in the temperature domain that may be used to detect fluid flow.
  • subsystems e.g. header tank, connecting pipes, bath, kitchen sink, etc.
  • Detecting fluid flow in pipe systems has known uses in the field of leak detection.
  • the temperature of a pipe in the pipe system and the ambient temperature near the pipe are monitored simultaneously. If there is no flow in the pipe, the pipe temperature will eventually converge to the ambient temperature. If there is flow in the pipe, the pipe will consistently have a lower temperature than the ambient temperature as the flow of new, cold water in the pipe keeps the temperature of the pipe low. Detecting the presence or lack thereof of temperature convergence between the pipe temperature and the ambient temperature may thus be used to detect flow in the pipe system and thus detect a leak in the pipe system. Examples of such approaches are described in WO2016/110696 A1 and WO2017/118834 A1.
  • a challenge of the approach described above is that not all flow in a pipe system is indicative of a leak.
  • the running of a cold tap in the pipe system will also result in detectable flow and may give a false impression that there is a leak.
  • the approaches of WO2016/110696 A1 and WO2017/118834 A1 seek to overcome this challenge by excluding certain parts of collected data. For example, they may ignore data from known busy times of the day (e.g. during a resident’s morning routine where it is likely they will be using the shower, bath, sink, etc and thus create flow in the pipe system). Instead, such approaches focus leak detection analysis on data from other, quieter parts of the day when it is known that any detected flow is likely to be due to a leak rather than normal use of the pipe system.
  • W02020/035696 A1 describes such an approach.
  • training data is obtained comprising known flow states of pipes together with the associated pipe temperature data and ambient temperature data in the vicinity of a pipe.
  • a decision tree is generated from the training data that may be used on pipes whose flow state is not initially known to estimate its flow state.
  • the decision tree approach of W02020/035696 A1 works well for the temperature based data it is said to use.
  • W02020/035696 A1 advocates a decision tree approach may include that decision tree approaches are efficient and fast when used on large but simple datasets, such as the temperature data sets referred to in W02020/035696 A1 .
  • WO2016/110696 A1 WO2017/118834 A1 and W02020/035696 A1
  • WO2017/118834 A1 a problem of all of the approaches of WO2016/110696 A1 , WO2017/118834 A1 and W02020/035696 A1 is that they do not generalise well to the wide variety of pipe systems that exist around the world.
  • the temperature of water entering a pipe system in a very cold country may be warmer than the ambient temperature the pipe being monitored.
  • Heating systems connected to the pipe system may cause changes in both the ambient and the pipe temperature readings at certain times.
  • Underfloor heating systems may effect temperature readings differently to other types of systems such as central heating systems and/or air conditioning based heating systems.
  • Water use routines of residents in properties varies by country, culture, age and other factors.
  • the present disclosure provides a computer implemented method that relies on a plurality of types of data sources in which event signatures manifest themselves including non-temperature types of data. Accordingly, the presented proposal exploits machine learning to map and diagnose a pipe system, using not just temperature domain event signatures but also other domain event signatures and/or relevant meta-data.
  • the present disclosure may rely at least on one or more of: visual data (such as an image of the pipe being monitored), pipe materials, pipe diameter, acoustic data indicative of vibrations in the pipe system (such as vibrations associated with different types of flow events, plumbing elements and/or length of piping in the pipe system), time and data date of flow events, geographic location data, user data indicative of a behaviour of a user of the pipe system, and other types of data.
  • visual data such as an image of the pipe being monitored
  • pipe materials such as pipe materials, pipe diameter
  • acoustic data indicative of vibrations in the pipe system such as vibrations associated with different types of flow events, plumbing elements and/or length of piping in the pipe system
  • time and data date of flow events such as vibrations associated with different types of flow events, plumbing elements and/or length of piping in the pipe system
  • time and data date of flow events such as vibrations associated with different types of flow events, plumbing elements and/or length of piping in the pipe system
  • time and data date of flow events such as vibrations
  • the present disclosure also provides a computer implemented method to check if a flow event detection apparatus is fitted correctly to a pipe being monitored. In this way, any problems in the accuracy of the data collected by the flow event detection apparatus can be determined before they propagate into an entire data set.
  • a computer implemented method for classifying one or more flow events in a pipe system into one or more classes of flow events comprising providing input data from a plurality of data sources, such as those types described above or other types.
  • the input data comprises flow event signals associated with one or more of said flow events during a time period.
  • the method comprises applying a first classifier to the input data to classify the one or more flow event signals into the one or more classes of flow events.
  • flow event signals means patterns, trends, and/or other features in the input data that are characteristic of a certain flow event. For example, a resident of a property with a combi-boiler preparing a warm bath will result in water flow in cold supply pipes of the pipe system (both to the bath faucet and to the combi-boiler for heating) as well as in the warm supply pipes (from the combi-boiler to the bath faucet where it will mix with the cold water).
  • the flowing water will cause a drop in temperature of a monitored cold supply pipe, vibrations will propagate through the pipe system caused by, for example, the actuation of the faucet at the bath, the water flow into and out of the combi-boiler and the actuation of one or more valves therein together with any vibrations caused by the combi-boiler’s water heating procedures, the resident may switch off the lights and/or other devices in the house and move to the bathroom, which leaves a footprint in any smart device logs in the property....
  • the first classifier may comprise one or more of a convolutional neural network (CNN) model and a recurrent neural network (RNN) model.
  • CNN convolutional neural network
  • RNN recurrent neural network
  • a CNN and/or RNN model instead of a decision tree approach overcomes the above described problems of a decision tree approach when applied to a wider variety of data types than just temperature data in the context of flow event detection.
  • algorithms that use a decision tree approach are known as “greedy” algorithms in that they search only some of the possibilities of a relatively larger hypothesis space and may miss the global optimum. Accordingly, an “optimum” solution outputting a probability value indicating the presence of a given flow event may in fact be inaccurate because the solution found was not optimum globally.
  • Using a CNN and/or RNN model instead of the decision tree model significantly reduces the risk of finding a local optimum and instead increases the likelihood that a solution is in fact a global optimum. Accordingly, the resulting output of the classifier is more accurate than outputs provided by the decision tree approach of W02020/035696 A1 .
  • the method may comprise generating a pipe system model from said classified flow event signals.
  • the pipe system model comprises information indicating the presence of one or more of said plumbing elements and/or length of piping present in the pipe system.
  • the present disclosure considers flow event signals associated with flow events other than leaks to be useful information that is not to be discarded but which may be used to beneficially create a model of the pipe system.
  • the model may for example comprise a list or database with entries relating to plumbing elements, piping lengths, connections between them, and/or other parameters that define the layout of the pipe system and its behaviour as a system.
  • Such information may find uses not only in the field of leak detection as is described in WO2016/110696 A1 , WO2017/118834 A1 and W02020/035696 A1 , but also in civil engineering, water usage analytics, marketing in the plumbing sector, and other fields in which such information may be combined with other information used to improve the efficiency of existing pipe system designs and/or develop new designs. For example, it may be apparent that a certain pipe system design whose pipe system model is generated according to the present disclosure has a higher instance of leaks, high/low water pressure, and similar events. This information may be used to modify such a design to reduce such leaks and events in future. It could also be used by local and national authorities to facilitate planning.
  • the step of generating the pipe system model may comprise associating the classes of flow events with the presence of one or more of said plumbing elements and/or length of piping in the pipe system, and adding a model of said associated plumbing element or piping length to the pipe system model (for example adding an entry corresponding thereto into a list or database and establishing a connection and/or functional to the other entries in the database) when a flow event signal is classified into one or more of said classes of flow events.
  • the plurality of data sources may comprise one or more of:
  • -temperature data based on one or more of: a pipe temperature of a monitored pipe, an ambient temperature in the vicinity of the monitored pipe, a ground and/or air temperature at the geographic location of the pipe system;
  • -second acoustic data indicative of vibrations in the pipe system associated with one or more of: a length of piping in the pipe system, a plumbing element in the pipe system;
  • the method may comprise collecting the first acoustic data by recording vibrations in one or more pipes of the pipe system during one or more of the flow events with one or more vibration sensors connected to the pipe system.
  • vibrations typically travel very efficiently and with very little loss in a pipe system so may be easily recorded with a high degree of fidelity by connecting one or more a vibration sensors (for example an accelerometer or piezo vibration sensor) to a pipe or pipes. Accordingly, recording these vibrations to collect acoustic data provides a new source of high quality flow event signals that have previously not been collected in leak detection systems such as those described in WO2016/110696 A1 , WO2017/118834 A1 , W02020/035696 A1.
  • acoustic data is largely unaffected by seasonal changes, variations caused by property heating systems, and the other factors described above in the background section of the present disclosure and thus provides an improved source of input data from which not only leaks but also other flow events may be classified.
  • the method may comprise collecting the second acoustic data by inducing vibrations at a predetermined frequency and amplitude in one or more pipes of the pipe system and recording these vibrations with one or more vibration sensors (for example those described above to collect the first acoustic data, or other vibration sensors such as accelerometers or piezo vibration sensors).
  • This second acoustic data may comprise data associated with reflections of acoustic waves of the induced vibrations off one or more of the plumbing elements of the pipe system.
  • inducing vibrations whose parameters such as frequency and parameters are known provides a way to actively probe the layout and design of the pipe system. This is in contrast with the passive recording described above in connection with the first acoustic data.
  • the first acoustic data may provide a good source of flow event signals associated with specific flow events which may be indicative of a plumbing element and/or length of piping
  • actively probing the pipe system with controlled vibrations provides additional information not obtainable from the passive recording of the first acoustic data. For example, reflections of acoustic waves off one or more plumbing elements and/or pipe lengths may be isolated and recorded from the induced vibration waves.
  • the time it takes for an induced vibration to do a round trip through the pipe system and be recorded at one or more of the vibration sensors allows the total length of piping in the system to be determined.
  • the reflections from different plumbing elements and/or length of pipe will have characteristic features that may be used to identify the presence of that plumbing element and/or length of pipe. For example, an acoustic reflection from boiler may look different to a reflection from a kitchen tap and the active probing by inducing vibrations allows these different elements and pipe lengths to be identified and optionally added to the plumbing system model.
  • the method may comprise collecting the image of the monitored pipe by capturing the image of the pipe with a portable electronic device.
  • the method may comprise applying a second classifier to the captured pipe image data to classify the monitored pipe into one or more classes of pipe.
  • the one or more classes of pipe may comprise different pipe diameters and/or pipe materials.
  • the second classifier may comprise one or more of a second CNN model and/or a second RNN model.
  • the physical material and/or diameter of the pipe being monitored may impact how the flow event signal manifests itself in the data.
  • copper piping conducts heat better than plastic piping and the wave speed of acoustic waves in copper piping is different to the wave speed in plastic piping.
  • the diameter of the pipe is correlated to the amount of material present and thus its capacity to absorb heat and/or is correlated to the shape of an acoustic wave propagating therethrough. Accordingly, obtaining this information may enhance the strength to noise ratio of associated flow event signals in the input data and may thus make such flow event signals easier to recognise by the first classifier. Accordingly, detecting this information using image recognition techniques, and in particular a second classifier (for example comprising a CNN and/or RNN trained for this task), may thus further improve the accuracy of the method according to the present disclosure.
  • a second classifier for example comprising a CNN and/or RNN trained for this task
  • the method may comprise generating plot image data of a first portion of the input data, the plot image data comprising a plot of said first portion of the input data against time, said plot comprising plot features indicative of said one or more flow event signals.
  • the applying of the first classifier may then comprise inputting said image data into the first CNN model to classify said plot features indicative of said one or more flow event signals into said classes of flow events.
  • CNN models are particularly effective at image recognition tasks. Accordingly, the inventors have appreciated that converting a portion of the input data into an image, for example a plot of the first portion of data against time, allows the advantages of CNN models in their effectiveness at image recognition tasks to be applied to data that may otherwise not have been analysed as an image. In particular, the inventors have unexpectedly found that flow event signals in pipe systems, in particular but not limited to, those in acoustic data manifest themselves in data plots in a similar manner to how heart murmurs manifest themselves in phonocardiogram plots.
  • CNN models have been used to classify signals in a phonocardiogram associated with heart murmurs to a high level of accuracy for example as is described in Alam, Shahnawaz & Banerjee, Rohan & Bandyopadhyay, Soma. (2016). Murmur Detection Using Parallel Recurrent & Con volutional Neural Networks, arXiv: 1808.04411, 13 August 2018. Accordingly, the inventors have appreciated that a similar level of accuracy may be achieved in the classification of flow event signals into one or more classes of flow events using a similar CNN architecture.
  • the CNN model learns the visual and time dependent characteristics of the flow event signals present in the above described data plots and provides improved accuracy and ability to generalise to the input data compared to, for example, the decision tree approach of W02020/035696 A1 .
  • the inventors have also appreciated flow event signals manifest themselves in the temperature data in a similar manner and accordingly that the first portion of input data from which the plot is generated may comprise one or more of the temperature data, acoustic data, and/or other data types described above where flow event signals manifest themselves in the data.
  • the recurrent neural network model described above may, where present, comprise a long short-term memory (LSTM) model, for example a bidirectional LSTM model.
  • the applying of the first classifier to the input data may accordingly further comprise inputting a second portion of the input data into the LSTM model to classify the one or more flow event signals in the second portion of the input data into the classes of flow events.
  • the second portion may partially or fully overlap with the first portion and/or be the same portion. It may also comprise multidimensional data for example corresponding to any of the data types described above.
  • the second portion of data may also be processed for example by performing a fourier transform to enhance any time dependent flow event signals in the data.
  • the applying of the first classifier may further comprise combining the first CNN model with the LSTM model by combining an output layer of the CNN model with an output layer of the LSTM model to form a fully connected layer.
  • the presently described first classifier may comprise some or all of the features of the CNN and LSTM described in Alam, Shahnawaz & Banerjee, Rohan & Bandyopadhyay, Soma. (2016). Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks, arXiv: 1808.04411, 13 August 2018.
  • the one or more classes of flow events may comprise, for example but not limited to, one or more of a fast leak event, a slow leak event, a bath running event, a shower running event, a washing machine running event, a dishwasher running event, a kitchen tap running event, a bathroom sink tap running event, a boiler running event, a header tank filling event, a low water pressure event, and/or a high water pressure event.
  • the one or more plumbing elements of a pipe system may comprise, for example but not limited to, one or more of a bath tap, a shower head, a washing machine, a dishwasher, a kitchen tap, a bathroom sink tap, a boiler (a combi-boiler or other boiler type), and/or a header tank.
  • the method comprises providing image data of the detection apparatus mounted to the pipe and applying a third classifier to classify the image data into the first class or the second class.
  • the third classifier may comprise one or more of a third convolutional neural network (CNN) model and a third recurrent neural network (RNN) model.
  • CNN convolutional neural network
  • RNN third recurrent neural network
  • any problems in the accuracy of the data collected by the flow event detection apparatus as a result of a bad or incorrect fitting can be determined before such problems (for example errors in the data) propagate into an entire data set.
  • a data-processing apparatus comprising means for carrying out the steps of any of the above described methods, a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of any of the above methods, and a computer readable storage medium having stored thereon such a computer program.
  • Figure 1 is a flowchart of a method according to the present disclosure.
  • Figure 2 is an illustration of a system and corresponding method according to the present disclosure.
  • Figure 3 is an illustration of a system and corresponding method according to the present disclosure.
  • Figure 4a illustrates an example classifier architecture according to the present disclosure.
  • Figure 4b illustrates an example classifier architecture according to the present disclosure.
  • Figure 5 illustrates an example classifier architecture according to the present disclosure.
  • Figure 6 illustrates a system and corresponding method according to the present disclosure.
  • Figure 7 illustrates a system and corresponding method according to the present disclosure.
  • Figure 8 illustrates a method according to the present disclosure.
  • Figure 9 illustrates an exemplary data-processing apparatus according to the present disclosure.
  • Figure 10 illustrates a method according to the present disclosure.
  • Figure 1 is a flowchart of a computer implemented method 100 according to the present disclosure for classifying one or more flow events in a pipe system into one or more classes of flow events, the pipe system having one or more lengths of piping and one or more plumbing elements connected thereto.
  • the method 100 comprises providing 101 input data form a plurality of data sources, the input data comprising flow event signals associated with one or more of said flow events during a time period over which monitoring is occurring and applying 102 a first classifier to the input data to classify the one or more flow event signals into the one or more classes of flow events.
  • FIG 2 illustratively shows a system 200 and corresponding method according to the present disclosure suitable for performing the method shown in Figure 1 .
  • Input data from a plurality of sources 201a, 201 b, 201c associated with a pipe system comprising one or more length of piping and one or more plumbing elements is provided and input into a data processing apparatus 202.
  • a data processing apparatus 202 Although only three data sources are shown in Figure 2, it is envisaged that this may be any number of different data sources, for example different types of data sources including one or more of those described above such as temperature data, acoustic data and others.
  • Manifested in the input data from these data sources are characteristic signatures of flow events in the pipe system described herein as flow event signals.
  • the data processing apparatus 202 is be configured to apply the first classifier 203 to the input data from the plurality of sources to classify these flow event signals into one or more classes of flow events. .
  • the data processing apparatus 202 may comprise means for carrying out the steps of any of the methods according to the present disclosure and accordingly may comprise a computer-readable storage medium having stored thereon a computer program which, when executed by the data processing apparatus 202, causes it to perform the steps of any of the methods according to the present disclosure.
  • the data processing apparatus 202 may be entirely contained within a single apparatus or it may be distributed amongst a number of different apparatuses such as servers in the cloud.
  • FIG. 3 illustratively shows a system 300 and corresponding method according to the present disclosure.
  • the system comprises a pipe system 301 having the one or more lengths of pipe 302 and plumbing elements 303 from which at least some of the input data may be collected.
  • the input data may be collected with, for example, a flow event detection system or apparatus 304 as is described in WO2016/110696 A1 and WO2017/118834 A1 which are incorporated in their entirety herein by reference.
  • the flow event detection apparatus 304 may comprise one or more sensors 305, for example one or more pipe temperature sensors 305a, ambient temperature sensors 305b and/or vibration sensors 305c connected to one or more pipes 302 or plumbing elements 303 of the pipe system 301.
  • the vibration sensor(s) are configured to record not only vibrations in one or more of the pipes of the pipe system caused by one or more of the flow events, but also vibrations intentionally induced in the pipe system (for example at a predetermined frequency and amplitude) by a vibration inducer to probe the lay out and design of the pipe system.
  • the flow event detection apparatus 304 may comprise a data processing apparatus 306 in communication with one or more network locations, for example in the cloud 307, which may include one or more additional data processing apparatuses 308.
  • the data processing apparatuses 306, 308 may individually perform all of the steps of the method according to the present disclosure or may perform only some of them, with the rest being performed by the one or more data processing apparatuses 306, 308 in a distributed manner as will be appreciated by the skilled person. Additional sources of input data may also be provided by other types of sources 309a, 309b, 309c, such as from user devices such as smartphones (e.g. pipe image data corresponding to an image of a monitored pipe in the pipe system captured by a camera, a user’s location data, user data indicative of a user’s behaviour (e.g. a morning shower routine etc) associated with the pipe system stored on such user devices) and smart devices in the vicinity of the pipe system (e.g.
  • user devices such as smartphones (e.g. pipe image data corresponding to an image of a monitored pipe in the pipe system captured by a camera, a user’s location data, user data indicative of a user’s behaviour (e.g. a morning shower routine etc) associated with the pipe system
  • Previously collected and stored data may also be provided as sources 310a 310b, 310c of input data. This may include, for example, previously collected and stored data of the types described above as well as information associated with the piping and or plumbing elements of the pipe system (such as pipe diameter and material and/or one or more of their physical properties). This type of data may be stored at one or more network locations, for example in the cloud.
  • the input data comprises flow event signals associated with one or more flow events in the pipe system during a time period being monitored.
  • the input data may be provided to the one or more data processing apparatuses 306, 308, where a first classifier is applied to the input data as is described above in connection with Figures 1 and 2 to classify the flow event signals into one or more classes of events such as, for example, one or more of: a fast leak event, a slow leak event, a bath running event, a shower running event, a washing machine running event, a dishwasher running event, a kitchen tap running event, a bathroom sink tap running event, a boiler running event, a header tank filling event, a low water pressure event, and a high water pressure event, and other events which may have flow event signals manifesting in the input data.
  • Figure 4a illustrates an example architecture 400 of a first classifier according to the present disclosure.
  • the architecture comprises a CNN model.
  • the input for the CNN model comprises image data of a plot 401 of a first portion of the input data against time for a monitored period.
  • the characteristic signatures of flow events, that is the flow event signals, in the first portion of the data may manifest as image features in the plot 401 (i.e. plot features) that will accordingly be indicative of the one or more flow event signals.
  • the CNN model is more effective at identifying and extracting such image features compared a decision tree model as is used in existing methods and it has surprisingly been found that using the image recognition capabilities of CNN models on images of the input data (rather than applying a decision tree model to the raw data) results in a higher level of accuracy in classification of flow event signals into classes of flow events.
  • the plot 401 used as input into the CNN model is a plot of ambient and pipe temperature data against time for a monitored period but the inventors have found that the same increase in accuracy compared to a decision tree model is observed for other types of the input data, in particular acoustic data as referred to herein.
  • a first convolutional layer 402 is followed by a 1 st pooling layer 403 which feeds into a 2 nd convolutional layer 404 and then into a 2 nd pooling layer 405, finally ending in a 3 rd convolutional layer 406 and optionally a softmax layer 407 the output of which may be used to determine classification into the one or more classes of flow events.
  • the CNN architecture provided herein is provided for illustrative purposes only and it is envisaged that other CNN architectures may also be used to achieve similar and/or better levels of accuracy.
  • FIG. 4b illustrates another example architecture 408 of a first classifier according to the present disclosure.
  • the architecture 408 comprises a RNN model, in particular a bidirectional long short-term memory (BiLSTM) model.
  • the input for the BiLSTM model comprises one or more portions 409a, 409b, 409c of the input data.
  • This may include raw numerical values (for example binary, discrete, continuous or other values) and/or other data that be derived from such values (for example, fourier transforms in the time/frequency domains and/or other pre-processing methods to convert the input data into a format suitable for input into a RNN model such as a BiLSTM model.
  • These portions of data may contain the same data and/or data derived from the image plot data described above in connection with Figure 4a.
  • the BiLSTM model is effective at extracting temporal patterns in the input data.
  • a many flow events have a temporal pattern in the input data.
  • a leaking tap may drip at in a regular manner at a specific frequency and accordingly such a flow event may have a flow event signal that manifests itself as a temporal pattern in one or more portions of the input data, for example as a regular acoustic vibration.
  • a user may take a shower at the same time every working day of every week of the year.
  • a corresponding flow event signal may manifest itself as a temporal pattern in the ambient and pipe temperature data derived values.
  • the inventors have thus found that a RNN such as the BiLSTM model shown in Figure 4b is effective at extracting these and other patterns and thus at classifying the corresponding flow event signals into the one or more corresponding flow event classes.
  • BiLSTM layers 410, 411 are provided after a BiLSTM input layer.
  • the BiLSTM layers have bidirectional flow to process the sequence of data in both the forward and backward direction and to feed forward to the output, which may optionally feed into a softmax layer 413, the output of which may be used to determine classification into the one or more classes of flow events.
  • Figure 5 illustrates an example architecture 500 of a first classifier according to the present disclosure.
  • the architecture combines the CNN model and RNN model architectures of Figures 4a and 4b into a single architecture.
  • image plot data 501 is input into the CNN model and the CNN model comprises a first convolutional layer 502 is followed by a 1 st pooling layer 503 which feeds into a 2 nd convolutional layer 504 and then into a 2 nd pooling layer 505, finally ending in a 3 rd convolutional layer 506.
  • portions of input data 507a, 507b, 507c form an input layer 508 which feed into 1 st and 2 nd BiLSTM layers 509, 510.
  • the output of the CNN model and the BiLSTM model are combined which includes flattening 511 the outputs of the CNN model and BiLSTM model and feeding these into a 1 st fully connected layer 512 and a 2 nd fully connected layer 513 before finally ending in an optional softmax layer 514 whose output may be used to determine the classification into the one or more flow event classes.
  • Figure 6 illustrates a system 600 and corresponding method according to the present disclosure.
  • a plurality of data sources 601a, 601b, 601c provided input data to a data processing apparatus 602 and flow event signals in the input data are classified by a classifier 603 into one or more flow event classes 604a, 604b, 604c in the same way as is illustrated and described in connection with Figures 1 and 2.
  • the data sources may correspond to and be collected by the system described in connection with Figure 3 and/or the classifier may have one or more of the architectures described in connection with Figures 4a, 4b and 5.
  • the data processing apparatus generates a pipe system model 605 from the classified flow event signals (or may be provided with a partly-generated or template of pipe system model).
  • the pipe system model 605 may for example comprise a list or database with entries relating to plumbing elements 606a, piping lengths 606b, connections between them, and/or other parameters that define the layout of the pipe system and its behaviour as a system.
  • Flow events in the pipe system are typically associated with the operation of one or more plumbing elements (for example, the running of a tap, activation of a boiler, filling of a header tank, etc). Accordingly, when one or more flow events signals have been classified, indicating a given flow event has taken place, the presence of a plumbing element corresponding to such a flow event may be identified as present in the pipe system.
  • a model of such a plumbing element 606c, 606d, 606e may accordingly be added to the pipe system model 605 (including any additions or modifications to lengths of piping present in the system). This may comprise, for example, adding new entries to a database and/or list, and/or recording any functional relationships between them.
  • the classes of flow events may be associated with the presence of one or more of the plumbing elements and/or length of piping in the pipe system, and a model of these may be added to the pipe system model when a flow event signal is classified into one or more of the corresponding classes of flow events.
  • the information contained in the generated pipe system model built up in this stepwise manner advantageously allows the behaviour of the pipe system to be modelled numerically.
  • the present disclosure allows an engineer obtain the functional details of a pipe system in a property from the classified flow events without needing to cause physical disruption to the property (for example, the engineer may no longer require access to pipe conduits and service rooms and/or the interior of property wall and ceiling cavities only accessible by damaging the property).
  • FIG. 7 illustrates a system 700 and corresponding method according to the present disclosure.
  • a pipe system 701 such as the one illustrated and described in connection with Figure 3 is shown.
  • the pipe system comprises one or more lengths of pipe 702 and plumbing elements 703.
  • a vibration inducer 704 (for example a transducer such as a mechanical actuator) is provided.
  • the vibration inducer 704 may be connected to the pipe system, for example to one or more of the pipes of the pipe system, and is configured to induce vibrations in the pipe system.
  • the acoustic data associated with these induced vibrations may be recorded by, for example, one or more vibration sensors 705 connected to the pipe system, for example the vibration sensor(s) illustrated in Figure 3.
  • the inventors may advantageously be used to probe the lay out of the pipe system and thus improve the accuracy of a generated pipe system model.
  • the recorded acoustic data may comprise data associated with reflections of acoustic waves of the induced vibrations off one or more plumbing elements. As described above, these reflections may be indicative of the presence of one or more plumbing elements in the pipe system, a total length of piping in the pipe system, and other details of the pipe system.
  • the acoustic data and or any additional pieces of information may be used as input data into the first classifier and, together with the other input data described herein further enhance the accuracy of the first classifier in classifying the one or more flow event signals into the classes of flow events.
  • the flow event signals in one or more portions of input data of a running bath faucet and of a running shower faucet may initially be similar and difficult for a classifier to distinguish.
  • Flowever, induced vibrations in the pipe system and the recorded acoustic data associated therewith may be indicative of the presence of only a shower faucet and not a bath faucet, accordingly the induced vibration acoustic data may assist the classifier in distinguishing between flow event signals which are otherwise difficult to distinguish between.
  • the induced vibration acoustic data may be used to further enhance the accuracy of the generated pipe system model, for example by increasing the estimate of total pipe length in the model and/or confirming whether or not an added plumbing element is likely to be correct or as a result of an incorrect classification.
  • Figure 7 also illustrates a portable electronic device 706 for example a smartphone with a camera which may be used to capture an image of a pipe being monitored.
  • a portable electronic device 706 for example a smartphone with a camera which may be used to capture an image of a pipe being monitored.
  • This may be useful in determining for example the material of the pipe and/or its diameter.
  • both the material and diameter of a pipe may affect temperature changes and/or acoustic wave propagation through the pipe system. Accordingly, by identifying this from an image captured by a portable electronic device, these parameters may be used as input data for the first classifier.
  • a second classifier may be applied to the image data of the image of the pipe to classify the monitored pipe into one or more classes of pipe.
  • the one or more classes of pipe may comprise different pipe diameters and/or pipe materials.
  • a CNN typically has strong image recognition capabilities and it is envisaged that the second classifier may comprise a second CNN model.
  • the second classifier may additionally and/or alternatively comprise a
  • a further use for an image of a monitored pipe is that it may be used to determine if a flow event detection apparatus of the type described in connection with Figure 3 and/or in WO2016/110696 A1 and WO2017/118834 A1.
  • Figure 8 illustrates such a method according to the present disclosure which may be used in connection with any of the methods and system described here. For example, using the image captured as described in connection with Figure 7.
  • the method 800 comprises providing 801 image data of the detection apparatus mounted to the pipe and applying 802 a third classifier to the image data of the detection apparatus mounted to the pipe to classify the image data into the first class or the second class.
  • the third classifier may comprise a CNN model and/or a RNN model.
  • an exemplary data-processing apparatus comprising means for carrying out the steps of the methods of any of the above embodiments.
  • the method steps herein are carried out entirely on a CPU which is cheaper than a GPU and may be more suitable for a consumer device, for example the flow event detection apparatus connected to a pipe system of a property.
  • the data-processing apparatus may comprise a processor 900 that is in communication with memory devices including secondary storage 901 (such as disk drives), read only memory (ROM) 902, random access memory (RAM) 903.
  • the processor 900 may be implemented as one or more CPU chips, which are cheaper than GPUs.
  • the data- processing apparatus may further comprise input/output (I/O) devices 904, and network connectivity devices 905.
  • the secondary storage 901 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 903 is not large enough to hold all working data. Secondary storage 901 may be used to store programs which are loaded into RAM 903 when such programs are selected for execution.
  • the secondary storage 901 has an order processing component 901a comprising non-transitory instructions operative by the processor 900 to perform various operations of the method of the present disclosure.
  • the ROM 902 is used to store instructions and perhaps data which are read during program execution.
  • the secondary storage 901 , the RAM 903, and/or the ROM 902 may be referred to in some contexts as computer readable storage media and/or non-transitory computer readable media.
  • I/O devices 904 may include printers, video monitors, liquid crystal displays (LCDs), plasma displays, touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
  • LCDs liquid crystal displays
  • plasma displays plasma displays
  • touch screen displays touch screen displays
  • keyboards keypads
  • switches dials
  • mice track balls
  • voice recognizers card readers, paper tape readers, or other well-known input devices.
  • the network connectivity devices 905 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards that promote radio communications using protocols such as code division multiple access (CDMA), global system for mobile communications (GSM), long-term evolution (LTE), worldwide interoperability for microwave access (WiMAX), near field communications (NFC), radio frequency identity (RFID), and/or other air interface protocol radio transceiver cards, and other well-known network devices. These network connectivity devices 905 may enable the processor 900 to communicate with the Internet or one or more intranets.
  • CDMA code division multiple access
  • GSM global system for mobile communications
  • LTE long-term evolution
  • WiMAX worldwide interoperability for microwave access
  • NFC near field communications
  • RFID radio frequency identity
  • RFID radio frequency identity
  • the processor 900 might receive information from the network, or might output information to the network in the course of performing the above-described method operations.
  • Such information which is often represented as a sequence of instructions to be executed using processor 900, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.
  • the processor 900 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 901), flash drive, ROM 902, RAM 903, or the network connectivity devices 905. While only one processor 900 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor, the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors.
  • the technical architecture may be formed by two or more computers in communication with each other that collaborate to perform a task.
  • an application may be partitioned in such a way as to permit concurrent and/or parallel processing of the instructions of the application.
  • the data processed by the application may be partitioned in such a way as to permit concurrent and/or parallel processing of different portions of a data set by the two or more computers.
  • virtualization software may be employed by the technical architecture to provide the functionality of a number of servers that is not directly bound to the number of computers in the technical architecture.
  • the functionality disclosed above may be provided by executing the application and/or applications in a cloud computing environment.
  • Cloud computing may comprise providing computing services via a network connection using dynamically scalable computing resources.
  • a cloud computing environment may be established by an enterprise and/or may be hired on an as-needed basis from a third party provider.
  • flow event signals manifest themselves in the input data, in particular but not limited to, the acoustic data in a similar manner to flow events in oil and gas pipelines and risers as is described in, for example, Jung S, Yang H, Park K, Seo Y, Seong W. Monitoring Severe Slugging in Pipeline-Riser System Using Accelerometers for Application in Early Recognition. Sensors (Basel). 2019; 19( 18):3930. Published 2019 Sep 12. doi:10.3390/s19183930.
  • the inventors have appreciated that a similar level of accuracy in identifying the flow event signals in a pipe system of the type described above may be achieved using the support vector machine (SVM) and NN architectures described in this paper.
  • SVM support vector machine
  • the first classifier may additionally and or alternatively comprise a support vector machine model and may have some or all of the features described in Jung S, Yang H, Park K, Seo Y, Seong W. Monitoring Severe Slugging in Pipeline-Riser System Using Accelerometers for Application in Early Recognition. Sensors (Basel). 2019;19(18):3930. Published 2019 Sep 12. doi:10.3390/s19183930.
  • the plurality of data sources in any of the systems and methods described herein may comprise for example one or more of:
  • -temperature data based on one or more of: a pipe temperature of a monitored pipe, an ambient temperature in the vicinity of the monitored pipe, a ground and/or air temperature at the geographic location of the pipe system;
  • -second acoustic data indicative of vibrations in the pipe system associated with one or more of: a length of piping in the pipe system, a plumbing element in the pipe system;
  • the classifiers described herein may be trained using one or more training methods which may be known to the skilled person on a suitably sized body of training data.
  • the training data may comprise data collected from a test property, for example a house, block of flats and/or other commercial and/or residential property having a pipe system whose sole purpose is to reproduce flow events in order to create a training data set with data in which flow event signals are present.
  • synthetic training data may be generated by taking existing training data and adding artificial noise to it as will be known to the skilled person.
  • the methods described herein may also comprise one or more further steps such as data storage steps and pre-processing steps that may be performed as part of the providing of input data from a plurality of data sources (for example the plurality of data sources as described and illustrated with reference to Figure 3).
  • Figure 10 provides an illustrative example of such steps.
  • Messages containing relevant data (e.g. temperature or other data) from the plurality of data sources 1000 may be sent to a centralised data storage 1001 such as, for example, a BigQueryTM cloud storage system or similar.
  • a centralised data storage 1001 such as, for example, a BigQueryTM cloud storage system or similar.
  • the messages may be provided as json files.
  • the data may be provided as a live stream from one or more of the devices which are collecting data.
  • Pre-processing 1002 may be performed on the data logs, this may include checking data integrity and/or parsing and formatting the data to ensure the data is in a useable state for further processing.
  • python code may be used to obtain all messages from a given 24 hour period to build a 24-hour log file (for example a csv file) for a given date and device ID.
  • the model outputs 1005 whether or not a flow event (e.g. a leak) is detected.
  • the output may be for example a probability that a given flow event such as a leak is detected in the period being analysed (for example, the 24 hour period, the quiet period, or some other period). This result may be fed further into other steps, for example, to alert a user that a leak has been detected.
  • one or more signals in the input data may be indicative of an incorrect installation of a flow event detection apparatus.
  • the device may be attached to an incorrect pipe in the pipe system or simply not attached correctly due to customer error and this may lead to inaccurate readings and false alarms.
  • the classifier described herein for example a CNN, RNN and/or a different model such as a decision tree model
  • an alert may be generated to indicate that the apparatus is not installed correctly.
  • the output 1006 of the model may be for example a probability that one or more detection apparatuses are incorrectly fitted.
  • one or more signals in the input data may be used as part of a predictive maintenance method.
  • subsystems of a pipe system can develop failures. Identifying these failures in advance may mitigate future damage that may otherwise be caused.
  • the output 1007 of the model may be for example a probability that one or more subsystems of the pipe system are likely to fail within a given period given the flow event signatures of the input data.
  • a pipe joint that is slowly dripping may in future develop a faster drip and/or even a catastrophic failure leading to a burst pipe joint.
  • the initial dripping may manifest itself as a signature in the acoustic data and this may be indicative that the pipe joint may fail within a number of months.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Examining Or Testing Airtightness (AREA)

Abstract

L'invention concerne un procédé mis en œuvre par ordinateur pour classifier un ou plusieurs événements d'écoulement dans un système de tuyaux en une ou plusieurs classes d'événements d'écoulement, le système de tuyaux ayant une ou plusieurs longueurs de tuyauterie et un ou plusieurs éléments de plomberie reliés à celui-ci, le procédé consistant à : fournir des données d'entrée provenant d'une pluralité de sources de données, les données d'entrée comprenant des signaux d'événement d'écoulement associés à un ou plusieurs desdits événements d'écoulement pendant une période de temps ; et appliquer un premier classificateur aux données d'entrée pour classifier le ou les signaux d'événement d'écoulement dans la ou les classes d'événements d'écoulement.
PCT/EP2021/059679 2020-06-01 2021-04-14 Classification d'événements d'écoulement WO2021244799A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB2008198.0 2020-06-01
GB2008198.0A GB2595660A (en) 2020-06-01 2020-06-01 Flow event classification

Publications (1)

Publication Number Publication Date
WO2021244799A1 true WO2021244799A1 (fr) 2021-12-09

Family

ID=71526396

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/059679 WO2021244799A1 (fr) 2020-06-01 2021-04-14 Classification d'événements d'écoulement

Country Status (2)

Country Link
GB (1) GB2595660A (fr)
WO (1) WO2021244799A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115264408A (zh) * 2022-06-23 2022-11-01 大连理工大学 一种基于音频处理的管道泄漏检测方法及系统

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016110696A1 (fr) 2015-01-07 2016-07-14 Homeserve Plc Dispositif de détection de débit
US20170076563A1 (en) * 2015-09-15 2017-03-16 General Electric Company Systems and methods to provide pipeline damage alerts
US20180341859A1 (en) * 2017-05-24 2018-11-29 Southwest Research Institute Detection of Hazardous Leaks from Pipelines Using Optical Imaging and Neural Network
CN109555979A (zh) * 2018-12-10 2019-04-02 清华大学 一种供水管网漏损监测方法
US20190146478A1 (en) * 2016-05-09 2019-05-16 Strong Force Iot Portfolio 2016, Llc Methods and systems of diagnosing machine components using analog sensor data and neural network
US20190187020A1 (en) * 2017-12-15 2019-06-20 International Business Machines Corporation Event driven gas sensing and source attribution
GB2576358A (en) * 2018-08-16 2020-02-19 Centrica Plc Sensing fluid flow for estimating fluid flow state

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104654024A (zh) * 2015-02-12 2015-05-27 常州大学 一种广义回归神经网络城市燃气管道泄漏定位分析方法
CN109636790B (zh) * 2018-12-13 2021-07-27 北京理工大学 一种管路结构的识别方法及装置
CN110185939B (zh) * 2019-05-16 2021-04-02 西北工业大学 基于卷积神经网络的气体管道泄漏识别方法

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016110696A1 (fr) 2015-01-07 2016-07-14 Homeserve Plc Dispositif de détection de débit
WO2017118834A1 (fr) 2015-01-07 2017-07-13 Homeserve Plc Appareil de détection d'écoulement de fluide
US20170076563A1 (en) * 2015-09-15 2017-03-16 General Electric Company Systems and methods to provide pipeline damage alerts
US20190146478A1 (en) * 2016-05-09 2019-05-16 Strong Force Iot Portfolio 2016, Llc Methods and systems of diagnosing machine components using analog sensor data and neural network
US20180341859A1 (en) * 2017-05-24 2018-11-29 Southwest Research Institute Detection of Hazardous Leaks from Pipelines Using Optical Imaging and Neural Network
US20190187020A1 (en) * 2017-12-15 2019-06-20 International Business Machines Corporation Event driven gas sensing and source attribution
GB2576358A (en) * 2018-08-16 2020-02-19 Centrica Plc Sensing fluid flow for estimating fluid flow state
WO2020035696A1 (fr) 2018-08-16 2020-02-20 Centrica Plc Détection d'écoulement de fluide pour estimer un état d'écoulement de fluide
CN109555979A (zh) * 2018-12-10 2019-04-02 清华大学 一种供水管网漏损监测方法

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ALAM, SHAHNAWAZBANERJEE, ROHANBANDYOPADHYAY, SOMA.: "Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks", ARXIV: 1808.0441 1, 13 August 2018 (2018-08-13)
ALAM, SHAHNAWAZBANERJEE, ROHANBANDYOPADHYAY, SOMA.: "Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks", ARXIV: 1808.04411, 13 August 2018 (2018-08-13)
JUNG SYANG HPARK KSEO YSEONG W: "Monitoring Severe Slugging in Pipeline-Riser System Using Accelerometers for Application in Early Recognition", SENSORS (BASEL), vol. 19, no. 18, 2019, pages 3930

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115264408A (zh) * 2022-06-23 2022-11-01 大连理工大学 一种基于音频处理的管道泄漏检测方法及系统

Also Published As

Publication number Publication date
GB2595660A (en) 2021-12-08
GB202008198D0 (en) 2020-07-15

Similar Documents

Publication Publication Date Title
Kang et al. Novel leakage detection by ensemble CNN-SVM and graph-based localization in water distribution systems
Hu et al. Review of model-based and data-driven approaches for leak detection and location in water distribution systems
CN108922622B (zh) 一种动物健康监测方法、装置及计算机可读存储介质
US20200258057A1 (en) Repair management and execution
Lee et al. Oil and gas pipeline failure prediction system using long range ultrasonic transducers and Euclidean-Support Vector Machines classification approach
JP2021532488A (ja) データセットのための機械学習モデルの好適性の決定
GB2567850A (en) Determining operating state from complex sensor data
US11017619B2 (en) Techniques to detect vehicle anomalies based on real-time vehicle data collection and processing
US8788222B2 (en) Detection of pipeline contaminants
US20140188778A1 (en) Computer-Implemented System for Detecting Anomaly Conditions in a Fleet of Assets and Method of Using the Same
CN105637432A (zh) 识别被监控实体的异常行为
CN111650922A (zh) 一种智能家居异常检测方法和装置
US20160110478A1 (en) System and methods for quantization and featurization of time-series data
Kammoun et al. Leak detection methods in water distribution networks: a comparative survey on artificial intelligence applications
GB2541510A (en) Systems and methods for verification and anomaly detection using a mixture of hidden markov models
WO2021244799A1 (fr) Classification d'événements d'écoulement
WO2020115456A1 (fr) Procédé et système de surveillance d'un système à distance
Akano et al. An assessment of ensemble learning approaches and single-based machine learning algorithms for the characterization of undersaturated oil viscosity
Legatiuk et al. Modeling and evaluation of cyber‐physical systems in civil engineering
KR20210135958A (ko) 클라우드 환경 기반 고온부품 수명 예측 방법 및 이를 위한 클라우드 서버
WO2019128522A1 (fr) Procédé, système et dispositif de mise à jour auxiliaire
CN109274562A (zh) 一种语音指令执行的方法、装置、智能家电设备及介质
WO2020145965A1 (fr) Maintenance de dispositifs informatiques
CN113079168B (zh) 一种网络异常检测方法、装置及存储介质
US11397491B2 (en) System and method for detecting imperfections in a screen

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21719582

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21719582

Country of ref document: EP

Kind code of ref document: A1