CA3062337C - Laboratory device monitoring - Google Patents

Abstract

A method of monitoring laboratory devices, the method comprising computer executed steps, the steps comprising: receiving a plurality of results, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results, determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types, for each specific one of the test types. normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result, and triggering a control operation upon identifying a deviating result among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.

Description

Laboratory Device Monitoring FIELD AND BACKGROUND OF THE INVENTION
The present invention relates to laboratory devices and, more particularly, but not exclusively to a system and method for monitoring and controlling laboratory devices.
Every day, millions of specimens such as blood samples, urine samples, etc., are analyzed in laboratories, using laboratory devices such as Extractors (say ultrasonic homogenizers) and cyclers (say PCR machines).
An Extractor is a preparation device for patient tissue samples that turns a tissue sample of a patient into a compound that can be evaluated, say by centrifuging solids, or by doing lysis on cell walls, as known in the art.
A cycler is a sample testing device, that produces machine readable results from measurements taken during a reaction or other process that is performed on a sample - say a thermal cycler used to amplify DNA segments using PCR
(Polymerase Chain Reaction) or to perform a restriction enzyme digestion, as known in the art.
In many cases, when mechanic components of such devices malfunction, a component inside the device gives a warning to that effect, alerting the user to trouble therein. In such cases, a user will resolve the trouble by following the operating manual, by calling a support center, etc.
However, in diagnostic testing, mechanical check of laboratory device is insufficient for good quality control, particularly in as far as devices used for biological specimen testing is concerned.
To that end, many laboratories implement quality control routines.

2 For example, in many laboratories, a same control substance is measured by a specific laboratory device every day, and monitoring is performed, so as to verify that results obtained using the specific device, are stable.
Many laboratories do not even have such a routine, and rely on end users such a researcher, a medical practitioner, a laboratory technician or another worker, to notice when results obtained from a specific test include suspicious results.
Further, currently used quality control routines are usually employed for a specific combination (say a combination of a same test type, machine or machines, and laboratory technician). As a result, very often, such deviations are detected in to significant delay, letting malfunctioning devices run on many samples (say for other test types) and generate many inaccurate results, before being identified as a malfunctioning.
SUMMARY OF THE INVENTION
According to one aspect of the present invention, there is provided a method of monitoring laboratory devices, the method comprising computer executed steps, the steps comprising: receiving a plurality of results, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results, determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types, for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result, and triggering a control operation upon identifying a deviating result among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
According to a second aspect of the present invention, there is provided a non-transitory computer readable medium storing computer processor executable instructions for performing steps of monitoring laboratory devices, the steps comprising: receiving a plurality of results, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for

3 obtaining the results, determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types, for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result, and triggering a control operation upon identifying a deviating result among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
According to a third aspect of the present invention, there is provided a system for monitoring laboratory devices, the system comprising, a circuit comprising a computer processor and a computer memory storing instructions that are executable by the computer processor, for performing steps of monitoring laboratory devices, the steps comprising: receiving a plurality of results, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results, determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types, for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result, and triggering a control operation upon identifying a deviating result among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
According to another aspect of the invention, there is provided a method of monitoring laboratory devices, the method comprising computer executed steps, the steps comprising: receiving a plurality of results of tests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results; determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types; for each specific one of the test types, Date Recue/Date Received 2020-10-05 /40 \
3a normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result; and triggering a control operation upon identifying a result deviating from among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
According to a further aspect of the invention, there is provided a non-transitory computer readable medium storing computer processor executable instructions for performing steps of monitoring laboratory devices, the steps comprising:
receiving a plurality of results attests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results;
determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types; for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result;
and triggering a control operation upon identifying a result deviating from among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
According to another aspect of the invention, there is provided a system for monitoring laboratory devices, the system comprising; a circuit comprising a computer processor and a computer memory storing instructions that are executable by the computer processor, for performing steps of monitoring laboratory devices, the steps comprising: receiving a plurality of results of tests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results;
determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types; for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value Date Recue/Date Received 2021-10-01 (4"is ,4==,\
3b of the reference parameter determined for the specific test type, to yield a respective unit-less result; and triggering a control operation upon identifying a result deviating from among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting.
Implementation of the method and system of the present invention involves performing or completing certain selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several Date Recue/Date Received 2021-10-01

4 selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof.
For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is herein described, by way of example only, with reference to the accompanying drawings.
With specific reference made to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in order to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention, the description taken with the drawings making apparent to those skilled in the art how several forms of the invention may be embodied in practice.
In the drawings:
Fig. 1 is a simplified flowchart illustrating a first exemplary method of monitoring laboratory devices, according to an exemplary embodiment of the present invention.
Fig. 2A is a first one of a series of simplified graphs illustrating an exemplary implementation scenario, according to an exemplary embodiment of the present invention.
Fig. 2B is a second one of a series of simplified graphs illustrating an exemplary implementation scenario, according to an exemplary embodiment of the present invention.

5 Fig. 2C is a third one of a series of simplified graphs illustrating an exemplary implementation scenario, according to an exemplary embodiment of the present invention.
Fig. 3 is a simplified block diagram schematically illustrating a non-transitory computer readable medium storing computer executable instructions for performing steps of monitoring laboratory devices, according to an exemplary embodiment of the present invention.
Fig. 4 is a simplified block diagram schematically illustrating an exemplary system for monitoring laboratory devices, according to an exemplary embodiment of the present invention.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
The present embodiments comprise a system and method of monitoring laboratory devices.
Laboratories have employed various methods to deal with quality control aspects of using laboratory devices such Cyclers (for example, thermal cyclers for DNA processes such as PCR (Polymerase Chain Reaction) or for restriction enzyme to digestion, etc.), Extractors (for example, ultrasonic homogenizers), etc.
For example, in many laboratories, a same control substance is measured by a specific laboratory device (say thermal cycler) every day, so as to verify that the device produces stable measurement results, as described in further detail he re inabove.
Many laboratories rather rely on researchers, medical practitioners, laboratory technicians, or other personnel, to notice when results obtained for a specific test appear to be suspicious, and then, alert on a problem to the laboratory's manager. The problem may be a potentially malfunctioning device that is used for obtaining at least some of the results, a technician who repeatedly fails to operate one of the devices properly, etc., as described in further detail hereinabove.
Thus, for example, an expert whose daily work includes using a computer to plot results obtained (say using different extractors used by a laboratory that provides

6 the results) for a specific type of test, on the computer's screen, may visually detect results that deviate from other results obtained for the specific test type.
However, often, such a deviation is interpreted by the researcher as a one-off, erroneous result, and is discarded without even considering the option that a device used to obtain the discarded result malfunctions and may generate other erroneous results, even if for other test types, as described in further detail hereinbelow.
Further, currently used quality control routines are usually employed for a specific combination - say for tests of a same type, that are run using a same machine or combination of machines (say a specific extractor and a specific thermal cycler), and are monitored by a same specific technician, etc.
As a result, usually, such deviations are detected one at a time and are very often discarded without further investigation.
Only after a large number of deviations are noticed for the specific combination (say results obtained for tests performed using different cyclers, but using a same extractor, and by a same, specific technician), does one even starts to suspect that a device used to perform the tests malfunctions and generates erroneous results.
Meanwhile, until being identified as malfunctioning, the device (say extractor) may run on many other samples and cause erroneous results for other combinations -say when used for another type of test, together with another device, by another laboratory technician, etc.
Embodiments of the present embodiments recognize that such laboratory practices may result in many erroneous results, and may lead to a variety of mistakes, due to inaccurate and potentially, too late diagnosis of medical conditions (say life-threatening diseases), food contaminations, etc.. or any combination thereof.
Indeed, many current laboratories process hundreds or even thousands of samples every day. Accordingly, any delay in detection of a malfunctioning device may prove critical for many patients, consumers, environmental agencies, etc.

7 Exemplary embodiments of the present invention may help detect which one Of any) of a group of factors of a same type appears to malfunction, or perform in a way that generates erroneous results, as described in further detail hereinbelow.
Specifically, exemplary embodiments of the present invention may help detect which one (if any) of a group of laboratory devices of a specific type (say a group of PCR machines, homoganizers, etc.) malfunctions, and trigger a control operation, as described in further detail hereinabove.
According to an exemplary embodiment of the present invention, there are received three or more results, say by one of the systems described in further detail hereinbelow.
Optionally, each one of the received results pertains to a respective one of two or more devices - say to a specific one of a number of PCR machines in use by a specific laboratory or to a specific one of a number of other thermal cyclers in used by the laboratory, and to a respective type of test - say to a specific type of PCR test, etc.
The received results may include, for example, blood concentrations of a specific pathogen (say in units/nil), fluorescence readings (say in RFU
(Relative fluorescence units) taken during a specific type of PCR process, etc., as described in further detail hereinbelow.
Optionally, each on of the results is received with an attribute that identifies the specific device used for obtaining the result.
The specific device may be one of two or more devices (say PCR machines), that are used (say by a specific laboratory) to measure the results, to take measurements that the results are based on, or rather, one of two or more devices used to prepare samples for a chemical (say PCR) process used to measure the results, as described in further detail hereinbelow.
Next, there are determined two or more values for at least one reference parameter, say for an average or a standard deviation, as described in further detail hereinbelow. Each one of the values determined for the reference parameter pertains to a respective, specific test type, as described in further detail hereinbelow.

8 Thus, in a first example, each one of the received results is a quantitative result obtained by a respective, specific device through fluorescence measurement, say by a specific one of a number of PCR machines in use by a specific laboratory, for a respective type of PCR based test, as described in further detail hereinbelow.
Thus, the received results may pertain to a variety of test types, performed using different devices that are in use by the laboratory, using different technicians, etc.
In the example, each one of the results is received with an attribute (say serial number) that identifies the specific device (say a specific one of a laboratory's PCR
machines) used to obtain the result, an attribute that indicates the type of test that the result pertains to, an attribute that indicates the technician in charge of the result, etc.
Accordingly, in the first example, the received results may be divided into several groups of results, according to one or more of the attributes, say according to device and result type.
Thus, a first group of the results received in the first example, includes quantitative results obtained by a first one of the laboratory's PCR machines when running tests of a first type. A second group of the first example's results includes quantitative results obtained by a second one of the laboratory's PCR machines when running tests of the same, first type.
Further in the first example, a third group of the received results includes quantitative results obtained by the first one of the laboratory's PCR
machines when running tests of a second type, and a fourth group that includes quantitative results obtained by a third one of the laboratory's PCR machines when running tests of the same, second type.
In the first example, there is determined an average and a standard deviation for each one of the two test types of the example.
Thus, in the first example, there is calculated an average and a standard deviation over all quantitative results obtained for the first test type -i.e. over the results obtained using the laboratory's first PCR machine or second PCR
machine (i.e.
over the results included in the first group and second group). Further in the example,

9 there is calculated an average and a standard deviation over all quantitative results obtained for the second test type ¨ i.e. over the results obtained using the first or third PCR machines (i.e. over the results included in the third group and fourth group).
Then, for each specific one of the test types, there is normalized each one of the results that pertain to that specific test type, using the value of the reference parameter determined for that specific test type, to yield a respective unit-less result.
Thus, in the first example, the average calculated over the quantitative results of the tests of the first type, is subtracted from each one of the quantitative results obtained for the first test type, to yield a respective difference. Then, each one of the differences is divided by the standard deviation calculated over the results obtained for the first test type, to yield a respective, unit-less value.
Each one of the results that pertain to the first test type is thus normalized, by replacing the result with the respective unit-less value calculated by the subtraction of the average from the received result, and the division of the difference by the standard deviation, as described in further detail hereinbelow.
Further in the first example, the results that pertain to the second test type, are similarly normalized using an average and a standard deviation that are calculated over the results obtained for tests of the second type.
Then, the normalized results are monitored, whether continuously or rather 26 periodically (say once an hour), as described in further detail hereinbelow.
In one example, the monitoring includes presenting the normalized results to a user (say to a laboratory manager or senior technician) using a GUI (Graphical User Interface), say as one or more machine-specific graphs, each of which graphs presents together normalized results obtained using a same specific device, but for different test types, as described in further detail hereinbelow.
During that monitoring, a deviating result may be identified among the normalized result (say a result that appears to deviate from the other normalized results that pertain to a specific one of the PCR machines of the first example), say automatically and according to a predefined parameter, or rather by a user of the GUI.

10 When the deviating result is identified, say by a user's clicking an element of the GUI (say a point on one of the device-specific graphs of the first example) that represents or is associated with the deviating result, or rather automatically, there is automatically triggered a control operation.
Optionally, the control operation suspends use of the device that the deviating result pertains to, as described in further detail hereinbelow.
The control operation may include, but is not limited to, for example, turning off the device that the deviating result pertains to, locking a chamber of the device that the deviating result pertains to, diverting automatic loading of samples from the device that the deviating result pertains to, to another device, etc. as described in further detail hereinbelow.
Thus, with the exemplary embodiments, results obtained using different extractor-technician, cycler-extractor, or any other combinations, may be used together simultaneously and comparatively, so as to provide for a potentially earlier, and thus more efficient, detection of malfunctioning devices.
The principles and operation of a system and method according to the present invention may be better understood with reference to the drawings and accompanying description.
Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings.
The invention is capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.
Reference is now made to Fig. 1, which is a simplified flowchart illustrating a first exemplary method of monitoring laboratory devices, according to an exemplary embodiment of the present invention.

I
A first exemplary method of monitoring laboratory devices may be implemented using electric circuits, computer instructions, etc., as described in further detail hereinbelow.
Thus, in one example, the method is executed by a system that includes a circuit (say an integrated electric circuit (IC)). The circuit includes one or more computer processors, one or more computer memories (say a DRAM (Dynamic Random Access Memory) component, an SSD (Solid State Drive) component, etc.), and one or more other components, as described in further detail hereinbelow.
The computer memory stores instructions, which instructions are executable by the system's computer processor, for performing the steps of the method, as described in further detail hereinbelow.
Optionally, the system is in communication with one or more devices that are to used for performing one or more tests on samples (say biological samples that are taken from human beings or from farm animals), say with one or more extractors, cyclers, robots, or other machines, as described in further detail hereinabove.
The communication between the system and the device(s) may be wireless (say over a Wi-Fi Connection, a Bluetooth* connection, etc., or any combination thereof), wired (say a communication over a wired LAN (local Area network), etc., or any combination thereof, as described in further detail hereinbelow.
In the method, there are received 110 a plurality of results. Each one of the =
received 110 results pertains to a respective one of a plurality of test types and to a respective, specific one of a plurality of devices used for obtaining the results, as described in further detail hereinbelow.
Optionally, each one of the results is received 110 through communication between the system and a specific device used to obtain the result, say between the system and one of a number of PCR machines in use by a large laboratory that operates the system, for obtaining the results, as described in further detail hereinbelow.
The received 110 results may include, for example, blood concentrations of a specific pathogen (say in units/ml), fluorescence readings (say in RF1J
(Relative fluorescence units) taken during a specific type of a PCR process or an ELBA
test, etc., or any combination thereof.
Optionally, the results include or are calculated from, measurements taken during a test (say a test based on a chemical process) - say of an electric property of the sample undergoing the process, of an optical property (say a one indicative of hemoglobin content of the sample), etc., as known in the art.
Optionally, each on of the results is received with an attribute that identifies the specific device used for obtaining the result (say a serial number of the device, an IP (Internet Protocol) address of the device, etc.), with an attribute that identifies the specific test type that the result pertains to, etc.
Optionally, the specific device is a device (say a specific one of a large laboratory's PCR machines) used to measure the result, or to take measurements that the result is based on, as described in further detail hereinbelow.
Alternatively, the specific device is rather, a specific device (say a specific one of several homogenizers used by the large laboratory) used to prepare samples for a chemical (say PCR) process or another process, and the measurements are rather taken by another device (say by a cycler), as described in further detail hereinbelow.
Next, there are determined 120 two or more values for at least one reference parameter, say for an average or a standard deviation, say by the exemplary system described in further detail hereinbelow. Each one of the values determined 120 for the reference parameter pertains to a respective, specific one of test types.
In a first example, each one of the received 110 results is a quantitative result (say copies per milliliter) obtained by a specific device (say by a specific one of several PCR machines in use by a large laboratory), and is a result of a specific type of tests (say a specific PCR test, a specific ELISA test, etc.).
The results received 110 in the first example, may thus be divided into a number of groups.
A first group of the results received 110 in the first example, includes quantitative results obtained by a first one of a large laboratory's PCR
machines when running tests of a first type. Further, a second group of the first example's received 110 results includes quantitative results obtained by a second one of the large laboratory's PCR machines when running tests of the same, first type.
Further in the example, a third group of the received 110 results includes quantitative results obtained by the first one of the laboratory's PCR
machines when running tests of a second type. A fourth group of the example's received 110 results includes quantitative results obtained by a third one of the laboratory's PCR
machines when running tests of the same, second type.
In the first example, there is determined 120 an average and a standard deviation for each one of the two test types of the example.
Thus, in the first example, there is calculated 120 an average and a standard deviation over all quantitative results obtained for the first test type -i.e. over the results obtained using the laboratory's first PCR machine or second PCR
machine (i.e.
over the results included in the first group and second group). Further in the example, there is calculated 120 an average and a standard deviation over all quantitative results obtained for the second test type ¨ i.e. over the results obtained using the first or third PCR machines (i.e. over the results included in the third group and fourth group).
Next in the method, for each specific one of the test types, there is normalized 130 each one of the results that pertain to that specific test type. using the value of the reference parameter determined 120 for that specific test type, to yield a respective unit-less result, say by the exemplary system described in further detail hereinabove.
Thus, in the first example, the average calculated 120 over the quantitative results of the tests of the first type, is subtracted from each one of the quantitative results obtained for the first test type, to yield a respective difference.
Then, each one of the differences is divided by the standard deviation calculated 120 over the results obtained for the first test type, to yield 130 a respective, unit-less value.
Each one of the results that pertain to the first test type is thus normalized 130, by replacing the result with the respective unit-less value calculated 130 by the subtraction of the average from the received result, and the division of the difference by the standard deviation, as described in further detail hereinbelow.
Optionally, the results are rather normalized 130 using values determined 120 for other reference parameters. The other reference parameters may include, for example, parameters that are based on Westgard Rules, EWMA (Exponentially Weighted Moving Average Chart), Root Mean Square Thresholds, CUSUM
(Cumulative Sum Control Chart), etc., as described in further detail hereinbelow.
Further in the first example, the results that pertain to the second test type, are similarly normalized 130 using an average and a standard deviation that are calculated to 120 over the results obtained for tests of the second type, or rather using one or more of the other reference parameters mentioned hereinabove.
In the method, upon identifying a deviating result among a group of the normalized 130 results, the normalized 130 results of the group pertaining to a same, specific one of the devices (say one of the PCR machines of the first example), there is automatically issued 140 a control operation, say by the exemplary system described in further detail hereinbelow.
Optionally, for identifying the deviating result, the method further includes a step of monitoring, in which step, the normalized 130 results are monitored continuously or rather periodically (say once an hour), as described in further detail hereinbelow.
Optionally, the step of monitoring includes presenting the normalized results to a user (say to a laboratory manager or senior technician) using a GUI
(Graphical User Interface), as described in further detail hereinbelow.
In one example, the GUI presents the results to the user using machine-specific graphs, each of which graphs presents together normalized results obtained using a same specific device, but for different test types, as described in further detail for the exemplary implementation scenario illustrated using Fig. 2A-2C
hereinbelow.
During that monitoring, a deviating result may be identified 140 among the normalized 130 results (say a result that appears to deviate from the other normalized 130 results that pertain to a specific one of the PCR machines of the first example), say automatically and according to a predefined parameter or rather by a user of the OW.
The parameter may be predefined, for example, by a user or programmer of the system described in further detail and illustrated using Fig. 4 hereinbelow.
For example, the programmer may define that when more than one normalized 130 results of a same test type, that also pertain to a same specific device, slip at least two standard deviations (SD) away from the average determined 120 for the specific test type, on a same specific day, the results are to be identified 140 as deviating.
Optionally, when the deviating result is identified 140 - say by a user's to clicking an element of the GUI (say a point on one of the device-specific graphs of the first example) that represents, or is associated with the deviating result, or rather automatically - there is automatically triggered 140 a control operation.
Optionally, the triggered 140 control operation suspends use of the specific device that the deviating result pertains to, as described in further detail hereinbelow.
The triggered 140 control operation may include, but is not limited to, turning off the specific device that the deviating pertains to, locking a chamber of the specific device that the deviating result pertains to, instructing a robot to stop loading samples on the specific device, etc.
In one example, the control operation is triggered 140 as an instruction that is issued and sent by the system that implements the first exemplary method, to a computer (say to an industrial controller or other computer, embedded or otherwise associated with the specific device), which computer controls the specific device or a part thereof.
The instruction may be sent to the computer that controls the device (or the part thereof), for example, over an Intranet network connection and using the received 110 attribute that identifies the specific device that the deviating result pertains to, say using the device's IP (Internet Protocol) address, as known in the art.
In a second example, there is additionally or alternatively, received 110 (say as one or more of the attributes) or generated circumstantial data that pertain to a specific one of the received 110 results, or rather to a group that includes some or all or the received 110 results.
The circumstantial data may include, but is not limited, to: time of receipt or of obtaining the result(s) using the specific device, data that identifies one or more other devices used for obtaining the result, data that identifies one or more technicians responsible for the result, etc.
Then, the circumstantial data may be used for identifying the specific device that the result pertains to (when not received 110 with the result), say by querying an ERP (Enterprise Resources Planning) database, for retrieving data that identifies the to specific device used by the specific technician, etc., as known in the art.
In the second example too, the control operation is issued 140 as an instruction that is sent using the retrieved data, by the system, say to the computer (say an industrial controller) that controls the specific device (or a part thereof), as described in further detail hereinabove.
Reference is now made to Fig. 2A-2C, which is series of simplified graphs illustrating an exemplary implementation scenario, according to an exemplary embodiment of the present invention.
In the exemplary scenario, a laboratory that is responsible for performing various tests in a specific water reservoir, uses a different device (say automatic colony counter) every day, for carrying out different test using that device.
In the example, the GUI could present results received 110 for a specific test type, say a specific pathogen's concentration given in CFU/ml (Colonies Formed/
Milliliter) as measured daily in the water reservoir, using different laboratory devices (say automatic colony counters), together, as illustrated in Fig. 2A.
In the exemplary graph depicted in Fig. 2A, the result 214 obtained on the 14 day, appears to slip two standard deviations away. The standard deviation and average are calculated based on the results obtained for that specific test type, but using different devices. However, the following results (i.e. the ones obtained on the next three days) seem to indicate a return to normality.

As a result, when a technician in charge of monitoring water reservoir is presented the results using that graph shown in Fig. 2A, the technician discards the result 214 obtained on the 14th as a one-off, without even considering the option that one of the devices may be malfunctioning. The result 218 obtained for the 18' day is similarly discarded by the technician.
Similarly, the exemplary graph depicted in Fig. 2B, presents together results received 110 for another specific test type, say a total count of another pathogen as measured daily on a specific surface of interest (say a specific filter used in a pump connected to the water reservoir), by the laboratory devices (i.e. using the automatic colony counters).
When a second technician, who is in charge of the pump's maintenance, is presented the results using that graph shown in Fig. 2B, the second technician too, discards the result 224 obtained for the filter on the 14th as a one-off, without even considering the option that one of the devices may be malfunctioning. The second technician discards the filter's result 224 for the 14th day, because the following results (i.e. the ones obtained on the next three days) seem to indicate a return to normality, and similarly discards the result 228 obtained for the 18th day.
As a result, the device(s) used on the 14th and 1 8th day would keep generating erroneous results, say for different test types.
However, in the exemplary implementation scenario, the received results 110 are further normalized 130 as taught by the first exemplary method, and are then, presented to a senior user (say a laboratory manager of senior technician) using a GUI
(Graphical User interface), say in a device-specific graph, as illustrated in Fig. 2C.
Alternatively, the normalized 130 results are rather presented in a multi-test, multi-device graph, say a one that presents all normalized 130 results.
Using the graph illustrated in Fig. 2C. the GUI allows the senior user to monitor the results that are normalized 130 using the average and standard deviation determined 120 per test type, and identify one or more deviating results amongst the normalized 130 results, as described in further detail hereinabove.

Optionally, the GUI further allow the senior user to trigger an automatic issuing of an instruction that triggers a control operation, say by clicking on a point on the graph, for identifying the deviating result(s), as described in further detail hereinabove.
More specifically, as shown in Fig. 2C, at least two 234 of the normalized results appear to slip two standard deviations away on the 141h day, which draws the senior user's attention to the option that the device used on the 14'h day, may be malfunctioning.
Alternatively, in the scenario, one or more of the deviating normalized 130 results 234 of the 14'h day, is identified automatically, even without presenting the normalized 130 results in a GUI, arid the control operation (say turning off the device) is accordingly, issued automatically, as described in further detail hereinabove.
Reference is now made to Fig. 3 which is a simplified block diagram schematically illustrating a non-transitory computer readable medium storing computer executable instructions for performing steps of monitoring laboratory devices, according to an exemplary embodiment of the present invention.
According to an exemplary embodiment of the present invention, there is provided a non-transitory computer readable medium 3000.
The medium 3000 may include, but is not limited to, a Micro SD (Secure Digital) Card, a CD-ROM, a USB-Memory, a Hard Disk Drive (HDD), a Solid State Drive (SSD), a computer's ROM chip, a DRAM (Dynamic Random Access Memory) or other RAM (Random Access Memory) component, a cache memory component of a computer processor, etc., or any combination thereof, as known in the art.
The computer readable medium 3000 stores computer executable instructions, for performing steps of monitoring laboratory devices, say according to steps of the first exemplary method described in further detail hereinabove, and illustrated using Fig. 1.
The instructions may be executed on one or more computer, say by the first system, as described in further detail hereinbelow and illustrated using Fig.
1.

The computer executable instructions include a step of receiving 310 a plurality of results. Each one of the received 310 results pertains to a respective one of a plurality of test types and to a respective, specific one of a plurality of devices used for obtaining the results, as described in further detail hereinabove.
Optionally, each on of the results is received 310 through communication with a specific device used to obtain the result, say between the system and one of a number of PCR machines in use by a large laboratory that operates the system, for obtaining the results, as described in further detail hereinbelow.
The communication may be wireless (say over a Wi-Fl Connection, a Bluetooth'g' connection, etc., or any combination thereof), wired (say a communication over a wired LAN (local Area network), etc., or any combination thereof, as described in further detail hereinabove.
The received 310 results may include, for example, blood concentrations of a specific pathogen (say in units/ml), fluorescence readings (say in RFU
(Relative fluorescence units) taken during a specific type of PCR (Polymerase Chain Reaction), etc., as described in further detail hereinabove.
Optionally, the results include, or are calculated from, measurements taken during a test (say a test based on a chemical process) - say of an electric property of the sample undergoing the process. of an optical property of the sample (say a one indicative of hemoglobin content), etc., as described in further detail hereinabove.
Optionally, each on of the results is received 310 with an attribute that identifies the specific device used for obtaining the result (say a serial number of the device, an IP (Internet Protocol) address of the device, etc.). with an attribute that identifies the specific test type that the result pertains to, etc.
Optionally, the specific device is a device (say a specific one of a large laboratory's PCR machines) used to measure the result, or to take measurentents that the result is based on, as described in further detail hereinabove.
Alternatively, the specific device is rather, a specific device (say a specific one of several homogenizers used by the large laboratory) used to prepare samples for a chemical (say PCR) process or other process, and the measurements are rather taken by another device (say by a cycler), as described in further detail hereinabove.
The computer executable instructions further include a step of determining 320 two or more values for at least one reference parameter, say for an average or a standard deviation, say by the exemplary system described in further detail hereinbelow. Each one of the values determined 320 for the reference parameter pertains to a respective, specific one of test types.
Inn first example, each one of the received 310 results is a quantitative result (say copies per milliliter) obtained by a specific device (say by a specific one of several PCR machines in use by a large laboratory), and is a result of a specific type of tests (say a specific PCR test, a specific ELISA test, etc.).
The results received 310 in the first example, may thus be divided into a number of groups.
A first group of the results received 310 in the first example, includes is quantitative results obtained by a first one of a large laboratory's PCR machines when running tests of a first type. Further, a second group of the results received 310 in the first example, includes quantitative results obtained by a second one of the large laboratory's PCR machines when running tests of the same, first type.
Further in the first example, a third group of the received 310 results includes quantitative results obtained by the first one of the laboratory's PCR
machines when running tests of a second type, and a fourth group of the received 310 results includes quantitative results obtained by a third one of the laboratory's PCR machines when running tests of the same, second type.
In the first example, there is determined 320 an average and a standard deviation for each one of the two test types of the example.
Thus, in the first example, there is calculated 320 an average and a standard deviation over all quantitative results obtained for the first test type -i.e. over the results obtained using the laboratory's first PCR machine or second PCR
machine (i.e.
over the results included in the first group and second group). Further in the example, there is calculated 320 an average and a standard deviation over all quantitative results obtained for the second test type ¨ i.e. over the results obtained using the first or third PCR machines (i.e. over the results included in the third group and fourth group).
The computer executable instructions further include a step in which, for each specific one of the test types, there is normalized 330 each one of the results that pertain to that specific test type. The result is normalized 330 using the value of the reference parameter determined 320 for that specific test type, to yield a respective unit-less result, say by the exemplary system described in further detail hereinabelow.
Thus, in the first example, the average calculated 320 over the quantitative results of the tests of the first type, is subtracted from each one of the quantitative results obtained for the first test type, to yield a respective difference.
Then, each one of the differences is divided by the standard deviation calculated 320 over the results obtained for the first test type, to yield 330 a respective, unit-less value.
Each one of the results that pertain to the first test type is thus normalized 330, IS by replacing the result with the respective unit-less value calculated 330 by the subtraction of the average from the received result, and the division of the difference by the standard deviation, as described in further detail hereinbelow.
Further in the first example, the results that pertain to the second test type, are similarly normalized 330 using an average and a standard deviation that are calculated 320 over the results obtained for tests of the second type.
The computer executable instructions further include a step of automatically triggering 340 a control operation, say by the exemplary system described in further detail hereinabove.
In the step of triggering 340 the control operation. upon identifying 340 a deviating result among a group of the normalized 330 results, the normalized results of the group pertaining to a same, specific one of the devices (say to one of the PCR machines of the first example), there is automatically triggered 240 a control operation.
Optionally, for identifying the deviating 340 result, the computer executable instructions further include a step of monitoring, in which step, the normalized 330 results are monitored continuously or rather periodically (say once an hour), as described in further detail hereinabove.
Optionally, the step of monitoring includes presenting the normalized results to a user (say to a laboratory manager or senior technician) using a GUI
(Graphical User Interface, as described in further detail hereinabove.
In one example, the GUI presents the results to the user using machine-specific graphs, each of which graphs presents together normalized results obtained using a same specific device, but for different test types, as described in further detail for the exemplary implementation illustrated using Fig. 2A-2C hereinbelow.
During that monitoring, a deviating result may be identified 340 among the normalized 330 results (say a result that appears to deviate from the other normalized 330 results that pertain to a specific one of the first example's PCR
machines), say automatically, or rather by a user of the GUI, as described in further detail hereinabove.
For example, the programmer may define that when more than two normalized 330 results of a same test type, that also pertain to a same specific device, slip at least two standard deviations (SD) away from the average determined 320 for the specific test type, on a same specific day, the results are to be identified 340 as deviating.
Optionally. the deviating result may be identified by a user's clicking an element of the GUI (say a point on one of the device-specific graphs of the first example) that represents, or is associated with the deviating result.
Alternatively, or additionally, the deviating result may be identified automatically.
With the computer executable instructions, upon the deviating result's being identified 340, there is automatically triggered 340 the control operation, as described in further detail hereinabove.
Optionally, the triggered 340 control operation suspends use of the specific device that the deviating result pertains to, as described in further detail hereinabove The triggered 340 control operation may include, but is not limited to, turning off the specific device that the deviating pertains to, locking a chamber of the specific device that the deviating result pertains to, instructing a robot to stop loading samples on the specific device, etc.
In one example, the control operation is triggered 340 by issuing an instruction that is sent by the system that implements the first exemplary method, to a computer (say to an industrial controller or other computer, embedded or otherwise associated with the specific device) that controls the specific device or a part thereof.
Optionally, the instruction is sent to the computer that controls the device (or a part of the device), say over an intranet network connection and using the received 310 attribute that identifies the specific device that the deviating result pertains to, say using the device's IP (Internet Protocol) address, as known in the art.
In a second example, there is additionally or alternatively, received 310 (say as one or more of the attributes) or generated circumstantial data that pertain to a specific one of the received 310 results, or rather to a group that includes some or all or the received 310 results.
The circumstantial data may include, but is not limited, to: time of receipt or of obtaining the result(s) using the specific device, data that identifies one or more other devices used for obtaining the result, data that identifies one or more technicians responsible for the result, etc.
Then, the circumstantial data may be used for identifying the specific device that the result pertains to (when not received 310 with the result), say by querying an ERP (Enterprise Resources Planning) database, for retrieving data that identifies the specific device used by the specific technician, etc., as known in the art.
" In the second example too, the control operation is issued as an instruction that is sent using the retrieved data, by the system, say to the computer (say an industrial controller) that controls the specific device or a part thereof, as described in further detail hereinabove.
Reference is now made to Fig. 4, which is a simplified block diagram schematically illustrating an exemplary system for monitoring laboratory devices, according to an exemplary embodiment of the present invention.

A system 4000 for monitoring laboratory devices, according to an exemplary embodiment of the present invention may be implemented using electric circuits, computer software, computer hardware, etc.. or any combination thereof.
According to an exemplary embodiment, the system 4000 includes a circuit that comprises a computer processor 401 and a computer memory 402.
The computer memory 402 may include, but is not limited to: a Hard Disk Drive (HDD), a Solid State Drive (SSD), a computer's ROM chip, a DRAM
(Dynamic Random Access Memory) component or another RAM (Random Access Memory) component, a cache memory component of the computer processor 401, etc., or any combination thereof.
Optionally, the system 4000 further includes a communication card 403, that is used for communicating over a remote (say a communication over the Internet), short-ranged (say a communication over a LAN (local Area network) or a Wi-Fi Connection), or other connection, as described in further detail hereinabove.
The computer memory stores instructions that are executable by the computer processor 401, for performing the steps of the method of monitoring laboratory devices, as described in further detail and hereinabove and as Nostraled using Fig. 1.
In the exemplary embodiment, the computer processor 401 is programmed to perform the instructions, thereby implementing the system's 4000 one or more additional parts, say the parts 410-440 shown in Fig. 5, as described in further detail hereinbelow.
Each one of the additional parts may be implemented as software - say by programming the computer processor to execute the steps of the method described in further detail and illustrated using Fig. 1 hereinbelow, as hardware - say as a hardware part of the electric circuit that implements at least a part of that method, etc., any combination thereof.
The system 4000 may thus include a result receiver 410.
The result receiver 410 receives a plurality of results. Each one of the received results pertains to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results, as described in further detail for the first exemplary method, hereinabove.
The system 4000 further includes a reference determiner 420, in communication with the result receiver 410.
The reference determiner 420 determines a plurality of values for at least one reference parameter, say an average, a standard deviation. etc.. or any combination thereof. Each one of the plurality of values determined for the reference parameter, pertains to a respective one of the test types, as described in further detail for the first exemplary method, hereinabove.
The system 4000 further includes a result normalizer 430, in communication with the reference determiner 420.
For each specific one of the test types, the result normalizer 430 normalizes each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result, as described in further detail for the first exemplary method, hereinabove.
The system 4000 further includes a control instruction issuer 440, in communication with the result normalizer 430.
Upon identifying a deviating result among a group of the normalized results, which group's results pertain to a same one of the devices, the control instruction issuer 540 triggers a control operation - say a one that is used to suspend use of the device that the deviating result pertains to, as described in further detail for the first exemplary method, hereinabove.
The control operation may include, for example, turning off the device that the deviating result pertains to, locking a chamber of the device that the deviating result pertains to, diverting automatic loading of samples from the device that the deviating result pertains to, to another device, etc., as described in further detail for the first exemplary method, hereinabove.
Optionally, the system 4000 further includes a result monitor (not shown), in communication with the result normalizer 430.

The result monitor monitors the results, whether continuously or periodically.

so as to identify the deviating result.
The result monitor may identify the deviating result automatically, or rather using a Graphical User Interface (GUI) that presents the normalized results to a user, and allows the user to identify a normalized result as deviating from other normalized results that pertain to a same device, as described in further detail for the first exemplary method, hereinabove.
GENERAL FURTHER DISCUSSION
Some exemplary embodiments combine using knowledge of an originating test, sample or device (e.g. sample preparation device, testing device, tissue origin, etc.) with a method to enable any results to be compared (setting parameters to determine standard deviation per subset, which delivers a normalized result).
By grouping and comparing normalized results, it becomes possible to monitor a range of equipment and analyzers, and act accordingly with a maintenance procedure and disqualification of results. By evaluating statistical rules across multiple machines, technicians (or other users) and/or tests, it may be possible to identify the cause of variation.
Background and Terminology of the discussion below Extractor: A preparation device for patient tissue samples. A device that turns a tissue sample of a patient into a compound that can be evaluated, say by centrifuging solids, or by doing lysis on cell walls, etc.
Cycler: A sample testing device that produces machine readable output Test: A composition used for evaluating the properties of a patient sample, when tested using a testing device such as a Cycler. For example, qPCR
Influenza Test, or Toxoplasmosis Antibodies ELYSA test.
Result: Numerical Output of a testing device as a function of a specific device, and test conducted.

Comparison: Appropriate statistical tool for measuring acceptable variation over multiple results. For example, cumulative sum, Standard deviation, CUMSUM, mean.
Normalized result: A result, scaled and adjusted according to a collection of similar results, so that it can be placed in a universal scale to allow direct comparison with results of different origin.
Data Organization System: A database of users, tests and equipment combined with appropriate user/auto-generated input, per analysis, to allow grouping of results according to various properties, such as specific Extractor, Specific technician, shift of the day, type of test, etc.
Statistical rules: A family of procedures that are used to detect variation of results over time. For example, Westgard, EWMA, Root Mean Square Thresholds, CUSUM, etc., as known in the art.
Discussion An operational tests laboratory, say, in a hospital, is likely to run several sample preparation devices, several testing devices and multiple tests from various sample sources eg blood, tissue, saliva, etc Devices can add variations to quantitative and qualitative tests being done Typically these variations are detected by evaluating an expected mean value and standard deviation of the results and then tracking how many standard deviations a given result (or set of consecutive results) deviate from the mean, or a Root Mean Squared threshold.
Mean/SD are in units of quantity (e.g. copies/m1), temperature (for HRM) an equivalent (such as CT values), etc.
The causes for large variations in tests can be due to equipment failure, user failure, consumable material failure, etc.
Current QC (Quality Control) rules also work only for very specific combinations ¨ i.e. a same test, same machines. a same sample type. As a result, warnings can be late, thus bad results could slip in.

Given a system that knows which machine, user, test executed, materials batch, the system knows which machine is in use when a test is run.
By evaluating statistical rules across machines or users, as well as across tests and materials batches, it may be possible to identify the cause of variation.
It may then be possible to take an automated action of disqualifying results given to patients, and invoke maintenance actions regarding machines and materials.
One example to an automated laboratory control process may include:
1. Test orders arrive.
2. Samples are marked and ordered.
3. Tests are prioritized and samples are queued up for the extraction machines.
4. Samples are placed in Extraction Machines for plate preparation for Testing.
5. Plates are moved to Thermal Cyclers 6. Thermal Cyders run tests.
7. Results are analyzed ¨ a QC monitoring module detects a problem on Extraction Machine No. 1.
8. A STOP command is placed on Extraction Machine No. I, and all samples are diverted to Extraction Machines No. 2, and 3.
9. Support Technician reviews the issue and resolves.
10. Extraction Machine No. 1 is brought back online.
As a preparation to direct comparison between results that originate from different experiment criteria, the numerical results may be divided by the standard deviation, or by another unit size (such as the maximum value), to produce a comparable pure (i.e. unit-less) number that is indicative of the offset from mean of a single result within the group of results.

Alternatively, a relative quantity can be used, by finding the ratio between the result and a control numerical result. Statistical rules are the applied to the unit-less normalized results, in conjunction to the group being evaluated. This can present errors as they form, even across different tests and operators.
Multiple errors for a sample preparation device, or multiple errors for a cycler, or multiple errors for a specific batch, can be indicative of a system error happening for that component of a laboratory.
Being unit-less, it is possible to plot together group behavior over time, for multiple groups simultaneously. The X axis can be time, and the Y axis can be the unit-less, normalized value, so that development of malfunctions can be followed.
In one example, the process may include, for example: user-specified mean (M) and standard deviation (SD) or preset calculation parameters, user-specified sort order (run date, input date, manual date, etc.), user-specified test result grouping (e.g.
by extraction device, pipetting instrument, processing device, control type, control volume, test being run and/or any combination or alternative thereof), Run control measurements for a run being verified (X), and/or associated metadata required for grouping.
Then, the process is run on received test results, through the following steps:
I. A Mean (M) and a Standard Deviation (SD) is calculated, based on past n results or on all past results for test parameters, as configured by user (or rather, pre-set values as stored on a computer readable medium, are used as values of reference parameters, instead of M and SD values).
2. Each one (X) of the results is normalized into a unit-less value (rSD), say using the formula rSD = (X-M) / SD.
3. Optionally, historic and new results are then plotted together on control chart (e.g. Shewhart, Levey-Jennings or any other chart) as n vs AO where n is order of use, as specified by user.
4. Optionally, a user-selected automated control validation rules such as Westgard, RMSD (Root Mean Standard Deviation), are applied to the normalized results, and optionally, the user is alerted when selected rules are triggered and/or a control operation is triggered, as described in further detail hereinabove.
It is expected that during the life of this patent many relevant devices and systems will be developed and the scope of the terms herein, particularly of the terms "Laboratory", "Device", "Extractor", "Cycler", "PCR Machine", "Homogenizer", "Compute r". "Computer Processor", "Circuit", "Micro SD", ='Card", "CD-ROM", "USS-Memory", "Hard Disk Drive (HOD)", "Solid State Drive (SSD)", "ROM" "ROM
chip", "Cache Memory", and "DRAM (Dynamic Random Access Memory)", is intended to include all such new technologies a priori.
It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination.
Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.
Accordingly, it is intended to embrace all such alternatives , modifications and variations that fall within the spirit and broad scope of the appended claims.
Citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.

Claims (22)

31

1. A method of monitoring laboratory devices, the method comprising computer executed steps, the steps comprising:
receiving a plurality of results of tests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results;
determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types;
for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result; and triggering a control operation upon identifying a result deviating from among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.

2. The method of claim 1, wherein the control operation is used to suspend use of the device that the deviating result pertains to.

3. The method of claim 1, further comprising presenting the normalized results to a user, using a Graphical User Interface (GUI).

4. The method of claim 1, wherein one of the at least one reference parameter is an average.

5. The method of claim 1, wherein one of the at least one reference parameter is a standard deviation.

6. The method of claim 1, wherein the control operation comprises turning off the device that the deviating result pertains to.

7. The method of claim 1, wherein the control operation comprises locking a chamber of the device that the deviating result pertains to.

8. A non-transitory computer readable medium storing computer processor executable instructions for performing steps of monitoring laboratory devices, the steps comprising:
Date Recue/Date Received 2021-10-01 receiving a plurality of results of tests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results;
determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types;
for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result; and triggering a control operation upon identifying a result deviating from arnong a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.

9. The computer readable medium of claim 8, wherein the control operation is used to suspend use of the device that the deviating result pertains to.

10. The computer readable medium of claim 8, wherein the steps further comprise presenting the normalized results to a user, using a Graphical User Interface (GUI).

11. The computer readable medium of claim 8, wherein the control operation comprises turning off the device that the deviating result pertains to.

12. The computer readable medium of claim 8, wherein the control operation comprises locking a chamber of the device that the deviating result pertains to.

13. A system for rnonitoring laboratory devices, the system comprising;
a circuit comprising a computer processor and a computer memory storing instructions that are executable by the computer processor, for performing steps of monitoring laboratory devices, the steps comprising:
receiving a plurality of results of tests, each one of the tests performed on at least one sample, each result pertaining to a respective one of a plurality of test types and to a respective one of a plurality of devices used for obtaining the results;
determining a plurality of values for at least one reference parameter, each one of the plurality of values determined for the reference parameter, pertaining to a respective one of the test types;
Date Recue/Date Received 2021-10-01 for each specific one of the test types, normalizing each one of the results pertaining to the specific test type, using the value of the reference parameter determined for the specific test type, to yield a respective unit-less result; and triggering a control operation upon identifying a result deviating from among a group of the normalized results, the normalized results of the group pertaining to a same one of the devices.

14. The system claim 13, wherein the control operation is used to suspend use of the device that the deviating result pertains to.

15. The system of claim 13, wherein the steps further comprise presenting the normalized results to a user, using a Graphical User Interface (GUI).

16. The system of claim 13, wherein one of the at least one reference parameter is an average.

17. The system of claim 13, wherein one of the at least one reference parameter is a standard deviation.

18. The system of claim 13, wherein the control operation comprises turning off the device that the deviating result pertains to.

19. The system of claim 13, wherein the control operation comprises locking a chamber of the device that the deviating result pertains to.

20. The system of Claim 13, wherein the control operation comprises diverting automatic loading of samples from the device that the deviating result pertains to.

21. The non-transitory computer readable medium of Claim 8, wherein the control operation comprises diverting automatic loading of samples from the device that the deviating result pertains to.

22. The method of Claim 1, wherein the control operation comprises diverting automatic loading of samples from the device that the deviating result pertains to.
Date Recue/Date Received 2021-10-01