WO2022048859A1

WO2022048859A1 - A concept for adapting a machine-learning model for determining potential regions of interest

Info

Publication number: WO2022048859A1
Application number: PCT/EP2021/071987
Authority: WO
Inventors: Mate BELJAN
Original assignee: Leica Microsystems Cms Gmbh
Priority date: 2020-09-02
Filing date: 2021-08-06
Publication date: 2022-03-10
Also published as: EP4208816A1

Abstract

Examples relate to a system for a microscope, a microscope system, a system for a mobile device, and to corresponding methods and computer programs. A system (110) for a microscope (120) is configured to obtain image data (140) from an optical imaging sensor of the microscope. The system is configured to determine, using a machine-learning model, a plurality of potential regions of interest (150) in the image data. The system is configured to provide review images of the potential regions of interest to a mobile device (300) of a user of the microscope. The system is configured to obtain feedback on the review images of the potential regions of interest from the mobile device of the user. The system is configured to adapt the machine-learning model based on the feedback.

Description

uiviazuu / u irAun rc i

A Concept for Adapting a Machine-Learning Model for Determining Potential Regions of Interest

Technical field

Examples relate to a system for a microscope, a microscope system, a system for a mobile device, a mobile device, and to corresponding methods and computer programs.

Background

A major use of microscopes lies in the analysis of samples, e.g. of samples comprising organic cells. These cells may be analyzed individually by users of the microscope, e.g. in order to detect cells with anomalies that are of interest for the research.

In some cases, machine-learning may be used to determine regions of interest (ROIs) within images taken by microscopes. However, these regions of interest are often identified using a generic machine-learning model that is not adapted to the current sample being analyzed, or to the user performing the analysis.

Summary

Various examples of the present disclosure are based on the finding that individual users of a microscope often use the microscope in long-time projects, in which various samples are being analyzed over a long time-span. However, the machine-learning models that are available for the detection of regions of interest are generic, and thus not adapted to the individual requirements of the specific user. As the same user performs the same type of analysis multiple times over a long time, it is feasible to perform a user-specific adaptation of the machinelearning model, based on the specific requirements of the user. To improve efficiency of the time available directly with the microscope system, a feedback mechanism is provided that uses a mobile device, such as a smartphone, of the user, as a feedback device, in order to provide feedback on potential regions of interest suggested by a machine-learning model. The feedback may be obtained via a user-friendly touch-based swiping mechanism, and may support various categories of regions of interest. This feedback is then used to adapt the machine-learning model, and thus, over time, generate better recommendations regarding potential regions of interest.

Various aspects of the present disclosure provide a system for a microscope. The system comprises one or more processors and one or more storage devices. The system is configured to obtain image data from an optical imaging sensor of the microscope. The system is configured to determine, using a machine-learning model, a plurality of potential regions of interest in the image data. The system is configured to provide review images of the potential regions of interest to a mobile device of a user of the microscope. The system is configured to obtain feedback on the review images of the potential regions of interest from the mobile device of the user. The system is configured to adapt the machine-learning model based on the feedback. By providing the review images to the mobile device, they can be reviewed away from the microscope, e.g. while the user is commuting, thus increasing the efficiency of the time spent with the microscope. By using the generated feedback to adapt the machine-learning model, the machine-learning model may be tailored to the specific use of the microscope by the user.

In general, the feedback may be used as training data to train the machine-learning model. For example, the feedback may be used to determine, which of the potential (i.e. proposed) regions of interest are of actual relevance for the user, or to which categories of interest the respective potential regions of interest belong (if they are of interest).

In some examples, the system is configured to use reinforcement learning to adapt the machine-learning model based on the feedback. For example, the feedback may be used to determine a reward in the reinforcement learning-based adaptation of the machine-learning model. Accordingly, the machine-learning model may be reinforced regarding potential regions of interest that are of actual interest to the user using reinforcement learning, and discouraged from other potential regions of interest that are not of interest to the user.

Alternatively (or additionally), the system may be configured to use supervised learning to adapt the machine-learning model based on the feedback. In this case, the feedback may be used as desired output value, with the potential regions of interest being used as training input values for the supervised learning-based training of the machine-learning model. In various examples, the machine-learning model is trained to generate bounding boxes around the potential regions of interest within the image data. Accordingly, the (entire) image data may be provided as input to the machine-learning model, and a location of the potential regions of interest may be provided as output of the machine-learning model. The generation of bounding boxes around regions of interest in image data is a major application of machinelearning in image analysis. For example, the review images may be based on the bounding boxes generated by the machine-learning model. In other words, the bounding boxes may be used to generate the review images, e.g. by including the content of the bounding boxes, and, optionally, a portion of the image data surrounding the bounding boxes (i.e. a padding region) within the review images.

In some cases, the user may be interested in different categories of cells, or in different categories of anomalies within the image data. In other words, the machine-learning model may be trained to determine the plurality of potential regions of interest in at least one of one or more categories of interest for the user. These categories of cells / anomalies may be handled separately, so the results of the machine-learning model can be retroactively filtered according to the categories, e.g. in case only a subset of categories are presently of interest, or in case the same machine-learning model is provided to another user, who might only be interested in a subset of the categories.

In some cases, an approach may be chosen that uses a generic machine-learning model for identifying candidates for the potential regions of interest, and with a specialized machinelearning model for determining, whether the identified potential regions of interest are likely of interest to the user (or to which category of interest they belong). In other words, the machine-learning model may comprise a first machine-learning model that is trained to generate bounding boxes around potential regions of interest within the image data and a second machine-learning model that is trained to estimate an interest of the user, in at least one of one or more categories of interest for the user, of the portion of the image data visible within the bounding boxes. The system may be configured to adapt the second machine-learning model based on the feedback on the user. For example, the first machine-learning model may remain as is (as it is generically used across users), and the second machine-learning model may be tailored to the needs of the user. Various aspects of the present disclosure provide a microscope system comprising a microscope and the above-referenced system. The system is coupled to the microscope within the microscope system. In general, the term “microscope system” may encompass various implementations. For example, the system may be co-located with the microscope. In other words, the microscope may be locally coupled with the system, enabling a local adaptation of the machine-learning model, without requiring transmission of large amounts of image data to a remote server. Alternatively, the system may be implemented by a remote server that is remotely coupled with the microscope via a network. A remote server may be continuously improved, e.g. a by a manufacturer of the microscope, and may thus provide a machine-learning model that is continuously improved based on feedback from a multitude of users.

Various aspects of the present disclosure provide a system for a mobile device. The system comprises one or more processors and one or more storage devices. The system is configured to receive review images of potential regions of interest in image data of a microscope from a microscope system, e.g. from a system component of the microscope system that is coupled with the microscope. The system is configured to provide the review images of the potential regions of interest to a user of the mobile device via a user interface of the mobile device. The system is configured to obtain feedback on the review images of the potential regions of interest from the user via the user interface. The system is configured to provide the feedback to the microscope system. Various aspects of the present disclosure further provide a mobile device comprising the system. The system for the mobile device is the counterpart to the above-reference system, and is used to obtain the feedback directly from the user.

As has been mentioned above, the feedback may be provided as training data for adapting a training of a machine-learning model being used to determine the potential regions of interest within the image data. For example, the feedback may be used to calculate a reward in the reinforcement learning-based adaptation of the machine-learning model, or as a desired output in a supervised learning-based adaptation of the machine-learning mode.

In some examples, the feedback may be obtained using gesture detection for detecting a swiping motion of the user in the user interface. A swiping-based interface may enable an intuitive evaluation of the potential regions of interest. Various aspects of the present disclosure further provide a system comprising the above-referenced microscope system and the mobile device.

Various aspects of the present disclosure further provide corresponding methods. For example, various aspects of the present disclosure further provide a method for a system for a microscope. The method comprises receiving image data from an optical imaging sensor of the microscope. The method comprises determining, using a machine-learning model, a plurality of potential regions of interest in the image data. The method comprises providing review images of the potential regions of interest to a mobile device of a user of the microscope. The method comprises receiving feedback on the review images of the potential regions of interest from the mobile device of the user. The method comprises adapting the machinelearning model based on the feedback.

Various aspects of the present disclosure further provide a method method for a mobile device. The method comprises receiving review images of potential regions of interest in image data of a microscope from a microscope system. The method comprises providing the review images of the potential regions of interest to a user of the mobile device via a user interface of the mobile device. The method comprises obtaining feedback on the review images of the potential regions of interest from the user via the user interface. The method comprises providing the feedback to the microscope system.

Various aspects of the present disclosure further provide a computer program with a program code for performing one of the above methods, when the computer program is executed on a processor.

Short description of the Figures

Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

Figs, la to 1c show block diagrams of examples of a system for a microscope and of a microscope system; Fig. Id shows a schematic diagram of image data of a microscope with corresponding potential regions of interest;

Fig. le shows a schematic diagram of a review image with a corresponding potential region of interest;

Fig. 2 shows a flow chart of an example of a method for a system for a microscope;

Fig. 3a shows a block diagram of an example of a system for a mobile device, and of a mobile device;

Fig. 3b shows a schematic diagram of an example of a swiping-based feedback mechanism;

Fig. 4 shows a schematic diagram of an example of a method for a mobile device; and

Fig. 5 shows a schematic diagram of a system comprising a microscope and a computer system.

Detailed Description

Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and/or regions may be exaggerated for clarity.

Various aspects of the present disclosure relate to a method (workflow) for quick selec- tion/evaluation of correct/interesting ROIs (Regions of Interest), in order to improve ROI detection (e.g. to find cells of interest faster).

The topic of detection of regions of interest is of relevance in the analysis of cells within a sample comprising different types of cells, which may be of interest to different users of a microscope, e.g. in a research setup where different users are focused on different types of cells. A user of a microscope may be relieved of some difficulty in the search for "his" cells within the sample. Tedious manual searching may be eliminated as far as possible. On the other hand, individual users often look at many samples with cells of one type. Here, the proposed concept may be used to adapt the ROI detection to the respective cell type. The microscope system may be configured to identify "interesting" areas (using image-based features) and suggest them to the user (as recommendations via Machine/Deep Learning).

The proposed concept is based on the idea, that through user feedback (classification), suggestions regarding the detection of ROIs become better and better and reflect the individual samples better. The feedback may be as easy and comfortable as possible for the user. Of course, this can also happen away from a "normal" Personal Computer (PC).

In the field of entertainment or social media, suggestions (recommendations) are often used. The user specifies his preferences and gets customized suggestions in the future. With the addition of feedback, these recommendations may become better.

In the field of microscopy, the work with microscopy data usually still takes place on the connected PC. Various aspects of the proposed concept are based on the finding, that the feedback required for improving the detection of ROIs can also happen physically detached via a mobile device (e.g. a "smartphone"). This can happen on the way to work or even at home. In this way, the resource “laboratory time” can be better used.

Figs, la to 1c show block diagrams of examples of a system 110 for a microscope 120 and of a microscope system 100. The system 110 comprises one or more processors 114 and one or more storage devices 116. Optionally, the system further comprises an interface 112. The one or more processors 114 are coupled to the optional interface 112 and the one or more storage devices 116. In general, the functionality of the system 110 is provided by the one or more processors 114, e.g. in conjunction with the optional interface 112 and/or the one or more storage devices 116.

The system is configured to obtain image data 140 from an optical imaging sensor of the microscope (e.g. via the interface 112). The system is configured to determine, using a machine-learning model, a plurality of potential regions of interest 150 in the image data. The system is configured to provide review images of the potential regions of interest to a mobile device 300 of a user of the microscope (e.g. via the interface 112). The system is configured to obtain feedback on the review images of the potential regions of interest from the mobile device of the user (e.g. via the interface 112). For example, the feedback may indicate whether the potential regions of interest are of interest to the user. The system is configured to adapt the machine-learning model based on the feedback.

Various aspects of the present disclosure relate to a system for a microscope, and to a corresponding microscope system comprising both the microscope and the above-referenced system, with the system being coupled to the microscope 120. In this context, the term “microscope system” may take one of multiple meanings. In some examples, the microscope system is a microscope system that forms a local unit of the microscope and the corresponding system. The system may be locally coupled to the microscope. In other words, the system may be arranged at the microscope, i.e. co-located with the microscope 120. Such an arrangement is shown, for example, in Fig. lb, where the system 110 is co-located with the microscope 120 and forms a cohesive unit with the microscope 120. In such setups, both the functionality of the system, as introduced in connection with Figs, la to le, and additional functionality, such as a control of one or more operational parameters of the microscope, may be performed by the system 110. Alternatively, a setup may be chosen where the system is a remote system, e.g. a server that is operated by a manufacturer of the microscope system. In this case, the microscope system may comprise a further system 130 that is locally coupled with the microscope 120, and the system 110 may be a remote system, e.g. a remote server. In other words, the system 110 may be implemented by a remote server that is remotely coupled with the microscope 120 via a network. The further system 130 may act as link between the microscope 120 and the system 110, e.g. by providing a remote connection to the system 110. In other words, the system may be located remotely from the microscope, e.g. in another room, in another building, or another city. In this setup, the further system 130 may provide the additional functionality, such as a control of one or more operational parameters of the microscope.

Accordingly, the interface 112 may serve different purposes, depending on which setup is chosen, and provide at least some of following connectivity. In general, for the communication between the mobile device and the system, a connection via a computer network may be used, e.g. via the internet. If the server is locally coupled with the microscope, a local connection might also be used, e.g. via a wireless local area network, via Bluetooth, or via a wired connection (e.g. via the Universal Serial Bus). Similarly, if the system is locally coupled with the microscope, a proprietary wired connection may be used to connect the system and the microscope. For example, in this case, the system may be tightly integrated with the microscope, with multiple connections between the system and the microscope. If the system is implemented by a remote server, a connection via a computer network may be used, e.g. via the internet. For example, the connection may be implemented between the system 110 and the further system 130. For example, the further system 130 may provide the local interface to the microscope 120 for the system 120. The interface 112 may provide the corresponding functionality for implementing the above-referenced connections.

The system is configured to obtain the image data 140 from the optical imaging sensor of the microscope. In general, a microscope is an optical instrument that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope may provide an optical magnification of an object. In modern microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensor of the microscope 120 of Figs, la to 1c. The microscope 120 may further comprise one or more optical magnification components that are used to magnify a view on the sample.

There are a variety of different types of microscopes. If the microscope system is used in the medical or biological fields, the object being viewed through the microscope may be a sample of organic tissue, e.g. arranged within a petri dish or present in a part of a body of a patient. For example, the microscope system 100 may be a microscope system for use in a laboratory, e.g. a microscope that may be used to examine the sample of organic tissue in a petri dish. Alternatively, the microscope 120 may be part of a surgical microscope system 100, e.g. a microscope to be used during a surgical procedure. Although examples are described in connection with a microscope system, they may also be applied, in a more general manner, to any optical device. For example, the microscope system may be a system for performing material testing or integrity testing of materials, e.g. of metals or composite materials.

For example, the optical imaging sensor of the microscope may comprise or be an APS (Active Pixel Sensor) - or a CCD (Charge-Coupled-Device)-based imaging sensor. For example, in APS-based imaging sensors, light is recorded at each pixel using a photodetector and an active amplifier of the pixel. APS-based imaging sensors are often based on CMOS (Complementary Metal-Oxide-Semiconductor) or S-CMOS (Scientific CMOS) technology. In CCDbased imaging sensors, incoming photons are converted into electron charges at a semicon- ductor-oxide interface, which are subsequently moved between capacitive bins in the imaging sensor modules by a control circuitry of the sensor imaging module to perform the imaging. The image data may be obtained by receiving the image data from the optical imaging sensor (e.g. via the interface 112 and/or the system 130), by reading the image data out from a memory of the imaging sensor (e.g. via the interface 112), or by reading the image data from a storage device 116 of the system 110, e.g. after the image data has been written to the storage device 116 by the optical imaging sensor or by another system or processor.

The system is configured to determine, using a machine-learning model, the plurality of potential regions of interest 150 in the image data. Thus, embodiments may be based on using a machine-learning model or machine-learning algorithm. Machine learning may refer to algorithms and statistical models that computer systems may use to perform a specific task without using explicit instructions, instead relying on models and inference. For example, in machine-learning, instead of a rule-based transformation of data, a transformation of data may be used, that is inferred from an analysis of historical and/or training data. For example, the content of images may be analyzed using a machine-learning model or using a machinelearning algorithm. In order for the machine-learning model to analyze the content of an image, the machine-learning model may be trained using training images as input and training content information as output. By training the machine-learning model with a large number of training images and/or training sequences (e.g. words or sentences) and associated training content information (e.g. labels or annotations), the machine-learning model "learns" to recognize the content of the images, so the content of images that are not included in the training data can be recognized using the machine-learning model. The same principle may be used for other kinds of sensor data as well: By training a machine-learning model using training sensor data and a desired output, the machine-learning model "learns" a transformation between the sensor data and the output, which can be used to provide an output based on non-training sensor data provided to the machine-learning model. The provided data (e.g. sensor data, meta data and/or image data) may be preprocessed to obtain a feature vector, which is used as input to the machine-learning model.

In the context of the present disclosure, the machine-learning model is used to determine the plurality of potential regions of interest 150 in the image data. In general, the determination of the potential regions of interest may be performed using machine-learning techniques such as semantic segmentation (where different portions of image data are assigned to a category) or object recognition (where an object, such as a potential region of interest, is detected within image data). In this context, the term “object” may be also used with reference to a cell type or pattern to be detected within the image data.

The functionality of machine-learning models is determined by their training. In other words, machine-learning models are trained to provide a certain functionality, such as the determination of the plurality of potential regions of interest. Machine-learning models may be trained using training input data. The examples specified above in the context of image recogniation often use a training method called "supervised learning". In supervised learning, the machine-learning model is trained using a plurality of training samples, wherein each sample may comprise a plurality of input data values, and a plurality of desired output values, i.e. each training sample is associated with a desired output value. By specifying both training samples and desired output values, the machine-learning model "learns" which output value to provide based on an input sample that is similar to the samples provided during the training. Apart from supervised learning, semi-supervised learning may be used. In semi-supervised learning, some of the training samples lack a corresponding desired output value. Supervised learning may be based on a supervised learning algorithm (e.g. a classification algorithm, a regression algorithm or a similarity learning algorithm. Classification algorithms may be used when the outputs are restricted to a limited set of values (categorical variables), i.e. the input is classified to one of the limited set of values. Regression algorithms may be used when the outputs may have any numerical value (within a range). Similarity learning algorithms may be similar to both classification and regression algorithms but are based on learning from examples using a similarity function that measures how similar or related two objects are. Apart from supervised or semi-supervised learning, unsupervised learning may be used to train the machine-learning model. In unsupervised learning, (only) input data might be supplied and an unsupervised learning algorithm may be used to find structure in the input data (e.g. by grouping or clustering the input data, finding commonalities in the data). Clustering is the assignment of input data comprising a plurality of input values into subsets (clusters) so that input values within the same cluster are similar according to one or more (pre-defined) similarity criteria, while being dissimilar to input values that are included in other clusters.

Reinforcement learning is a third group of machine-learning algorithms. In other words, reinforcement learning may be used to train the machine-learning model. In reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Based on the taken actions, a reward is calculated. Reinforcement learning is based on training the one or more software agents to choose the actions such, that the cumulative reward is increased, leading to software agents that become better at the task they are given (as evidenced by increasing rewards).

In the context of the present disclosure, the machine-learning model is trained to determine the plurality of potential regions of interest within the image data. In general, supervised learning and/or reinforcement learning may be used to perform this training. For example, to train the machine-learning model using supervised learning, training image data may be provided as training input to the machine-learning model, and a corresponding map of potential regions of interest may be provided as desired output data for the supervised learning-based training of the machine-learning model. To train the machine-learning model using reinforcement learning, the same training image data may be used as input, and the corresponding map of potential regions of interest may be used to construct the reward function of the reinforcement learning-based training. For example, the reward function may be chosen such, that the reward is increased if a region specified by the machine-learning model matches the respective regions of the map of potential regions of interest, and that the reward is decreased if a region specified by the machine-learning model lies outside the respective regions of the map of potential regions of interest.

In general, the machine-learning model may be trained to provide the potential regions of interest as so-called bounding boxes. Bounding boxes are sets of coordinates that specify an extent of the potential region of interest, in terms of a rectangular shape. In other words, the machine-learning model may be trained to generate bounding boxes around the potential regions of interest within the image data. Fig. Id shows a schematic diagram of image data 140 of a microscope with corresponding potential regions of interest. In Fig. Id, the potential regions of interest are highlighted with bounding boxes 150. In general, the image data may be provided as input to the machine-learning model and a location of the potential regions of interest may be provided as output of the machine-learning model. In other words, the machine-learning model may be configured to determine the location of the potential regions of interest in response to the image data being provided at the input of the machine-learning model. Accordingly, if the machine-learning model provides a location at its output, this location may be deemed to relate to a potential region of interest. In general, the term “region of interest” or “potential region of interest” may take different meanings. In the context of the present disclosure, however, the “potential regions of interest” relate to the interest of a specific user (or group of users) of the microscope, i.e. the user to which the review images are being provided. In other words, the determined potential regions of interest are determined based on the interest of the specific user (or groups of users). Accordingly, the machine-learning model, or at least a component of the machine-learning model, is trained to determine the potential regions on interest based on the specific interest of the user (or groups of users). In the field of microscopy of an organic sample, a region of interest may be a certain cell type, an anomaly within an organic sample, pathologic cells etc. In the field of material testing or integrity testing, a region of interest may be a portion of the workpiece to be tested with an anomalous thickness or dimension, a broken weld seam or weld joint etc.

In some examples, the machine-learning model may be/comprise in fact two machine-learning models - one for determining candidates for the plurality of regions of interest, and one for determining, whether the candidates might, in fact, be of interest to the user. In other words, the machine-learning model may comprise a first machine-learning model that is trained to generate bounding boxes around potential regions of interest within the image data (i.e. around candidates for potential regions of interest). The machine-learning model may comprise a second machine-learning model that is trained to estimate an interest of the user, in at least one of one or more categories of interest for the user, of the portion of the image data visible within the bounding boxes. Accordingly, for the first machine-learning model, a generic machine-learning model may be used to determine candidates for the potential regions of interest. The second one may be tailored to the user, and thus adapted based on the feedback of the user. These machine-learning models may be used sequentially - first the first machinelearning model to determine the candidates, and then the second machine-learning model to determine the potential regions of interest among the candidates.

In various examples, the potential regions of interest may relate to, and/or be limited to, one or more categories of interest for the user. For example, the proposed concept may be used in large research scenarios, where different users analyze different types of cells of a large sample being represented by the image data. For example, the sample may comprise cell types A through E, and the user (of the mobile device) might only be interested in cell type A or in cell types A and B. Accordingly, the potential regions of interest may relate to cell type A, or to cell type A and B. Candidates for potential regions of interest that relate to other cell types, e.g. C through E, might not be included in the potential regions of interest, or, if they are included, the feedback may indicate, that these cell types might not be of interest to the user. Accordingly, the machine-learning model may be trained to determine the plurality of potential regions of interest in at least one of one or more categories of interest for the user. Accordingly, the second machine-learning model may be trained to assign the potential regions of interest to the one or more categories of interest (if they belong into one of the categories, i.e. if they are of interest to the user).

The system is configured to provide the review images of the potential regions of interest to the mobile device 300 of the user of the microscope. In some examples, the review images may be cropped versions of the image data, e.g. cropped version of the image data that show at least the potential regions of interest. For example, each review image may represent a single region of interest or a group of regions of interest having a pre-defined maximal area. The review images may be provided separately to the mobile device (e.g., as separate file or image stream) and/or received separately by the mobile device. For example, portions of the image data outside the regions of interest (and, optionally, a padding region around the regions of interest) might not be provided to the mobile device. Alternatively, the review images may be processed versions of the image data, e.g. of portions of the image data showing the potential regions of interest. In some examples, the potential regions of interest may be scanned at a higher zoom rate than the image data. In this case, the system may be configured to request and receive higher-resolution image data (with an image resolution per area that is higher than the image resolution per area of the image data) of the potential regions of interest from the optical imaging sensor of the microscope. Accordingly, the review images may be based on the higher-resolution image data of the potential regions of interest. For example, the review images may represent the potential regions of interest in multiple zoom levels.

In general, the review images may be based on the bounding boxes generated by the machinelearning model. Accordingly, the review images may show at least the potential regions of interest (as delimited by the bounding boxes). Additionally, the review images may comprise portions of the image data located adjacent to the potential regions of interest. For example, a padding region surrounding the potential regions of interest, e.g. surrounding the respective bounding boxes may be added. Accordingly, on the mobile device, the review images may be scrollable/moveable/zoomable. Fig. le shows a schematic diagram of a review image with a corresponding potential region of interest. In Fig. le, the bounding box 150 demarking the potential region of interest is shown, surrounding the bounding box 150, another box 160 is shown, which comprises a padding region around the bounding box 150.

The system is configured to obtain feedback on the review images of the potential regions of interest from the mobile device of the user. In general, two types of feedback may be used - one type of feedback that indicates whether a potential region of interest is of interest to the user, and another type of feedback that indicates whether a potential region of interest belongs into one of the one or more categories of interest. In other words, the feedback may indicate whether (and which of) the potential regions of interest are of interest to the user, e.g. using a binary feedback mechanism. Additionally or alternatively, the feedback may indicate, for each potential region of interest, whether the potential region of interest belongs into one of the one or more categories of interest. If a potential region of interest belongs into one of the one or more categories of interest, it may be deemed to be of interest to the user.

The system is configured to adapt the machine-learning model based on the feedback. In this context, the term “adapting the machine-learning model” indicates that the machine-learning model is changed, based on the feedback, to reflect the feedback given by the user. For example, the machine-learning model may be re-trained based on the feedback. For example, the machine-learning model may be an artificial neural network (ANN). ANNs are systems that are inspired by biological neural networks, such as can be found in a retina or a brain. ANNs comprise a plurality of interconnected nodes and a plurality of connections, so-called edges, between the nodes. There are usually three types of nodes, input nodes that receiving input values, hidden nodes that are (only) connected to other nodes, and output nodes that provide output values. Each node may represent an artificial neuron. Each edge may transmit information, from one node to another. The output of a node may be defined as a (non-linear) function of its inputs (e.g. of the sum of its inputs). The inputs of a node may be used in the function based on a "weight" of the edge or of the node that provides the input. The weight of nodes and/or of edges may be adjusted in the learning process. In other words, the training of an artificial neural network may comprise adjusting the weights of the nodes and/or edges of the artificial neural network, i.e. to achieve a desired output for a given input. In the present context, the machine-learning model is adapted after its initial training. For example, the weights of the egdes of the ANN may be adapted based on the feedback. As has been mentioned above, two types of machine-learning that can be used to train, and adapt, the machine-learning model - supervised learning and reinforcement learning. In both cases, the feedback may be used as (part of) the training data for re-training (i.e. adapting) the machine-learning model. In other words, the feedback may be used as training data to train the machine-learning model.

For example, the system may be configured to use reinforcement learning to adapt the machine-learning model based on the feedback. In other words, the system may be configured to re-train the machine-learning model using reinforcement learning. As has been outlined above, in reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Based on the taken actions, a reward is calculated. Reinforcement learning is based on training the one or more software agents to choose the actions such, that the cumulative reward is increased, leading to software agents that become better at the task they are given (as evidenced by increasing rewards). In the context of the present disclosure, the software agents may be trained to identify the plurality of potential regions of interest. If the software agents succeed in identifying the potential regions of interest, the cumulative reward may be increased, if they do not, the cumulative reward may be decreased. To calculate the reward, the feedback may be used, as the feedback indicates, which of the potential regions of interest are of actual interest to the user, or to which category of interest they belong. In other words, the feedback may be used to determine a reward in the reinforcement learning-based adaptation of the machine-learning model. For example, the feedback, e.g. the interest of the user in the potential regions of interest, or an assignment performed by the user between the potential regions of interest and the one or more categories of interest, may be compared with the actions of the software agents. As far as the actions of the software agents correspond with the feedback, the cumulative reward may be increased, and, as far as the actions of the software agents conflict with the feedback, the reward may be decreased. Over multiple iterations, e.g. based on two or more samples of image data and corresponding potential regions of interest and feedback, the machine-learning model may be adapted to the interest of the user, or to the user-selected assignment between potential regions of interest and categories of interest.

Alternatively, the system may be configured to use supervised learning to adapt the machinelearning model based on the feedback. In this case, image data, or the individual potential regions of interest, may be provided as training input data to the machine-learning model, and the feedback may be used as desired output, e.g. regarding the actual interest of the user in the respective potential regions of interest, or regarding the assignment between potential regions of interest and categories of interest.

As has been outlined above, in some examples, two machine-learning models may be used - a first for determining candidates for the potential regions of interest, and a second for determining whether the candidates might be of actual interest to the user, or for determining a category of interest of the potential candidates of interest. In this case, the first machine-learning model may remain untouched, whereas the second machine-learning model may be adapted based on the feedback, as it reflects the user-specific interest or categorization of the potential regions of interest. Accordingly, the system may be configured to adapt the second machine-learning model based on the feedback on the user.

In various examples, the system 110 (or the further system 130) may be configured to provide a display signal to a display (e.g. ocular displays) of the microscope 120, the display signal comprising a visualization of the plurality of potential regions of interest.

The interface 112 may correspond to one or more inputs and/or outputs for receiving and/or transmitting information, which may be in digital (bit) values according to a specified code, within a module, between modules or between modules of different entities. For example, the interface 112 may comprise interface circuitry configured to receive and/or transmit information. In examples the one or more processors 114 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. In other words, the described function of the one or more processors 114 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. In at least some examples, the one or more storage devices 116 may comprise at least one element of the group of a computer readable storage medium, such as an magnetic or optical storage medium, e.g. a hard disk drive, a flash memory, Floppy-Disk, Random Access Memory (RAM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), an Electronically Erasable Programmable Read Only Memory (EEPROM), or a network storage. More details and aspects of the system and microscope system are mentioned in connection with the proposed concept or one or more examples described above or below (e.g. Fig. 2 to 5). The system and microscope system may comprise one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

Fig. 2 shows a flow chart of an example of a corresponding (computer-implemented) method for a system for a microscope. For example, the system, and the microscope, may be implemented similar to the system and microscope system described in connection with Figs, la to le. The method comprises receiving 210 image data from an optical imaging sensor of the microscope. The method comprises determining 220, using a machine-learning model, a plurality of potential regions of interest in the image data. The method comprises providing 230 review images of the potential regions of interest to a mobile device of a user of the microscope. The method comprises receiving 240 feedback on the review images of the potential regions of interest from the mobile device of the user. The method comprises adapting 250 the machine-learning model based on the feedback.

As indicated above, features described in connection with the system 110 and the microscope system 100 of Figs, la to le may be likewise applied to the method of Fig. 2.

More details and aspects of the method are mentioned in connection with the proposed concept or one or more examples described above or below (e.g. Fig. la to le, 3a to 5). The method may comprise one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

Fig. 3a shows a block diagram of an example of a system 310 for a mobile device 300, and of a mobile device 300 comprising such a system 310. The system 310 comprises one or more processors 314 and one or more storage devices 316. Optionally, the system further comprises an interface 312. The one or more processors 314 are coupled to the optional interface 312 and the one or more storage devices 316. In general, the functionality of the system 310 is provided by the one or more processors 314, e.g. in conjunction with the optional interface 312 and/or the one or more storage devices 316. Fig. 3a further shows the mobile device 300 comprising the system 310 (and the user interface 320). Fig. 3a further shows a system comprising the system 110 and the mobile device 300. Figs, la to 1c further show a system comprising the (entire) microscope system 100 and the mobile device 300.

The system is configured to receive review images of potential regions of interest 150 in image data 140 of a microscope 120 from a microscope system 100 (e.g. via the interface 312). The system is configured to provide the review images of the potential regions of interest to a user of the mobile device via a user interface 320 of the mobile device. The system is configured to obtain feedback on the review images of the potential regions of interest from the user via the user interface. The system is configured to provide the feedback to the microscope system 100 (e.g. via the interface 312).

While the system, method and computer program of Figs, la to 2 relate to the entity that processes the image data provided by the microscope, and that determines the plurality of potential regions of interest, the system for the mobile device and the corresponding mobile device may be used to review and provided feedback on the potential regions of interest, e.g. on the go and/or away from the microscope. In general, any type of mobile device may be used, such as a smartphone, tablet computer, or also a laptop computer. Preferably, the mobile device may be a touch-based mobile device, such as smartphone or tablet computer. Accordingly, the user interface may be a touch-based user interface, which may be provided by an operating system and/or application program of the mobile device.

The system is configured to receive the review images of the potential regions of interest 150 in image data 140 of a microscope 120 from a microscope system 100. As has been outlined in connection with Figs, la to 2, the review images may, in some examples, be cropped versions of the image data, e.g. cropped version of the image data that show at least the potential regions of interest. Alternatively, the review images may be processed versions of the image data, e.g. of portions of the image data showing the potential regions of interest. Alternatively, the review images may be based on the higher-resolution image data of the potential regions of interest. For example, the review images may represent the potential regions of interest in multiple zoom levels. For example, the review images may be received as individual files or image data streams, or the review images may be received as part of a large image, e.g. of a portion of the image data comprising (all of) the potential regions of interest. The system is configured to provide the review images of the potential regions of interest to a user of the mobile device via the user interface 320 of the mobile device. In general, the user interface may be a touch-based user interface, i.e. a user interface in which information is provided via a display of a touch screen, and user input is obtained via a touch sensor of the touch screen. Alternatively, other types of user interfaces may be used, such as a user interface comprising a display for displaying the review images, and a microphone for obtaining a voice command for controlling the user interface. In general, however, the system may be configured to provide the review images via a display of the user interface. In other words, the review images may be shown, one at a time, on the user-interface. For example, the preview images may be provided (e.g., shown) separately to the user, e.g., such that the user is shown a single region of interest at a time. For example, the review images may be shown such, that the user is provided with functionality to scroll and/or zoom the review images via the user interface.

The system is also configured to obtain the feedback on the review images of the potential regions of interest from the user via the user interface. As has been pointed out before, the user interface may be a touch-based user interface. Accordingly, the feedback may be obtained via a detection of touch input into the user interface. In some examples, however, more complex gestures may be considered. For example, the feedback may be obtained using gesture detection for detecting a swiping motion of the user in the user interface. In other words, the system may be configured to perform gesture detection on touch input to the user interface, and to obtain the feedback based on the gesture detection.

In some examples, the feedback may indicate whether the potential regions of interest are of interest to the user. For example, the system may be configured to determine that a potential region of interest is of no interest to the user if a swiping motion to the left is detected, and that the potential region of interest is of interest to the user if a swiping motion to the right is detected (or vice versa). In other words, the feedback on whether the potential regions of interest are of interest to the user may be obtained based on a directionality of a swiping motion performed by the user (e.g. right or left).

Fig. 3b shows a schematic diagram of an example of a swiping-based feedback mechanism. In Fig. 3b, the mobile device 300 is shown with a touch-based user interface 320. On the user interface, a review image is shown, with a bounding box 150 showing the potential region of interest. The user may use put a finger on the touch-based user interface and swipe left or right to indicate their interest regarding the displayed potential region of interest.

The same system may be extended and be applied to different categories of interest to the user. For example, the system may be configured to determine that a potential region of interest is of no interest to the user if a swiping motion to the left is detected, that the potential region of interest belongs into a first category of interest if a swiping motion to the right is detected, that the potential region of interest belongs into a second category of interest if a swiping motion to the top is detected, and that the potential region of interest belongs into a third category of interest if a swiping motion to the bottom is detected etc. Also, diagonal swiping motions may be used if more than three categories of interest are used, and fewer directions may be used is fewer than three categories of interest are used.

Alternatively, as has been pointed out above, a microphone may be used to obtain the feedback, e.g. via voice commands. Again, the feedback may indicate whether the potential regions of interest are of interest to the user, and/or that they belong to a category of interest to the user.

The system is configured to provide the feedback to the microscope system 100. As has been outlined in connection with Figs, la to 1c, the feedback may be provided as training data for adapting a training of a machine-learning model being used to determine the potential regions of interest within the image data. For example, the provided feedback may comprise information whether the potential regions of interest are of interest to the user, and/or information on which categories of interest the respective potential regions belong (if they are of interest). In general, the feedback may be transmitted, e.g. uploaded, to the microscope system 100. For example, the feedback may be provided to the system 110 of the microscope system 100. Accordingly, the review images may be obtained from the system 110 of the microscope system 100. In general, for the communication between the mobile device and the system 110, a connection via a computer network may be used, e.g. via the internet. Accordingly, the interface 112 may be configured to communicate via the computer network, e.g. via the internet.

The interface 312 may correspond to one or more inputs and/or outputs for receiving and/or transmitting information, which may be in digital (bit) values according to a specified code, within a module, between modules or between modules of different entities. For example, the interface 312 may comprise interface circuitry configured to receive and/or transmit information. In examples the one or more processors 314 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. In other words, the described function of the one or more processors 314 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. In at least some examples, the one or more storage devices 316 may comprise at least one element of the group of a computer readable storage medium, such as an magnetic or optical storage medium, e.g. a hard disk drive, a flash memory, Floppy-Disk, Random Access Memory (RAM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), an Electronically Erasable Programmable Read Only Memory (EEPROM), or a network storage.

More details and aspects of the system and mobile device are mentioned in connection with the proposed concept or one or more examples described above or below (e.g. Fig. la to 2, 3a to 5). The system and mobile device may comprise one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

Fig. 4 shows a schematic diagram of an example of a corresponding (computer-implemented) method for a mobile device. For example, the mobile device may be the mobile device shown in connection with Figs. 3a and/or 3b. The method comprises receiving 410 review images of potential regions of interest in image data of a microscope from a microscope system. The method comprises providing 420 the review images of the potential regions of interest to a user of the mobile device via a user interface of the mobile device. The method comprises obtaining 430 feedback on the review images of the potential regions of interest from the user via the user interface. The method comprises providing 440 the feedback to the microscope system.

As indicated above, features described in connection with the system 310 and the mobile device 300 of Figs. 3a to 3b may be likewise applied to the method of Fig. 4. More details and aspects of the method are mentioned in connection with the proposed concept or one or more examples described above or below (e.g. Fig. la to 3b, 5). The method may comprise one or more additional optional features corresponding to one or more aspects of the proposed concept or one or more examples described above or below.

In connection with Figs, la to 4, systems, methods and computer programs for identifying potential regions of interest, for obtaining feedback on said potential regions of interest and for adapting a machine-learning model based on the feedback have been introduced. Through the use of the proposed systems, methods and computer programs, reporting feedback becomes easier and faster. No computationally powerful input device may be required for this feedback. Annotations on the computer by mouse and keyboard are tiring. Such annotations are usually provided in one go at the beginning (e.g. via existing benchmark data sets or via an outsourced service provider). The proposals are then practically "set in stone" and no longer adapt to the user's concrete samples.

To perform the annotation (i.e. the feedback process), after the user has inserted their sample, the user may start an overview scan. Depending on the size of the sample, this may take a while. During this time, the user may already leave the microscope and later be notified by a simple push message on his mobile device/phone (on the way to work, for example in the canteen) that suggestions (ROIs) are available.

By means of image-based features, the suggestions (i.e. the review images of the potential regions of interest) may be presented to the user individually on the (mobile) screen and the user may classify them as useful or useless (i.e. whether the suggestions are of interest to the user) with a simple interaction (e.g. to swiping to the right or left). The user feedback may be fed back to the microscope system and improve the future ROI detection (e.g. using Reinforcement Learning).

The user may also define own categories/types for the samples, so that the suggestions remain specific and useful for certain types and are not "diluted" by heterogeneous samples. These categories can also be shared by several users (community), thus creating a larger database for better proposals more quickly. Various aspects of the present disclosure may improve efficiency, as the proposed classification allows to quickly process large amounts of proposals without having to correct pixel- exact annotations for each proposal. Interaction on the touch screen is commonplace and very familiar to many people. Also, through the use of Reinforcement Learning, a continuous improvement of the suggestions and adaptation to the sample/user requirements may be provided. It may be difficult to make good suggestions for all the different samples right from the start. Thus, an approach that adapts to the use may prove useful.

As used herein the term “and/or” includes any and all combinations of one or more of the associated listed items and may be abbreviated as “/”.

Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

Some embodiments relate to a microscope comprising a system as described in connection with one or more of the Figs, la to 4. Alternatively, a microscope may be part of or connected to a system as described in connection with one or more of the Figs. 1 to 4. Fig. 5 shows a schematic diagram of a system comprising a microscope and a computer system. Fig. 5 shows a schematic illustration of a system 500 configured to perform a method described herein. The system 500 comprises a microscope 510 and a computer system 520. The microscope 510 is configured to take images and is connected to the computer system 520. The computer system 520 is configured to execute at least a part of a method described herein. The computer system 520 may be configured to execute a machine learning algorithm. The computer system 520 and microscope 510 may be separate entities but can also be integrated together in one common housing. The computer system 520 may be part of a central processing system of the microscope 510 and/or the computer system 520 may be part of a subcomponent of the microscope 510, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 510.

The computer system 520 may be a local computer device (e.g. personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g. a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and/or one or more remote server farms and/or data centers). The computer system 520 may comprise any circuit or combination of circuits. In one embodiment, the computer system 520 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g. camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 520 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 520 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and/or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 520 may also include a display device, one or more speakers, and a keyboard and/or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 520.

Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein. A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

Some techniques may be applied to some of the machine-learning algorithms. For example, feature learning may be used. In other words, the machine-learning model may at least partially be trained using feature learning, and/or the machine-learning algorithm may comprise a feature learning component. Feature learning algorithms, which may be called representation learning algorithms, may preserve the information in their input but also transform it in a way that makes it useful, often as a pre-processing step before performing classification or predictions. Feature learning may be based on principal components analysis or cluster analysis, for example.

In some examples, anomaly detection (i.e. outlier detection) may be used, which is aimed at providing an identification of input values that raise suspicions by differing significantly from the majority of input or training data. In other words, the machine-learning model may at least partially be trained using anomaly detection, and/or the machine-learning algorithm may comprise an anomaly detection component.

In some examples, the machine-learning algorithm may use a decision tree as a predictive model. In other words, the machine-learning model may be based on a decision tree. In a decision tree, observations about an item (e.g. a set of input values) may be represented by the branches of the decision tree, and an output value corresponding to the item may be represented by the leaves of the decision tree. Decision trees may support both discrete values and continuous values as output values. If discrete values are used, the decision tree may be denoted a classification tree, if continuous values are used, the decision tree may be denoted a regression tree.

Association rules are a further technique that may be used in machine-learning algorithms. In other words, the machine-learning model may be based on one or more association rules. Association rules are created by identifying relationships between variables in large amounts of data. The machine-learning algorithm may identify and/or utilize one or more relational rules that represent the knowledge that is derived from the data. The rules may e.g. be used to store, manipulate or apply the knowledge.

Machine-learning algorithms are usually based on a machine-learning model. In other words, the term "machine-learning algorithm" may denote a set of instructions that may be used to create, train or use a machine-learning model. The term "machine-learning model" may denote a data structure and/or set of rules that represents the learned knowledge (e.g. based on the training performed by the machine-learning algorithm). In embodiments, the usage of a machine-learning algorithm may imply the usage of an underlying machine-learning model (or of a plurality of underlying machine-learning models). The usage of a machine-learning model may imply that the machine-learning model and/or the data structure/set of rules that is the machine-learning model is trained by a machine-learning algorithm.

In some embodiments, the machine-learning model may be a support vector machine, a random forest model or a gradient boosting model. Support vector machines (i.e. support vector networks) are supervised learning models with associated learning algorithms that may be used to analyze data (e.g. in classification or regression analysis). Support vector machines may be trained by providing an input with a plurality of training input values that belong to one of two categories. The support vector machine may be trained to assign a new input value to one of the two categories. Alternatively, the machine-learning model may be a Bayesian network, which is a probabilistic directed acyclic graphical model. A Bayesian network may represent a set of random variables and their conditional dependencies using a directed acyclic graph. Alternatively, the machine-learning model may be based on a genetic algorithm, which is a search algorithm and heuristic technique that mimics the process of natural selection. 5

PAGE RECEIVED BLANK UPON FILING

List of reference Signs

100 Microscope system

110 System for a microscope

112 Interface

114 One or more processors

116 One or more storage devices

120 Microscope

130 Further system

140 Image data

150 Potential region of interest/bounding box

160 Padding

210 Receiving image data

220 Determining potential regions of interest

230 Providing review images

240 Receiving feedback

250 Adapting a machine-leaning model

300 Mobile device

310 System for a mobile device

312 Interface

314 One or more processors

316 One or more storage devices

320 User interface

410 Receiving review images

420 Providing the review images to a user via a user interface

430 Obtaining feedback on the review images via the user interface

440 Providing the feedback

500 System

510 Microscope

520 Computer system

Claims

1. A system (110; 520) for a microscope (120; 510), the system comprising one or more processors (114) and one or more storage devices (116), wherein the system is configured to: obtain image data (140) from an optical imaging sensor of the microscope; determine, using a machine-learning model, a plurality of potential regions of interest (150) in the image data; provide review images of the potential regions of interest to a mobile device (300) of a user of the microscope; obtain feedback on the review images of the potential regions of interest from the mobile device of the user; and adapt the machine-learning model based on the feedback.

2. The system according to claim 1, wherein the feedback is used as training data to train the machine-learning model.

3. The system according to one of the claims 1 or 2, wherein the system is configured to use reinforcement learning to adapt the machine-learning model based on the feedback, or wherein the system is configured to use supervised learning to adapt the machinelearning model based on the feedback.

4. The system according to one of the claims 1 to 3, wherein the machine-learning model is trained to generate bounding boxes around the potential regions of interest within the image data, the image data being provided as input to the machine-learning model and a location of the potential regions of interest being provided as output of the machine-learning model, wherein the review images are based on the bounding boxes generated by the machine-learning model. The system according to one of the claims 1 to 4, wherein the machine-learning model is trained to determine the plurality of potential regions of interest in at least one of one or more categories of interest for the user. The system according to one of the claims 1 to 5, wherein the machine-learning model comprises a first machine-learning model that is trained to generate bounding boxes around potential regions of interest within the image data and a second machine-learning model that is trained to estimate an interest of the user, in at least one of one or more categories of interest for the user, of the portion of the image data visible within the bounding boxes, wherein the system is configured to adapt the second machinelearning model based on the feedback on the user. A microscope system (100; 500) comprising a microscope (120; 510) and the system (110; 520) according to one of the claims 1 to 6, the system being coupled to the microscope (120). The microscope system according to claim 7, wherein the system (110; 520) is colocated with the microscope (120; 510). The microscope system according to claim 7, wherein the system (110; 520) is implemented by a remote server that is remotely coupled with the microscope (120; 510) via a network. A system (310) for a mobile device (300), the system comprising one or more processors (314) and one or more storage devices (316), wherein the system is configured to: receive review images of potential regions of interest (150) in image data (140) of a microscope (120) from a microscope system (100); provide the review images of the potential regions of interest to a user of the mobile device via a user interface (320) of the mobile device; obtain feedback on the review images of the potential regions of interest from the user via the user interface; and provide the feedback to the microscope system (100). The system according to claim 10, wherein the feedback is obtained using gesture detection for detecting a swiping motion of the user in the user interface. A mobile device (300) comprising the system (310) according to one of the claims 10 or 11. A method for a system for a microscope, the method comprising: receiving (210) image data from an optical imaging sensor of the microscope; determining (220), using a machine-learning model, a plurality of potential regions of interest in the image data; providing (230) review images of the potential regions of interest to a mobile device of a user of the microscope; receiving (240) feedback on the review images of the potential regions of interest from the mobile device of the user; and adapting (250) the machine-learning model based on the feedback. A method for a mobile device (300), the method comprising: receiving (410) review images of potential regions of interest in image data of a microscope from a microscope system; providing (420) the review images of the potential regions of interest to a user of the mobile device via a user interface of the mobile device; obtaining (430) feedback on the review images of the potential regions of interest from the user via the user interface; and providing (440) the feedback to the microscope system. A computer program with a program code for performing the method according to claim 13 or the method according to claim 14 when the computer program is executed on a processor.