EP4637618A1

EP4637618A1 - Apparatus for an optical imaging system, display device, optical imaging system, method and computer program

Info

Publication number: EP4637618A1
Application number: EP23837671.9A
Authority: EP
Inventors: Manon ROSTYKUS; Wei Thiam Neo; Robert Paulus
Original assignee: Leica Microsystems CMS GmbH; Leica Instruments Singapore Pte Ltd
Current assignee: Leica Microsystems CMS GmbH; Leica Instruments Singapore Pte Ltd
Priority date: 2022-12-22
Filing date: 2023-12-21
Publication date: 2025-10-29
Also published as: WO2024133840A1; DE102022134578A1

Abstract

Examples relate to an apparatus for an optical imaging system. The apparatus comprises one or more processors and one or more storage devices. The apparatus is configured to receive first sensor data of a first sensor of the optical imaging system. The first sensor data is received from the optical imaging system. The first sensor data is indicative of a first view of a sample through a microscope of the optical imaging system. Further, the apparatus is configured to obtain error data indicative of an error of second sensor data of a second sensor or in receiving the second sensor data. The second sensor data is indicative of a second view of the sample through the microscope of the optical imaging system. Further, the apparatus is configured to generate display information for a display device indicative of the view of the sample to be displayed on multiple channels of the display device based on the first sensor data and the error data. The apparatus is further configured to transmit the display information to the multiple channels of the display device.

Description

Apparatus for an Optical Imaging System, Display Device, Optical Imaging System, Method and Computer Program

Technical field

Examples relate to an apparatus for an optical imaging system, a display device, an optical imaging system, a method and a computer program.

Background

In optical imaging system, e.g., surgical imaging system, a surgeon often uses a main pair of oculars or a head-mounted displays to perform a surgical procedure. To achieve a three-dimensional view of the sample under examination multiple channels may be used. For example, a first channel associated with a first eye of the surgeon and a second channel associated with a second eye of the surgeon may be used. However, the use of multiple channels can increase the susceptibility to errors. Thus, there may be a desire for an improved concept to maintain a display device coupled to an optical imaging system.

Summary

This desire is addressed by the subject-matter of the independent claims.

The concept proposed in the present disclosure is based on the insight, that a reliability of a display device using multiple channels can be increased by duplicating information of the first channel to the second channel. If a first view of a sample of a first channel is corrupted, e.g., caused by an error during data transmission, a second view of the sample of the second channel can be duplicated such that the second view is displayed on both channels. Thus, a two- dimensional image can be provided to the user, especially for both eyes. For example, displaying a black screen on one channel can be avoided.

Examples provide an apparatus for an optical imaging system. The apparatus comprises one or more processors and one or more storage devices. The apparatus is configured to receive first sensor data of a first sensor of the optical imaging system. The first sensor data is received from the optical imaging system. The first sensor data is indicative of a first view of a sample through a microscope of the optical imaging system. Further, the apparatus is configured to obtain error data indicative of an error of second sensor data of a second sensor or in receiving the second sensor data. The second sensor data is indicative of a second view of the sample through the microscope of the optical imaging system. Further, the apparatus is configured to generate display information for a display device indicative of the view of the sample to be displayed on multiple channels of the display device based on the first sensor data and the error data. The apparatus is further configured to transmit the display information to the multiple channels of the display device. For example, the error data may indicate that the second view is not available, e.g., due to a transmission error. Thus, the apparatus can duplicate the first view to be displayed on both channels. In this way, the user can see on both channels the same view. Displaying a black screen on the second channel can be avoided. Thus, a user experience can be increased.

In an example, the apparatus may be further configured to trigger a warning to a user of the display device to inform the user about the error data. For example, the user can be informed that no stereoscopic view of the sample can be provided any longer, because the second sensor data is erroneous. Informing the user about the error data can increase an awareness of the user that no three-dimensional view is available. In this way, the user can take certain actions to adapt to the situation.

In an example, the apparatus may be further configured to receive the second sensor data and check a quality of the second sensor data. Further, the apparatus may be configured to generate the image data if the quality of second sensor data is below a threshold. The quality of the second sensor data can be checked to determine if the second sensor data is suitable to be displayed on the second channel. If the quality is not suitable, the error data can be generated. In this way, the apparatus can verify the second sensor data and the user experience can be increased.

In an example, the apparatus may be further configured to generate the error data if an error in receiving the second sensor data occurs. For example, the apparatus may be unable to receive the second sensor data. In this way, the apparatus can generate the error data if needed. Examples provide a display comprising an apparatus as described above. Examples provide an optical imaging system comprising an apparatus as described above.

Examples provide a method for an apparatus for an optical imaging system for increasing a reliability of a display device. The method comprises receiving, from the optical imaging system, first sensor data of a first sensor of the optical imaging system, the first sensor data indicative of a view of a sample being observed through a microscope of the optical imaging system. The method further comprises obtaining error data indicative of an error of second sensor data indicative of a view of the sample being observed through a microscope of the optical imaging system. The method further comprises generating display information for the display device indicative of the view of the sample to be displayed on multiple channels of the display device and transmitting, to the display device, the display information.

Various examples of the present disclosure relate to a corresponding computer program with a program code for performing the above method when the computer program is executed on a processor.

Short description of the Figures

Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

Figs, la, lb and 1c show schematic diagrams of examples of an apparatus for an optical imaging system and of a corresponding optical imaging system comprising the apparatus;

Fig. 2 shows a block diagram of an example of an optical imaging system;

Fig. 3 shows an example of a method for an apparatus for an optical imaging system for increasing a reliability of a display device; and

Fig. 4 shows a schematic diagram of a system comprising a microscope and a computer system. Detailed Description

Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and/or regions may be exaggerated for clarity.

Figs, la, lb and 1c show schematic diagrams of examples of an apparatus 130 for an optical imaging system 100 and of a corresponding optical imaging system 100 comprising the apparatus 130. The apparatus 130 is tasked with controlling various aspects of a microscope 120 of the optical imaging system 100, which may be a surgical imaging system, and of the entire optical imaging system and/or with processing various types of sensor data of the optical imaging system 100. Consequently, the apparatus 130 may be implemented as a computer system, which interfaces with the various components of the optical imaging system, e.g., the sensors 122, 124.

The apparatus 130 comprises, as shown in Fig. la, one or more processors 134 and one or more storage devices 136. Optionally, the apparatus 130 further comprises one or more interfaces 132. The one or more processors 134 are coupled to the one or more storage devices 136 and to the optional one or more interfaces 132. In general, the functionality of the apparatus 130 may be provided by the one or more processors 134 (e.g., for generating the display information), in conjunction with the one or more interfaces 132 (for exchanging information, e.g., with the sensors 122, 124 or the display device 180, e.g., to transmit the display information) and/or with the one or more storage devices 136 (for storing and/or retrieving information).

The apparatus 130 is configured to receive first sensor data of a first sensor, e.g., the sensor 122, of the optical imaging system. The first sensor data is received from the optical imaging system 100. The first sensor data is indicative of a first view of a sample 110 through a microscope 120 of the optical imaging system 100. For example, the first sensor data is to be transmitted to the first channel of the display device 180. The first view is to be displayed to an eye of a user of the display device 180. The display device 180 may comprise two different optical paths, one path for each eye of the user. Thus, for each eye of the user a different view of the sample 110 can be displayed. An optical path may be assigned to a channel. In this way, the display device 180 can provide a three-dimensional view if both channels are provided with different views of the sample 110.

Further, the apparatus 130 is configured to obtain error data indicative of an error of second sensor data of the second sensor, e.g., the sensor 124, or in receiving the second sensor data (from the optical imaging system 100, e.g., from the sensor 124). The second sensor data is indicative of a second view of the sample through the microscope of the optical imaging system 100. For example, the second view and the first view can be combined to generate a three- dimensional view of the sample 110. The first view can be provided to the first channel of the display device 180 and the second view can be provided to the second channel of the display device 180. In this way, the display device 180 can display a three-dimensional view of the sample 110 to the user. However, if the second sensor data is erroneous and/or receiving of the second sensor data is disturbed, no view of the sample 110 can be displayed using the second channel of the display device 180. Thus, the user may see a black screen on the second channel. To overcome the degradation of the user experience due to the black screen, the apparatus 130 can be used.

The apparatus 130 is configured to generate display information for a display device 180 indicative of the view of the sample 110 to be displayed on multiple channels of the display device 180 based on the first sensor data and the error data. Thus, the first view of the sample 110 can be displayed on both channels of the display device 180. For example, if the error data indicates an error in receiving the second sensor data the apparatus 130 may generate the display information such that the display information comprises only the first view of the sample 110. No second view of the sample 110 may be part of the display information. In this way, the display device can be triggered to display on both channels the same view, namely the first view of the sample 110. Thus, a black screen of the second channel can be avoided. In this way, a user experience can be increased.

The apparatus 130 is further configured to transmit the display information to the multiple channels of the display device 180. For example, the display information comprising the first view may be transmitted to multiple channels of the display device 180. Thus, each channel of the display device 180 can be provided with the same view of the sample 110. In this way, the erroneous second sensor data for the second channel of the display device can be replaced by the first sensor data. Thus, the user can be shown the same view on both channels of the display device without a black screen appearing on one of the channels.

The apparatus 130 can ensure that the user is using a functional display device 180 without a critical error. This can increase risk prevention. The option of duplicating the first view of a first channel, e.g., a signal transmitted to the first channel, to a non-working second channel can ensure that the user can still see useful information (e.g., images from the surgical imaging system such like the sample 110) to finish some work, even though not stereoscopically.

The proposed concept is built around two main components - the microscope 120, which comprises the optical components, and which may house display devices being used to view the sample 110, and the apparatus 130, which is used to control the optical imaging system 100, process sensor data (e.g., the first sensor data, the second sensor data) of the microscope 120, e.g., the sensor 122, 124, and to generate display information.

In general, a microscope, such as the microscope 120, is an optical instrument that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope 120 may provide an optical magnification of a sample, such as a sample 110 shown in Fig. la. In modern microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensors 122 of the microscope 120. The microscope 120 may further comprise one or more optical magnification components that are used to magnify a view of the sample 110, such as an objective.

There are a variety of different types of optical imaging systems. For example, a microscope, an exoscope, an endoscope may be an imaging device of an optical imaging system. If the optical imaging system 100 is used in the medical or biological fields, the sample 110 may be a sample of organic tissue, e.g., arranged within a petri dish or present in a part of a body of a patient. In some examples of the present disclosure, e.g., as shown in Fig. lb, the optical imaging system 100 may be a surgical imaging system, e.g., a microscope system that is to be used during a surgical procedure, such as an oncological surgical procedure or during tumor surgery. However, the proposed concept may also be applied to other types of microscopy, e.g., microscopy in a laboratory or microscopy for the purpose of material inspection. As is evident, the optical imaging system 100 comprises a number of components, such as the apparatus 130, the microscope 120 with the at least one optical imaging sensors 122, an optional main pair of ocular displays 140, an optional secondary pair of ocular displays 145 and a head-mounted display 180. In the configuration shown in Fig. la, the microscope 120 may be a stereoscopic microscope 120 that supports a three-dimensional view of the sample 110 using the sensors 122, 124. However, other implementations may be possible, such as an implementation with a single sensor and a postprocessing of the sensor data of the single to generate a stereoscopic view.

Fig. lb shows a schematic diagram of an example of a (surgical) optical imaging system 100 comprising the microscope 120 and the apparatus 130. In general, a (surgical) optical imaging system is a system that comprises a microscope 120 and additional components, which are operated together with the microscope 120. In other words, a (surgical) optical imaging system is a system that comprises the microscope 120 and one or more additional components, such as the apparatus 130 (which may be a computer system being adapted to control and, for example, generate the control signal), an illumination system (which is used to illuminate an object being imaged by the microscope), additional sensors, displays etc.

The (surgical) optical imaging system 100 shown in Fig. lb comprises a number of optional components, such as a base unit 105 (which may comprise the apparatus 130) with a stand, ocular displays 140; 145 that are arranged at the microscope 120, a head-mounted display 180, and a (robotic or manual) arm 160 which holds the microscope 120 in place, and which is coupled to the base unit 105 and to the microscope 120. In general, these optional and non- optional components may be coupled to the apparatus 130, which may be configured to control and/or interact with the respective components.

In an example, the apparatus 130 may be further configured to trigger a warning to a user of the display device 180 to inform the user about the error data. In this way, an awareness of the user that the current view of the sample 110 is only a two-dimensional view can be increased. In this way, a user of the display device 180 can still be provided with an improved non-stereoscopic view of the sample 110. However, the user can be informed about the error data. This allows the user to wait until a stereoscopic view of the sample 110 may be available again before performing certain actions, such as a critical action during surgery. In an example, the apparatus 130 may be further configured to receive the second sensor data and check a quality of the second sensor data. For example, the apparatus 130 may check if a quality of the second view is good enough to be displayed on the second channel of the display device 180. For example, the quality may depend on the perceptual assessments that make an image pleasant for human viewers or an image quality assessment (e.g., based on computational models that can predict perceptual image quality).

Further, the apparatus 130 may be configured to generate the error data if the quality of second sensor data is below a threshold. For example, the threshold may be a value of the image quality assessment. In this way, the apparatus 130 can determine whether the second sensor data suitable to be displayed on the second channel of the display device 180.

In an example, the apparatus 130 may be further configured to generate the error data if an error in receiving the second sensor data occurs. For example, no second sensor data may be received from the optical imaging system at the apparatus 130. In this case, the second sensor data can be erroneously. Thus, the apparatus 130 may generate the error data if needed. In this way, the user can be informed about a reason of the error in the second sensor data. For example, the user can be informed about a problem in receiving second sensor data from the optical imaging system 100. Thus, the user or service person assisting the user can take certain actions to solve the problem in receiving the second sensor data.

In an example, the display device 180 may comprise multiple different display devices. For example, the display device 180 may comprise a first display device 180 (as shown in Fig. lb) and a second display device (not shown in Fig. lb). The first display device 180 may be associated with a (main) surgeon. The second display device may be associated with an assistant. The display information may be indicative of the view of the sample 110 to be displayed on multiple channels of the multiple different display devices. That is, at least one channel of the multiple channels may be associated with the first display device 180 and at least one further channel of the multiple channels may be associated with the second display device. The display information may be indicative of the view of the sample 110 to be displayed on the at least one channel of the first display device 180 and on the at least one further channel of the second display device. The multiple different display devices can display the same view of the sample 110. For example, a main surgeon of an optical imaging system 100 may use a first display device 180 and an assistant may use a second display device. Both, the first display device 180 and the second display device may receive in a normal mode sensor data from a respective optical imaging sensor of the optical imaging system 100. For example, the surgeon may receive a first view of the sample 110 and the assistant may receive a second view of the sample 110 different from the first view using the respective first display device 180 and the second display device. For example, the assistant may stand beside the surgeon and thus may receive a second view rotated with respect to the first view of the surgeon. In this case, the apparatus 130 can be configured to transmit display information to multiple display devices.

For example, the second sensor data can be associated with the second display device. That is, the error data can be indicative of an error of second sensor data of a second sensor or in receiving the second sensor data to be displayed on the second display device. In this case, the first display device 180 may not be affected by the error data. The apparatus 130 can than generate display information for multiple display devices based on the first sensor data (associated with the first display device 180). That is, the apparatus 130 can duplicate the first view of the sample 110 of a first channel, e.g., a signal transmitted to the first display device 180, to a non-working second channel, e.g., to the second display device. In this way, the assistant can see the same image as the surgeon. Thus, it can be ensured that the assistant can still see useful information (e.g., images from the optical imaging system 110 such like the sample 110) to finish some work, even if the information is not specifically associated with a viewing angle of the assistant. Thus, the apparatus 130 can be used to duplicate a first channel to a second channel of only one display device, e.g., the display device 180, and/or to duplicate a first channel of a first display device 180 to a second channel of second display device. That is, the apparatus 130 can transmit the display information to multiple channels of only one display device and/or to at least one channel of multiple display devices.

In an example, the apparatus 130 may be configured to determine, based on the error data, a source of an error related to the second sensor data and to trigger a measure to counteract the source of the error. For example, a source of the error data may be that the second sensor data could not be received from the second sensor. In this case, the apparatus 130 could trigger a restart and/or a control a diagnosis process of the second sensor. In this way, the apparatus may trigger a removal of the source of the error. In an example, the error data may be indicative of an error in transmitting the display information to a channel of the multiple channels of the display device. That is, the apparatus 130 may obtain information about an error of a transmission of the display information to the display device 180. For example, the display device 180 may comprise two viewing displays, namely a first viewing display and a second viewing display, each associated with a first channel and a second channel, respectively. One viewing display, e.g., a second viewing display associated with the second channel, may have a malfunction. Thus, the source of the error may be the malfunction of the second viewing display. The apparatus 130 may determine the source of the error and may, for example, trigger a restart and/or a diagnosis process of the second viewing display. In this way, the apparatus 130 may trigger a removal of the source of the error. That is, the apparatus 130 can trigger a removal of the source of the error at the display device 180 and/or at another component of the optical imaging system 100, e.g., an optical imaging sensor. Thus, the apparatus 130 can be configured to determine, based on the error data, a source of an error related to the second sensor data and trigger a measure to counteract the source of the error.

In an example, the apparatus 130 may be configured to check whether the source of the error is removed by the measure triggered and when the source of the error is not removed, to stop transmitting of the display information. In this way, the apparatus can check if a measure triggered has resolved the error or removed the source of the error. In this case, the optical imaging system 100 and/or the display device 180 may be fully functional. When the source of the error is not removed, stopping the transmission may allow the user to restore the fully functionality of the optical imaging system 100 and/or the display device 180. For example, the source of the error may be an unplugged cable and thus cannot be removed by a restart of a component. To restore the fully functionality of the optical imaging system 100 and/or the display device 180 the apparatus 130 may stop the transmission of the display information. In this way, the user can be forced to perform certain measures to counteract the source of the error. Optionally, the apparatus 130 can trigger displaying a standard image, such like an adjustment image, for the display device 180. The standard image displayed to the user may allow to inform the user in an improved way about a malfunction of the optical imaging system 100 and/or the display device 180. Optionally, when the apparatus 130 transmits the display information to multiple display devices, the transmission could be stop for each display device of the multiple display device. That is, no user of the optical imaging system 100 may receive any longer a view of the sample 110. In this way, it can be ensured that all users are aware of a (temporary) error and/or may pause a usage of the optical imaging system 100 and/or a display device.

In an example, the display information may be indicative of the source of the error. In this way, the user can be informed about the source of the error. For example, the warning triggered to the user may comprise information about the source of the error. That is, the warning triggered to the user may be part of the display information.

In an example, the apparatus 130 may be configured to determine, based on the error data, a time of occurrence of a source of an error related to the second sensor data and when the time exceeds a threshold, to stop transmitting of the display information. In this way, the apparatus 130 can stop the transmission of the display information when an occurrence of a source of an error exceeds a certain threshold. For example, a cable may be unplugged. Thus, the source of the error could only be removed by an action of a user. Therefore, after a certain time, the user may be forced by the apparatus 130 to perform certain measure to counteract the source of the error. In this way, it can be ensured that the user is not working with a decreased performance for long time. The threshold can be set to any desired value, such like 5 s, 10 s, 15s , 1 min or 2 min.

As shown in Fig. la the optional one or more interfaces 132 is coupled to the respective one or more processors 134 at the apparatus 130. In examples the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the one or more processors 134 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The one or more processors 134 is capable of controlling the one or more interfaces 132, so that any data transfer that occurs over the one or more interfaces 132 and/or any interaction in which the one or more interfaces 132 may be involved may be controlled by the one or more processors 134. In an embodiment the apparatus 130 may comprise a memory, e.g., the one or more storage devices 136 and at least one or more processors 134 operably coupled to the memory and configured to perform the method described below.

In examples the one or more interfaces 132 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g. any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The one or more interfaces 132 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.

The apparatus 130 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system. The apparatus 130 may be part of the display device 180 (as shown in Fig. 1c). Alternatively, the apparatus 130 may be extern to the display device 180 and may transmit the display information to the display device 180 (as shown in Fig. la and Fig. lb).

More details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described below (e.g., Fig. 2 - 4).

Fig. 2 shows a block diagram of an example of an optical imaging system 200. The optical imaging system 200 comprises a microscope 220, a hub 228, and the display device 280. The microscope 220 may be configured to transmit two different signals 204, 206 to the hub 228. The first signal 204 may comprise the first sensor data. The second signal 206 may comprise the second sensor data. The hub 228 may comprise the apparatus 130 as described in Fig. 1. The apparatus 130 may receive the first signal 204 and the second signal 206 from the microscope 220.

The apparatus may generate the display information as described above. The generated display information may be transmitted to the display device 280. The hub 228 may be configured to transmit two different signals, a first display signal 284 and a second display signal 286 to the display device 280. Each display signal 284, 286 may be associated with a channel of the display device 280. In this way, the display device can receive a separated display signal 284, 286 for each channel. The hub 228 may be device to control the display device 280 and/or the microscope 220. For example, the hub 228 may be a computer on which a software for controlling the microscope 220 and/or the display device 280 is installed.

For example, if the second signal 206 is not working properly the apparatus 130 may generate the display information as described above. The generated display information may comprise the first sensor data part of the first signal 204. For example, the generated display information may be identical to the first signal 206.

The generated display information is transmitted to the display device 280. For example, the first display signal 284 may be identical to the second display signal 286. In this way, the display device 280 can receive a view of the sample to be displayed on each channel, even if for one channel no signal may be available.

More details and aspects are mentioned in connection with the examples described above and/or below. The example shown in Fig. 2 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1) and/or below (e.g., Fig. 3 - 4).

Fig. 3 shows an example of a method 300 for an apparatus for an optical imaging system for increasing a reliability of a display device. The method 300 comprises receiving 310, from the optical imaging system, first sensor data of a first sensor of the optical imaging system, the first sensor data indicative of a view of a sample being observed through a microscope of the optical imaging system. The method 300 further comprises obtaining 320 error data indicative of an error of second sensor data indicative of a view of the sample being observed through a microscope of the optical imaging system. The method 300 further comprises generating 330 display information for the display device indicative of the view of the sample to be displayed on multiple channels of the display device and transmitting 340, to the display device, the display information. The method may be performed by an apparatus as described in Fig. 1. More details and aspects are mentioned in connection with the examples described above and/or below. The example shown in Fig. 3 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 2) and/or below (e.g., Fig. 4).

Some embodiments relate to a microscope comprising an apparatus as described in connection with one or more of the Figs. 1 to 3. Alternatively, a microscope may be part of or connected to an apparatus as described in connection with one or more of the Figs. 1 to 3. Fig. 4 shows a schematic illustration of a system 400 configured to perform a method described herein, e.g., with reference to Figs. 4 or 5. The system 400 comprises a microscope 410 and a computer system 420. The microscope may comprise the apparatus as described above, e.g., with reference to Fig. 1 and/or the device as described above, .e.g., with reference to Fig. 2. The microscope 410 is configured to take images and is connected to the computer system 420. The computer system 420 is configured to execute at least a part of a method described herein. The computer system 420 may be configured to execute a machine learning algorithm. The computer system 420 and microscope 410 may be separate entities but can also be integrated together in one common housing. The computer system 420 may be part of a central processing system of the microscope 410 and/or the computer system 420 may be part of a subcomponent of the microscope 410, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 410.

The computer system 420 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and/or one or more remote server farms and/or data centers). The computer system 420 may comprise any circuit or combination of circuits. In one embodiment, the computer system 420 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g., camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 420 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 420 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and/or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 420 may also include a display device, one or more speakers, and a keyboard and/or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 420.

More details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 4 may comprise one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 3).

Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein. A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.

The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example. List of reference Signs

100 optical imaging system

105 base

110 sample

120 microscope

122, 124 sensor

130 apparatus

132 interface

134 processor

136 storage device

140, 145 ocular display

160 arm

180 head-mounted display

204 first signal

206 second signal

220 microscope

228 hub

280 display device

284 first display signal

286 second display signal

300 method for increasing a reliability of a display device

310 receiving first sensor data

320 obtaining error data

330 generating display information

340 transmitting the display information

400 System

410 Microscope

420 Computer system

Claims

1. An apparatus (130) for an optical imaging system, comprising one or more processors (132) and one or more storage devices (134), wherein the apparatus (130) is configured to: receive, from the optical imaging system, first sensor data of a first sensor of the optical imaging system, the first sensor data indicative of a first view of a sample through a microscope of the optical imaging system; obtain error data indicative of an error of second sensor data of a second sensor or in receiving the second sensor data, the second sensor data indicative of a second view of the sample through the microscope of the optical imaging system; generate display information for a display device (180) indicative of the view of the sample to be displayed on multiple channels of the display device (180) based on the first sensor data and the error data; and transmit, to the multiple channels of the display device (180), the display information.

2. The apparatus (130) according to claim 1, further configured to trigger a warning to a user of the display device (180) to inform the user about the error data.

3. The apparatus (130) according to any one of the preceding claims, further configured to: receive the second sensor data; check a quality of the second sensor data; and generate the error data if the quality of the second sensor data is below a threshold.

4. The apparatus (130) according to any one of the preceding claims, further configured to generate the error data if an error in receiving the second sensor data occurs.

5. The apparatus (130) according to any one of the preceding claims, wherein the display device (180) comprises multiple different display devices; and the display information is indicative of the view of the sample to be displayed on multiple channels of the multiple different display devices, such that the multiple different display devices can display the same view of the sample.

6. The apparatus (130) according to any one of the preceding claims, wherein the apparatus (130) is configured to: determine, based on the error data, a source of an error related to the second sensor data; and trigger a measure to counteract the source of the error.

7. The apparatus (130) according to any one of the preceding claims, wherein the error data is indicative of an error in transmitting the display information to a channel of the multiple channels of the display device; and wherein the apparatus (130) is configured to: determine, based on the error data, a source of an error related to the second sensor data; and trigger a measure to counteract the source of the error.

8. The apparatus (130) according claim 6 or 7, wherein the apparatus (130) is configured to: check whether the source of the error is removed by the measure triggered; and when the source of the error is not removed, stop transmitting of the display information.

9. The apparatus (130) according claim 6, 7 or 8, wherein the display information is indicative of the source of the error.

10. The apparatus (130) according to any one of the preceding claims, wherein the apparatus (130) is configured to: determine, based on the error data, a time of occurrence of a source of an error related to the second sensor data; and when the time exceeds a threshold, stop transmitting of the display information.

11. A display device (150) comprising an apparatus (130) according to any one of the preceding claims.

12. An optical imaging system comprising at least one of an apparatus (130) according to any one of the claims 1 - 10 or a display device according to claim 11.

13. A method (300) for an apparatus for an optical imaging system for increasing a reliability of a display device; receiving (310), from the optical imaging system, first sensor data of a first sensor of the optical imaging system, the first sensor data indicative of a view of a sample being observed through a microscope of the optical imaging system; obtaining (320) error data indicative of an error of second sensor data indicative of a view of the sample being observed through a microscope of the optical imaging system; generating (330) display information for the display device indicative of the view of the sample to be displayed on multiple channels of the display device; and transmitting (340), to the display device, the display information.

14. A computer program having a program code for performing a method (300) according to claim 13 when the program is executed on processor.