Classifications

• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/24—Base structure
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/0004—Microscopes specially adapted for specific applications
  • G02B21/0012—Surgical microscopes
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B21/00—Microscopes
  • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
  • G02B21/365—Control or image processing arrangements for digital or video microscopes
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B19/00—Programme-control systems
  • G05B19/02—Programme-control systems electric
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B2219/00—Program-control systems
  • G05B2219/30—Nc systems
  • G05B2219/35—Nc in input of data, input till input file format
  • G05B2219/35453—Voice announcement, oral, speech input
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10064—Fluorescence image

Definitions

• the invention relates to a method for controlling a microscope and to a microscope suitable for carrying out the method.
• control commands of a control program by an operator of the microscope are selected.
• the selected control commands are transmitted by a control unit to the microscope, whereupon this is controlled according to the transmitted control commands.
• the user of the microscope must be guided by a menu of the
• Control program navigate and may only after a variety of selections to the desired control command. If the user is not familiar with the control program or not familiar with it in detail, the operation of the
• the gestures In order that the gestures can be clearly detected, they must be executed within a detection range of the device and with a sufficient uniqueness for the detection and possibly several times.
• the invention has for its object to propose a way by which a simplified control of a microscope is possible.
• the invention is further based on the object of specifying an easily controllable microscope.
• the microscope control method is used to operate a microscope and comprises the following steps: detection of acoustic, graphically displayed and / or electronically coded voice information, in particular by means of a detector, an optical detection unit and / or a
• a voice information is understood to be information given by a sender of the voice information and understandable by a receiver.
• the language information can come from any spoken or written language.
• the speech information does not represent information that is directly usable for controlling the microscope.
• information in a so-called machine language is not voice information in the sense of this
• Lighting and / or drive units suitable machine language.
• the speech information can be spoken and is detected acoustically, for example by means of a detector in the form of a microphone.
• a graphically displayed language information is given by a static character, such as a graphic, a symbol, a letter, a number or a word, even if the language information is displayed only temporarily, for example on a display, graphically.
• the graphical language information may also be a string of characters in the form of a string.
• the graphical language information is a text.
• the graphical language information may be visualized in any manner and may be written, printed, projected or displayed, for example.
• the graphical voice information is detected optically, for example by means of a camera.
• the speech information and / or the graphic speech information are present as an electronic file, for example as a text file or a graphics file
• the speech information is referred to as electronically coded speech information.
• An electronically encoded voice information is for example also a file whose data represent an acoustic voice information, for example a sonogram.
• the electronically coded voice information is detected by means of a suitable receiving device, for example by means of a computer unit, an infrared receiver or a radio receiver.
• a voice command is an instruction whose meaning is directed to the targeted execution of a control process of the microscope.
• a voice information may be completely formed by one or more voice commands.
• the voice information can be next to the one for controlling the Microscope not relevant information, such as filler words, including one or more voice commands.
• a composite voice command consists of at least two in the
• Voice information sound sequences, characters or electronically encoded data.
• a reference command becomes a generic term of a number of possible ones
• the lens is moved in the Z direction and, for example defocused or focused an observation or scanning beam.
• Such an assignment takes place, for example, by comparing a number of previously known terms, which are each assigned to a reference command, with the acquired speech information. Additionally or alternatively, an algorithm can be used by which the terms contained in the language information with respect to their similarity in terms of their phonetics, meaning and / or
• Character similarity can be compared with one of the available reference commands.
• the reference command to which the terms most closely resemble is selected when the similarity has reached or exceeded a predetermined minimum.
• Speech information analyzed and determined depending on the detected language information at least one probable future course of the method.
• the language information recorded in this case is converted by means of the method according to the invention into a voice command, by which at least one control command, in particular a generated control command, is generated, in addition to the immediately commanded Process steps coded for further required process steps.
• a voice command by which at least one control command, in particular a generated control command, is generated, in addition to the immediately commanded Process steps coded for further required process steps.
• the compulsory use of a particular filter may be provided, although its use by the user has not expressly been instructed to be of use for the performance of the invention
• Each reference command is representative of a control command, in the transmission of which a specific control operation is performed on the microscope.
• a control command can be assigned several reference commands.
• a reference command can be assigned a fixed control command, which is always executed in the same way.
• An invariable control command is, for example, "HALT”, with which all functions of the microscope, in particular all currently executed movements, are stopped.
• the reference command may have a prescription assigned to form a generated control command.
• the control command is generated after selecting the reference command according to the rule.
• the generated control command may include variable components so that it is not always executed in the same manner as will be explained in more detail below.
• Microscope control method is the detected language information on
• Parameter information is a subsection of the voice information that is in a logically logical connection with the determined voice command. For example, the voice command “move to Z" and the reference command “height adjustment" have been selected.
• a voice command “move to Z” and the reference command “height adjustment” have been selected.
• Control technology logically associated parameter information, for example, the direction of height adjustment and / or the amount or the step size of the height adjustment.
• the parameter information forms part of a
• Parameter information combined to a composite voice command and the selected reference command is supplemented with the parameter information to a generated control command.
• the generated control command therefore has a component determined by the reference command and at least one component determined by a parameter information.
• the parameter information is, for example, information selected from a group comprising a location, a direction, a distance, an amount or a step size, an identification of an element of the microscope and a parameter value.
• Keywords are terms or combinations of terms, characters and / or numbers stored in a database that are common technical terms, for example, in the field of microscopy.
• the keywords can be kept in several languages.
• the use of keywords allows a faster and more accurate determination of the voice command, since the keywords have clear and known meanings in the voice information.
• the database can be designed as a so-called look-up table.
• the databases can be formed in further embodiments by suitable logical structures. For example, these structures define additional properties that provide important clues to an algorithm when interpreting the language information.
• Determined and selected keywords of a voice information are assigned an assignment rule by which the parameter information associated with the keyword is identified in the voice information and the keyword is combined with the parameter information into a composite voice command.
• a keyword is detected in the acquired language information and the keyword, for example, a parameter information belongs to a
• Voice information can be expected directly before or after the keyword or away from it.
• the language information is checked for the presence of a spatial and / or temporal sequence of keywords at least in a subsection of the language information, wherein in the presence of the sequence of keywords this sequence is determined as a voice command.
• At least the portion of the speech information containing the sequence is assigned to the activation state with which the sequence is at least in coincides with a predetermined dimensions, that is sufficiently similar or identical to this.
• a state of the microscope can be set in only one voice command, which is otherwise reachable only by a plurality of individual voice commands.
• the microscope can by means of a corresponding drive state representing sequence of
• discrete parameter values which are parameter information associated with the keyword, may become parameter value ranges
• the microscope control method according to the invention can be executed more efficiently in further possible embodiments if a selection of reference commands is taken from a set of initially available reference commands. Only the selected reference commands are used to determine the voice command.
• the method according to the invention solves the technical problem of setting a configuration of a complex instrument such as a microscope on the basis of acquired measured values, here in the form of recorded voice commands.
• detectors At least one optical detector
• the object is further achieved by means of a microscope, which has a computer and control unit.
• the computer and control unit is designed to have an acoustic, a graphically represented and / or an electronically coded
• the computer and control unit is further configured to generate at least one control command suitable for operation of the microscope, wherein the control command is either a fixed reference command associated with the invariable control command or the control command in response to a command associated with the reference command to form a generated control command is produced.
• the microscope is controllable by the computer and control unit.
• An associated or generated control command may be constructed so that with this or with particular portions of the control command a number of units of the microscope, such as motor drives, adjustable filters, spatial light modulator (SLM), acousto-optic
• Modulators AOM
• detectors and / or a light source can be controlled or controlled (complex control command
• a control command has several components or subcommands which serve to control different units, these can be assigned or generated as a function of the current operating state of the microscope or the respective unit to be controlled.
• current operating conditions By taking into account current operating conditions, a high reactivity of the method for controlling the microscope is achieved.
• Current operating conditions can be detected with suitable sensors and evaluated by means of a suitably configured computing unit and made available for the process.
• this has at least one motor drive, which can be controlled by means of a control command. If several drives are present on the microscope, instead of a complex control command, too
• control commands are assigned or generated.
• an experiment with fluorescence recording is to be carried out.
• the aim of the experiment is to study the functional role of a particular protein during cell division.
• the experiment consists of observing the protein associated with another protein during cell division using confocal fluorescence microscopy.
• these two proteins are specifically labeled, which can be done, for example, genetically or in fixed cells with antibody stains.
• the two fluorophores EGFP and mCherry, both proteins are used for labeling.
• the experimenter now wants to observe aspects of cell division three-dimensionally over time and, for example, briefly writes his measurement protocol for his new student or for technical assistance as follows:
• a complex control command is assigned or generated.
• correspondingly many individual control commands are assigned or generated.
• the recognized voice commands and keywords represent a first step of the method. However, this detected information is meaningless
• laser lines of 488 nm and 561 nm are used with a laser power ⁇ 2%.
• Existing detection filters are automatically adapted to the dye emission spectra of these laser lines.
• a lens of the microscope is selected which can serve a desired size image field with diffraction-limited resolution
• a 40 x 1, 2W for example, a 40 x 1, 2W.
• a time series is defined in which, during three hours of every five seconds, a picture stack with diffraction-limited resolution is scanned in all three dimensions.
• the sampled volume is 50 ⁇ x 50 ⁇ x 10 ⁇ .
• a sensitivity of the detector used is set to a preset value.
• the computer and control unit is designed to carry out a plausibility check of a control command to be generated or generated before it is executed.
• Fig. 1 is a schematic representation of an embodiment of a
• a microscope 1 has a computer and control unit 2, an objective 3 and a lens drive 4 for adjusting the objective 3.
• the computer and control unit 2 is connected to a detector 5 in the form of a microphone for the detection of acoustic information, in particular of acoustic voice information Sl in combination.
• an optical detector 5 in the form of a microphone for the detection of acoustic information, in particular of acoustic voice information Sl in combination.
• Detection unit 6 in the form of a camera and an interface 7 are provided, which are each in communication with the computer and control unit 2 in a form suitable for the exchange of data.
• the optical detection unit 6 is designed for the optical detection of graphically represented speech information S1 and is for example a CCD camera, a CMOS camera or a video camera.
• the optical detection unit 6 is a scanner.
• the interface 7 is designed to detect electronically coded voice information Sl.
• the computer and control unit 2 has an integrated database 8 in which reference commands RB, control commands StB and / or keywords SW are repeatedly retrievably stored.
• the database 8 is programmable in a further possible embodiment, in particular the stored in the database 8 reference commands RB, control commands StB and / or keywords SW by a user of the
• Microscope 1 or by means of a self-learning program supplements, erasable and / or changeable.
• the database 8 is embodied outside the computer and control unit 2 and stands with it in a connection suitable for exchanging data.
• the lens drive 4 can be controlled and the lens 3 in the Z direction Z adjustable.
• Lens 3 in the Z direction Z whose distance to an object plane 9, in which, for example, a sample to be microscoped can be arranged changed.
• the computer and control unit 2 is configured to compare the acoustic, the graphically displayed and / or the electronically coded voice information Sl with the stored reference commands RB and a voice command SB in response to a predetermined degree of agreement between at least a portion of the voice information Sl and one of the
• Reference commands RB to determine. Subsequently, that reference command RB is selectable, with which the voice command SB to a predetermined extent
• the computer and control unit 2 is further configured to generate at least one control command StB suitable for operating the microscope, the control command StB being assigned either to the selected reference command RB invariable control command StB or the control command StB is generated as a function of the reference command RB associated regulation to form a generated control command StB.
• the microscope 1 can be controlled by the computer and control unit 2.
• Microscope 1 according to the invention is exemplified the
• An acoustic voice information Sl is detected acoustically by means of the microphone 5.
• a graphically represented speech information S 1 which has the same meaning as the acoustic speech information S 1, is detected, for example read in or scanned, by means of the optical detection unit 6.
• the graphically represented speech information S1 may be a graphic, an image of at least one word, at least one character and / or at least one number.
• the graphically represented language information S1 can also be from a document, for example a handwritten or
• an electronically coded speech information S 1 is acquired by means of the interface 7.
• Voice information Sl is a text file, for example an electronic file
• SMS Short message
• MMS Mobile Broadcast
• the acquired speech information Sl is transmitted to the computer and control unit 2 and there converted into a format that an analysis of the detected
• the acquired speech information Sl is subdivided by means of a suitable program into individual units, for example into individual words, characters and / or numbers.
• the thus divided speech information Sl is analyzed and examined for the presence of speech commands SB.
• the speech information Sl in a possible embodiment of the
• LSI LSI Indexing
• Fill words contained in the speech information S1 are identifiable in one embodiment of the method and are not taken into account in the determination of a speech command SB.
• a search for keywords SW takes place in the acquired speech information S1.
• the keywords include, for example, technical terms and often used
• Such terms refer, for example, to contrast methods (eg fluorescence, DIC, digital interference contrast, digital interference contrast), parameters for setting a microscope (eg detector gain, laser power, filter), fluorescent dyes (eg GFP, Alexa 488) and recording modes (eg z-stack, multichannel image).
• contrast methods eg fluorescence, DIC, digital interference contrast, digital interference contrast
• parameters for setting a microscope eg detector gain, laser power, filter
• fluorescent dyes eg GFP, Alexa 488
• recording modes eg z-stack, multichannel image.
• each keyword SW can be present in the database 8, the information whether at least one parameter information PI is expected before or after the keyword SW, through which the concept of the keyword SW is concretized.
• the term "zoom 5" is found in the acquired speech information S 1 based on the analysis of the acquired speech information S 1 and the
• the keyword SW Zoom is linked to the information that usually after the keyword SW a number is specified as parameter information PI, which specifies the magnification factor (zoom factor). In this case, the keyword SW “Zoom” is followed by the number "5". Thus, the keyword SW "Zoom 5" is detected, which is stored as a reference command RB in the database 8.
• the reference command RB assigned to the keyword SW can be "zoom 5."
• the reference command RB is stored, for example, as "zoom 0-10".
• the parameter information PI falls within the parameter range of the reference command RB thus specified, and therefore a very high degree of agreement between the keyword SW and the reference command RB is established. Thereafter, the reference command RB is selected and the presence of a voice command SB verified, which is verified by the very high degree of coincidence.
• the selected reference command RB is a rule for forming a
• Control unit 2 for performing a plausibility check of the generated control command StB formed before its execution. It is checked whether the generated control command StB within the technical specifications of the
• Microscope 1 is, for example, whether a 5-fold magnification is even possible. It can also be checked whether the generated control command StB logically fits into the technological microscopy process that has taken place up to this point.
• the generated control command StB is classified as plausible, it is executed. If there is no plausibility, the execution of the generated control command StB is either aborted or its execution does not take place at all. It is also possible that in the absence of plausibility, a confirmation of the generated
• Control command StB is requested by the user of the microscope 1 and the generated control command StB is executed only after a confirmation.
• Speech information Sl performed in different variants.
• control states (configurations) of the microscope 1 are stored in a space (dictionary space). In the room becomes after a next neighbor, thus after one
• Control state sought which is as similar as possible to a sequence of keywords SW.
• the space in further embodiments of the method is reduced in its dimensionality.
• a selection of the reference commands RB can be made, by means of which the presence of a voice command SB is determined.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Analytical Chemistry (AREA)
• Optics & Photonics (AREA)
• Chemical & Material Sciences (AREA)
• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Automation & Control Theory (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Surgery (AREA)
• Microscoopes, Condenser (AREA)
• User Interface Of Digital Computer (AREA)
• Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=57042899

Family Applications (1)

Country Status (6)

Families Citing this family (1)

Family Cites Families (36)

• 2015
  • 2015-10-02 DE DE102015219121.4A patent/DE102015219121A1/de active Pending
• 2016
  • 2016-09-30 EP EP16774694.0A patent/EP3356876A1/de not_active Withdrawn
  • 2016-09-30 WO PCT/EP2016/073435 patent/WO2017055558A1/de active Application Filing
  • 2016-09-30 CN CN201680057418.3A patent/CN108139580B/zh active Active
  • 2016-09-30 US US15/765,467 patent/US11054626B2/en active Active
  • 2016-09-30 CN CN202110629683.7A patent/CN113253446B/zh active Active
  • 2016-09-30 JP JP2018516672A patent/JP7199965B2/ja active Active
• 2021
  • 2021-06-01 US US17/303,517 patent/US11681136B2/en active Active

Also Published As

Similar Documents

Legal Events