DE102016214391B4 - Method for controlling an operation of a coordinate measuring machine and coordinate measuring machine with control - Google Patents

Abstract

A method for controlling the operation of a coordinate measuring machine (211), wherein • a controller (220) of the coordinate measuring machine (211) continuously sends control signals to control (220) the operation of the coordinate measuring machine (211) to at least one device of the coordinate measuring machine (211) to be controlled outputs and evaluates control commands received at a first frequency via a command input and taken into account when generating and outputting the control signals, • body parts (70-75) of a user of the coordinate measuring device (211) are continuously recorded by at least one camera (15) so that with a second frequency that is greater than or at least equal to the first frequency, a temporal sequence of images of at least one gesture of the user is generated, • the sequence of images (51a, 51n) is made available to an evaluation device (222) and the gesture is recognized by the evaluation device (222) from the time sequence of images (51a, 51n) annt, • the evaluation device (222) generates at least one control command according to the recognized gesture and the control command is transmitted to the controller (220), • the controller (220) outputs the control signals depending on the control command.

Description

The invention relates to a method for controlling the operation of a coordinate measuring machine, a control of the coordinate measuring machine emitting control signals for controlling the operation of the coordinate measuring machine to at least one device of the coordinate measuring machine to be controlled. The device to be controlled can e.g. a disconnection device for disconnecting at least one drive of a movable part of the coordinate measuring machine or such a drive. The invention also relates to a coordinate measuring machine which has the control.

A coordinate measuring machine is understood to mean a device with which the coordinates of a workpiece and in particular surface coordinates can be measured. A relative movement of the workpiece and a sensor for measuring the coordinates is possible so that coordinates cannot be measured at a single position on the workpiece. In particular, it is possible that the workpiece (e.g. by moving a measuring table on which the workpiece is arranged) and / or the sensor or sensors can be moved. The coordinate measuring machine has at least one movement device in order to carry out the movement or movements. In particular, coordinate measuring machines with at least one drive for driving such a movement are known. Often the drives are assigned to a specific linear axis for the movement, i.e. the drive or a group of drives drives a movement in a straight line direction. As an alternative or in addition, drives of coordinate measuring machines are known which cause other movements, in particular rotary movements about an axis of rotation. In particular, movements with respect to several linear and / or rotational degrees of freedom of the movement of the sensor and / or the workpiece can be achieved by several drives. The mechanical and kinematic construction of the coordinate measuring machine (hereinafter: CMM for short) can therefore be different. Common are e.g. Coordinate measuring machines in portal design, articulated arm design, hexapod design, design with parallel kinematics, gantry design and horizontal arm design, to name just a few examples.

E.g. the sensor is a measuring head that is attached to a moving part (e.g. a quill or an arm) of the CMM. In particular, a button (e.g. a stylus) can be attached to the measuring head, with which the CMM touches the surface of the workpiece in order to generate the sensor signals of the measuring head. Therefore, in particular, a button for tactile probing of the workpiece to be measured is an example of a sensor or part of the sensor. The measuring head has, in particular, a sensor system that generates measurement signals, through the evaluation of which the coordinates can be determined. However, other sensors are also used in coordinate metrology. For example, the sensor can only trigger the measurement of the coordinates. This is the case, for example, with a switching measuring head, which generates a switching signal upon contact with the workpiece to be measured. B. triggers by reading the scales of the moving part or parts of the CMM. In principle, the sensors can be divided into sensors that measure by contact (tactile probing of the workpiece) and sensors that do not measure by contact. For example, optical or capacitive sensors for coordinate measurement are sensors that are not based on the principle of tactile scanning. It is also possible to classify sensors according to the type or size of the area of the workpiece, in particular simultaneously detected. In particular, sensors can measure coordinates of only one point or an area on the surface or also inside the workpiece or measure coordinates of a volume of the workpiece.

It is common to operate a coordinate measuring machine with a control panel, i.e. the user operates the control panel and in this way generates control commands for controlling the operation of the coordinate measuring machine, in particular for controlling the drive or drives. A control panel can e.g. have one or more joysticks for controlling the movement of at least one movable part of the CMM. Typically, some control elements are also part of the control panel, which are operated by finger pressure. In addition, an emergency shutdown button is usually part of the control panel. In many cases, processes for measuring workpieces are automated, i.e. The corresponding movements and measuring actions of the CMM are determined for a later execution before the actual measurement of a workpiece or by recording the processes during the measurement of a first copy of a mass-produced workpiece. The process of this definition also represents an operating mode of the coordinate measuring machine.

A disadvantage of using a control panel is the fact that the user has to look at the control panel at least from time to time. The user must also be within reach of the control panel. The user can therefore not fully focus his attention on the moving parts of the CMM and the workpiece and cannot move far from the control panel.

In the EP 2 788 714 A1 has already been proposed to use a so-called range camera, which is suitable for a user of the coordinate measuring machine to be directed to record a sequence of user range images. A control of the coordinate measuring machine is suitable for controlling a drive mechanism for driving a sensor head on the basis of changes in a current user range image compared to one or more previous user range images. For example, a controller determines a direction in which the user points and ensures that the sensor head executes a corresponding movement. The user can give commands to control the coordinate measuring machine by moving his extremities or other movements. For example, the user can intervene with regard to the movement path, define additional measuring points or carry out another calibration. He can also end the measurement.

US 2014/0 327 920 A1 describes a method for controlling an operation of a laser tracker. The laser tracker is an example of a coordinate measuring machine. There is an association between a command from a plurality of commands and a gesture from a plurality of gestures. The gestures can be a body position or a movement of a body part of a user. When the user is moving, a camera can take pictures of the user. The receipt of the command gesture can optionally be confirmed. The gestures can be used to control the measurement of an object.

It is an object of the present invention to specify a method for controlling the operation of a coordinate measuring machine which enables the user to give at least one control command for the operation of a coordinate measuring machine quickly, precisely and with little delay without operating a control panel. Another object of the present invention is to specify a coordinate measuring machine that implements control commands from a user quickly, precisely and with little delay.

According to a basic idea of the present invention, body parts of a user of the coordinate measuring machine are continuously recorded by at least one camera, so that a time sequence of images of at least one gesture by the user is generated with an image recording frequency. Furthermore, the control of the coordinate measuring machine, which controls the operation of the coordinate measuring machine, continuously generates control signals and outputs these to at least one device of the coordinate measuring machine to be controlled, in particular to at least one drive of a movement device. The controller evaluates control commands received via a command input at a first frequency and takes the control commands into account when generating and outputting the control signals to the at least one device to be controlled. I.e. on the basis of the control signals, the control commands are updated, confirmed and / or changed at most with a frequency that corresponds to the first frequency. In order to be able to use the control commands of the user quickly, precisely and with little delay to generate the control signals, the images of the body parts of the user are recorded with an image recording frequency that is greater than or at least equal to the (first) frequency the receipt of the control commands and their consideration by the controller.

In any case, as a result of the recording of the camera images, there is no delay in the acquisition and implementation of the control commands from the user. The high frequency of the recording of the camera images also enables precise recording and implementation of the control commands of the user, since a corresponding amount of information about the gesture or gestures of the user is available.

Under a control command is e.g. to understand a direct command to the controller, e.g. B. to make the transition from a previous operating mode to another operating mode. A control command is also given when the evaluation device receives information such as B. the direction and / or height of the speed of the gesture-controlled movement are transmitted and the controller has to take this information into account when executing its operation.

Preferably, from the temporal sequence of images recorded by the at least one camera, continuously and in each case taking into account the most recently recorded camera images as part of the temporal sequence, it is evaluated at a third frequency which gesture the user is currently displaying or performing. A currently valid gesture by the user corresponds to the most recently recorded (newest) camera images. This gesture is recognized by an evaluation device from at least one, in particular the most recently recorded, camera image or from a plurality of camera images of the sequence, taking into account the most recently recorded camera image of the sequence. It is possible that a recorded camera image is not assigned to the sequence of images that are evaluated if it does not meet a predefined criterion. This will be discussed in more detail below. When evaluating the sequence of images, however, it is assumed that at each point in time of the evaluation and thus recognition of a gesture or the attempt to recognize a gesture, at least one camera image is present as part of the sequence of images.

The third frequency of the evaluation is preferably also greater than or at least as great as the first frequency of the consideration of control commands by the controller.

A gesture is understood to mean both a static posture that does not change over time and a posture of the user that changes over time. The term posture relates to several body parts of the user that are arranged in an arrangement corresponding to the gesture. The body parts that define the gesture have at least one body joint and preferably a plurality of body joints. Different postures therefore differ in that the at least one joint is in different movement positions. In the case of non-static gestures, the respective gesture is also defined by the change in the movement position of the at least one joint. In particular, a plurality of gestures is predefined and corresponding information for checking whether the user is performing one of these predefined gestures is stored. When evaluating the sequence of images, the information about the predefined gestures is used to determine whether the user is currently executing one of these gestures or several of these gestures. For this purpose, as already mentioned, at least one of the camera images is evaluated if the evaluation determines one of the predefined gestures, because the body parts of the user recorded with the at least one camera image correspond to the predefined gesture, this gesture or these gestures are recognized. In this case, the evaluation device can output a signal corresponding to the gesture or gestures. In particular, a control command to the controller can be generated on the basis of this signal, or the signal itself is already the control command to the controller.

The frequencies mentioned do not necessarily have to be constant, unchangeable frequencies during the entire operation of the coordinate measuring machine, even if this is also possible. However, it is also possible for at least one of these frequencies to be an average frequency averaged over an operating period of the coordinate measuring machine. The operating period can in particular be the period in which the coordinate measuring machine is in the same operating mode, e.g. Calibration of a sensor or measurement of a workpiece or measurement of a workpiece with a fixed position and orientation of the workpiece relative to the sensor.

The sequence of images can be images from a single camera, especially if the camera itself provides information about the distances of the objects and body parts to the camera, as is the case with a range camera or a TOF (Time of Flight) camera. The camera images can, however, also be images from a plurality of cameras that were recorded simultaneously and from which three-dimensional images of the temporal sequence of images are preferably created. E.g. A pair of cameras, which record the user from different angles, provide a three-dimensional stereo image after appropriate image processing. In particular if the user complies with special requirements for executing gestures, such as Alignment of the body or body parts in the direction of the camera, then camera images from a conventional camera that takes two-dimensional images without distance information can also be sufficient for gesture recognition.

• • eine Steuerung des Koordinatenmessgerätes, die fortlaufend Steuersignale zur Steuerung des Betriebes des Koordinatenmessgerätes an zumindest eine zu steuernde Einrichtung des Koordinatenmessgerätes ausgibt und mit einer ersten Frequenz über einen Befehlseingang empfangene Steuerbefehle auswertet und bei der Erzeugung und Ausgabe der Steuersignale berücksichtigt,
• • Körperteile eines Nutzers des Koordinatenmessgerätes von zumindest einer Kamera fortlaufend aufgenommen werden, so dass mit einer zweiten Frequenz, die größer ist als oder zumindest gleich groß ist wie die erste Frequenz, eine zeitliche Folge von Bildern zumindest einer Geste des Nutzers erzeugt wird,
• • die Folge von Bildern einer Auswertungseinrichtung zur Verfügung gestellt wird und die Geste von der Auswertungseinrichtung aus der zeitlichen Folge von Bildern erkannt wird,
• • von der Auswertungseinrichtung entsprechend der erkannten Geste zumindest einen Steuerbefehl erzeugt und der Steuerbefehl zu der Steuerung übertragen wird,
• • die Steuerung die Steuersignale abhängig von dem Steuerbefehl ausgibt.

In particular, the following is proposed: A method for controlling the operation of a coordinate measuring machine, wherein

• A control of the coordinate measuring machine which continuously outputs control signals for controlling the operation of the coordinate measuring machine to at least one device of the coordinate measuring machine to be controlled and evaluates control commands received via a command input at a first frequency and takes them into account when generating and outputting the control signals,
• • Body parts of a user of the coordinate measuring machine are continuously recorded by at least one camera, so that a time sequence of images of at least one gesture by the user is generated at a second frequency that is greater than or at least equal to the first frequency,
• • the sequence of images is made available to an evaluation device and the gesture is recognized by the evaluation device from the temporal sequence of images,
• • at least one control command is generated by the evaluation device in accordance with the recognized gesture and the control command is transmitted to the controller,
• • the control outputs the control signals depending on the control command.

• • eine Steuerung, die ausgestaltet ist, fortlaufend Steuersignale zur Steuerung des Betriebes des Koordinatenmessgerätes an zumindest eine zu steuernde Einrichtung des Koordinatenmessgerätes auszugeben und mit einer ersten Frequenz über einen Befehlseingang empfangene Steuerbefehle auszuwerten und bei der Erzeugung und Ausgabe der Steuersignale zu berücksichtigen,
• • zumindest eine Kamera, die ausgestaltet ist, fortlaufend Körperteile eines Nutzers des Koordinatenmessgerätes aufzunehmen, so dass mit einer zweiten Frequenz, die größer ist als oder zumindest gleich ist wie die erste Frequenz, eine zeitliche Folge von Bildern zumindest einer Geste des Nutzers erzeugt wird, und
• • eine Auswertungseinrichtung zur Auswertung der zeitlichen Folge von Bildern, wobei die Auswertungseinrichtung ausgestaltet ist, die zeitliche Folge von Bildern gemäß einer der Ausgestaltungen des Verfahrens auszuwerten und zumindest einen Steuerbefehle zu erzeugen und zu der Steuerung zu übertragen, sodass die Steuerung die Steuersignale abhängig von dem Steuerbefehl ausgibt.

Furthermore, a coordinate measuring machine is proposed which has:

• A controller which is designed to continuously output control signals for controlling the operation of the coordinate measuring machine to at least one device of the coordinate measuring machine to be controlled and to receive them at a first frequency via a command input Evaluate control commands and take them into account when generating and outputting control signals,
• • at least one camera which is designed to continuously record body parts of a user of the coordinate measuring machine, so that a time sequence of images of at least one gesture by the user is generated at a second frequency that is greater than or at least equal to the first frequency, and
• An evaluation device for evaluating the temporal sequence of images, wherein the evaluation device is designed to evaluate the temporal sequence of images according to one of the embodiments of the method and to generate and transmit at least one control command to the controller, so that the controller generates the control signals depending on the Outputs control command.

The continuous output of control signals by the controller is understood to mean that control signals are output repeatedly (e.g. according to a work cycle) or are output continuously. In the latter case, when the control signals change, the continuously output control signal is changed accordingly.

Refinements of the method are described below. Corresponding configurations of the coordinate measuring machine result from this, since the method can be carried out in particular using the evaluation device and control of the coordinate measuring machine.

The following embodiments can also be carried out regardless of whether the recording and / or evaluation of the camera images takes place at a frequency that is the same or greater than the evaluation of the control commands by the controller.

In this respect, the configurations described below represent independent aspects of controlling the operation of a coordinate measuring machine by means of gestures.

It is preferred that three-dimensional images are evaluated by the evaluation device as parts of the time sequence of images. By also taking into account the information about the distance of the recorded parts and body parts from the camera, it is possible to determine with greater accuracy and more reliably which gesture is currently being displayed by the user.

A preferred group of body parts of the user for the representation of gestures is the hand. Since there is a right hand and a left hand, it is preferred that the hand for which a gesture was first recognized is followed up in the chronological sequence of images. The persecution ends e.g. only when this hand no longer represents a gesture or no longer represents the first recognized gesture. In particular, a gesture represented by the other hand can then trigger the pursuit of the other hand. Of course, it is also possible that, after each gesture has been formed, both hands are followed in the chronological sequence of images, i.e. for both hands e.g. an evaluation takes place for each image of the temporal sequence of images.

In particular, the evaluation device can create control commands in a format that is the same as the format that is used by a control panel for generating and transmitting control commands to the control of the CMM. The frequency of the transmission of the control commands can also be the same as with a control panel. This makes it possible to implement gesture control by a user instead of a control panel or in addition to a control panel. For example, control panels work with a command frequency of 10 Hz and the control is therefore z. B. also designed to evaluate control commands received via a command input at a frequency of 10 Hz and to take them into account when generating and outputting the control signals. A control panel is also understood to mean a touch-sensitive screen that may be present.

If a control panel is also used, settings can also be made with regard to gesture control via the control panel. For example, a maximum speed of movement of the moving part of the CMM that is to be controlled by gesture control can be made via the control panel. The maximum speed can, however, alternatively or additionally be set by displaying a gesture by the user.

If the user leaves a predetermined detection area of the at least one camera, the gesture control can be automatically deactivated. Alternatively or additionally, the user can deactivate the gesture control by displaying a gesture, in particular before leaving the specified detection area of the camera.

Gesture recognition per se is well known and, as mentioned above, it is in particular from the EP 2 788 714 B1 known to use a gesture to control operation of a CMM. It is therefore not necessary to describe the recognition of gestures in detail here. However, it has already been mentioned that in gesture recognition, the arrangement of a plurality of body parts of the user is determined from at least one camera image. In particular, it can be determined in each image which positions of the recognized joints of the user are. For example, a connection vector can be determined from the relative position in three-dimensional space of two adjacent joints of the user, which are connected to one another without another joint via bones of the user, which thus has an orientation in the two-dimensional or three-dimensional image. The advantage of this approach is that the length of the connection vector in all images of the time sequence corresponds to the same unchangeable length of the connection between the two joints. This makes it possible to determine the position of a joint that may not be recognized in a single image or that may not be reliably recognized. It should also be taken into account that when a large number of joints are recognized, the position of individual joints can be determined by several such connection vectors. There are therefore physical conditions for individual joints that must be met. This makes it easier to recognize the positions of the joints required for gesture recognition.

The determination of the positions of joints over several successively recorded images also contains information that can be taken into account when recognizing the gestures. So z. B. over a period of less than 100 milliseconds a part of the body of the user has not been moved arbitrarily far from its previous position. Taking into account the time interval between two or more evaluated images, an upper limit value for the change in the position (displacement) of a joint can therefore be established quickly and easily. This makes it easier to find joints in consecutive images. In addition, this criterion can be used to check whether a subsequent image has led to plausible results in the recognition of the joints, body parts and gestures. As an alternative to determining the position of joints, angles can be determined between bones that are articulated to one another via the joint. This can increase the detection accuracy, in particular because the angles can also occur in the same way in different people, whereas the size of joints can differ in different people. The angle or angles can also be angles of bones that are used to represent the same gesture, but are not articulated to one another via a common joint. An example is the above-mentioned gesture with forearms crossed in front of the torso.

In addition to recognizing gestures from camera images on the basis of predetermined gestures, via which information is stored in a manner accessible from the outset for the evaluation device, gestures can also be defined by machine learning. To do this, the user performs a gesture while a sequence of camera images is recorded. This is also known in principle and is not explained in more detail below.

In particular, gestures can be defined to which a control command is assigned that is used to switch from a first operating mode to a second operating mode of the CMM. For example, such a control command can switch over from moving the measuring head along linear movement axes to moving the measuring head or the sensor attached to it about at least one axis of rotation, or vice versa. The assigned gesture is, for example, the gesture in which the user holds his hands against one another above his head. Although this gesture and also certain other gestures cannot readily be recognized intuitively as being assigned to the assigned function of the CMM, they have the advantage that they are shown inadvertently with a low probability. They are therefore suitable for control commands that only need to be generated by briefly displaying the gesture. In the case of movement control over a longer period of time by displaying gestures, on the other hand, it is preferred that a check take place repeatedly to determine whether the same gesture is being displayed as at the beginning of the control. In the case of the gestures that only have to be carried out briefly in order to generate a control command, it can additionally be specified that the user is in a specific, specified position and / or orientation with respect to the at least one camera. Otherwise, the gesture is not recognized as the predefined gesture assigned to the control command. Another example of a control command is the command to start the movement sequence of a movable part of the CMM into a predetermined position (e.g. a so-called zero position). A gesture by which the control command is generated consists e.g. B. in the fact that the user keeps his forearms crossed in front of the upper body aligned in the direction of the camera.

Another gesture, e.g. B. to switch off the control, the user can hold an open hand with upwardly extended fingers with at least extended forearm and palm facing forward at a distance from the upper body.

If a gesture consists of a sequence of movements of body parts of the user, in particular individual segments of the movement are determined individually from images recorded one after the other. When all of the segments required to perform the gesture have been recognized, the entire gesture recognized. In this case, or if no gesture could be recognized, the recognition of gestures can be reset so that the recognition of a gesture can be started again. In the case of gestures with only a single gesture segment or static gestures, in order to increase the recognition reliability, the gesture segment or the gesture can only be classified as recognized if the segment or the gesture was recognized from a plurality of successively recorded images.

The device to be controlled can e.g. be a drive of a movement device of the KMG. Another example is an emergency shutdown device. Furthermore, the device to be controlled can be a higher-level control unit, e.g. implemented automatically by means of computer software controls a movement sequence of the KMG. With appropriate control signals from the control to the control unit, e.g. such a sequence of movements can be started, changed, interrupted or ended.

Controls of existing CMMs already have a command input to which e.g. the above-mentioned control panel is connected. Commands that the user generates using the control panel are transmitted to the control via the command input. Control commands generated according to the present invention, which are generated by evaluating the sequence of images of body parts of the user, can be transmitted to the controller via the same command input, or via a parallel command input if a control panel is also connected.

In particular, the evaluation device can repeatedly determine from the temporal sequence of images whether the recognized gesture is the same gesture that has already been recognized repeatedly from the temporal sequence of images during an ongoing period of a current operating state of the coordinate measuring machine Outputs control command to terminate the current operating state of the coordinate measuring machine to the controller when the same gesture is no longer recognized.

The repeated determination of whether the recognized gesture is still the same as before has the advantage that the user can continuously influence operation of the CMM. E.g. the user can control the movement of a moving part of the CMM by making a gesture with a group of body parts (e.g. the hand) and then moving the group of body parts. In addition to the gesture, in this case the movement of the group of body parts is also recorded from the sequence of images and corresponding control commands are generated which cause the controller to execute the movement of the associated movable part of the CMM accordingly. E.g. the movement speed and / or the direction of movement can be specified by the movement of the group of body parts of the user.

The termination of the current operating state when the same gesture is no longer recognized as before has the advantage of a high level of operational reliability. The user can end the display of the gesture and thus intentionally end the current operating state (e.g. the movement of the movable part by gesture control). However, there can also be a disruption in the detection of the user's gestures, e.g. when another person moves between the user and the camera. In this case, the current operating state is automatically terminated because the gesture displayed by the user can no longer be recognized.

In particular, at the beginning of the period of the current operating state of the coordinate measuring machine, a gesture can be recognized and a control command for starting the current operating state can be output. The control command corresponds to the recognized gesture. This makes it possible in particular to assign at least one gesture to different operating states of the CMM. When the respective gesture is recognized, the assigned operating state is started and maintained until the gesture is no longer recognized from current images of the sequence of images.

An example of a gesture which, when recognized for the first time from the sequence of images, can start an operating state of the coordinate measuring machine is a closed hand position, e.g. a fist. Since the user does not have a closed hand when the hand is in a relaxed state, this is a gesture that is very likely to be intentionally displayed by the user. A closed hand position can also be easily recognized from the images because there are no stretched fingers or individually protruding fingers that can easily be recognized.

Coordinate measuring machines often have a measuring head to which a sensor (e.g. an optical or tactile sensor) or a plurality of sensors can be attached. The measuring head or the at least one sensor attached to it is the preferred example of a movable part of the CMM, the movement of which can be controlled by gesture control.

Another embodiment of the gesture-based control of the operation of a CMM is used to move a movable part of the CMM. There there is the problem that the movement is to be carried out with one direction and one speed and there is therefore a need to be able to specify both the direction and the speed by the user. This is also possible with static gestures. For example, a first group of body parts of the user can represent a gesture which corresponds to the operating state to be carried out, namely the movement of the part of the coordinate measuring device to be moved. Depending on the direction of movement, the first group of body parts can be positioned in one direction relative to a second group of body parts of the user and this direction can be interpreted as a specification of the direction of movement. To preset the speed, the distance between the first group and the second group of body parts can be selected by the user and the speed of the movement can be preset according to the distance. For example, the first group of body parts can be one or two hands of the user and the other group of body parts can be the upper body or at least one of the shoulders of the user. Such a specification of the direction and speed of the movement to be carried out by static gestures is complicated and cannot be carried out intuitively for the user.

It is therefore proposed that the evaluation device repeatedly determine a movement vector of a movement of body parts of the user from at least two images of the time sequence of images, the evaluation device repeatedly issuing a control command for executing a movement of a movable part of the body corresponding to the movement of the body parts Coordinate measuring machine outputs. The controller receives the control commands and controls the corresponding devices to be controlled so that the direction and speed of movement specified by the user are implemented. The motion vector, which the user specifies and which is determined from the sequence of images, determines the direction of movement with its direction and with its length (which is optionally related to the time difference between the evaluated images) the height of the speed of the moving part of the KMG. The speed of the movement of the user can be converted into the speed of the movement of the moving part of the CMM according to a specification. E.g. a conversion factor can be specified and / or changed by the user.

If the movement vector is determined on the basis of the movement of a first group of body parts of the user (e.g. a hand), then a movement of other body parts of the user can lead to a change in the displayed movement unintentionally by the user. E.g. the user's upper body can move, while the user's hand used to represent the movement also moves relative to the upper body.

To overcome this problem, it is proposed that the evaluation device determine the movement vector as a movement vector of a first group of body parts relative to a second group of body parts of the user. E.g. therefore, the movement of the upper body does not distort the representation of the movement by the hand intended by the user. E.g. therefore, both the hand and the associated shoulder of the user are recognized from the sequence of images. If the shoulder moves in the comparison between the at least two images, the movement vector of the shoulder (in general: the second group of body parts) can be subtracted from the movement vector of the hand (in general: the first group of body parts) to obtain the desired one Obtain motion vector for generating control commands. Even if the first group of body parts is preferably the hand or the forearm of the user, other groups of body parts can also be considered as the first group, e.g. the head or a foot. In the case of the head e.g. the movement relative to the shoulder can be evaluated. In the case of the foot, the movement relative to the hip can be evaluated.

When specifying the movement of the user with detection of movement vectors, there is the problem that the movement range for the user is limited. This applies in particular when only the relative movement of a first group of body parts of the user to a second group of body parts of the user is evaluated. The user can therefore not continue the movement indefinitely. It can happen that the user reaches the limit of the possible range of motion with his movement before the movable part of the CMM controlled by him through gesture control has reached its limit of the range of motion.

To solve this problem, it is proposed that the evaluation device continue to issue control commands to continue the movement of the moving part of the CMM if the same gesture as before is still recognized from the temporal sequence of images, although the movement vector determined has a length equal to zero or less than a has specified limit value. The movement of the moving part of the CMM can thus be continued. On the other hand, the user has control over the movement process in that he cancels the gesture formed in particular with the first group of body parts so that it is no longer recognized. For example, the gesture can be the hand closed into a fist. If the If the user opens his hand, the control system stops the movement of the movable part because it receives a corresponding control command from the evaluation device, which in turn has recognized the resolution of the gesture.

Errors can occur in the recording of camera images and also in the further processing of the image data, including the transmission to the evaluation device and the evaluation by the evaluation device. In particular, the recording of the sequence of camera images can be temporarily disrupted. One possibility of a disturbance is a sudden change in lighting. Another possibility is that another person enters the detection range of the camera.

It is therefore proposed that the evaluation device recognize a currently valid gesture from a plurality of images of the temporal sequence of images, each of the plurality of images being recorded during a different time interval. It is thus possible for the evaluation device to recognize disturbances in a single one of the images and to exclude the image from being taken into account in the evaluation or to evaluate the majority of the images without recognizing a specifically disturbed image so that the influence of a single disturbed image or a small number disturbed images on the recognition result is low. In particular, the recording of the camera images at a higher frequency than the reception frequency of the controller for control commands enables this type of consideration of disturbances.

In a specific embodiment, the evaluation device can continuously recognize a currently valid gesture from a plurality of the images of the temporal sequence of images, the majority of the images being taken one after the other, the majority of the images containing the last image taken by the at least one camera and the majority of the Images does not contain more than a specified number of images.

Optionally, the last recorded image of the at least one camera can be the last recorded valid image. I.e. It can be decided by preprocessing not to include the last image recorded by the camera in the sequence of images that are available for evaluation, or to remove it therefrom. Such preprocessing will be discussed in more detail below.

In particular, the evaluation device can continuously determine a currently valid motion vector of the movement of body parts of the user from a plurality of at least three images of the temporal sequence of images, the majority of the images being recorded one after the other, containing the last image recorded by the at least one camera and no more than a predetermined number of images. Furthermore, the first recorded image of the plurality of images can no longer be taken into account when recognizing the currently valid gesture if a new image is added to the sequence of images and the specified number of images would be exceeded as a result.

Not only with regard to the recognition of a currently valid gesture and / or the determination of a currently valid motion vector, the majority of the images can be recorded, for example, in what is known as a ring memory, which only has storage space for the specified number of images. If a new picture that was recorded by the camera is recorded in the fully filled ring buffer, the oldest picture recorded by the camera is automatically removed or overwritten in the ring buffer. At the beginning of the chronological sequence of images, the ring buffer can contain fewer than the specified number of images. The evaluation by the evaluation device takes place at an evaluation time with the images stored in the ring memory.

E.g. When determining the motion vector, the motion vector can be determined for all pairs of images of the plurality of images recorded immediately one after the other and can e.g. a mean value of the motion vectors of all these image pairs can be determined. Alternatively or additionally, a motion vector can be excluded from further consideration if it is implausible, e.g. B. differs in an implausible way from the other determined motion vectors. When determining whether a motion vector is implausible, predefined mathematical criteria, such as e.g. Deviation by more than a specified maximum permitted deviation amount from all other vectors, and / or based on physical criteria, such as e.g. Consideration of the inertia of moving parts.

If a motion vector is determined from a plurality of successively recorded images, either the evaluation device or the control of the CMM can convert the coordinate system of the camera image or of the user into the coordinate system of the CMM, i.e. the motion vector determined from the camera images is transformed into the coordinate system of the CMM. In particular, information about the position of the user relative to the at least one camera can be used. Such information can be permanently specified or before and / or during the Operation of the KMG. The person skilled in the art of coordinate measuring machines is familiar with the transformation of vectors into other coordinate systems, so that no further details are given here.

In the following, an embodiment of the preprocessing of images is described in order to be able to decide whether an image is included in the sequence of images available for the evaluation or is removed from it.

It is proposed that the evaluation device tries to identify a plurality of specified body parts of the user from each image in the sequence of images and, based on a specified criterion, decides from the identified body parts whether sufficient information is provided in the respective image to recognize gestures of the user is available. The respective image is not taken into account for the recognition of gestures by the user if it is decided on the basis of the specified criterion that there is insufficient information in the respective image. E.g. the specified criterion can consist in the fact that a minimum number of joints of the user are recognized from the image. Alternatively or additionally, the predefined criterion can only be met if certain predefined joints of the user are recognized from the image and / or at most a predefined number of predefined joints of the user are not recognized from the image.

• • eine Steuerung des Roboters oder der Werkzeugmaschine, die fortlaufend Steuersignale zur Steuerung des Betriebes an zumindest eine zu steuernde Einrichtung ausgibt und mit einer ersten Frequenz über einen Befehlseingang empfangene Steuerbefehle auswertet und bei der Erzeugung und Ausgabe der Steuersignale berücksichtigt,
• • Körperteile eines Nutzers von zumindest einer Kamera fortlaufend aufgenommen werden, so dass mit einer zweiten Frequenz, die größer ist als oder zumindest gleich groß ist wie die erste Frequenz, eine zeitliche Folge von Bildern zumindest einer Geste des Nutzers erzeugt wird,
• • die Folge von Bildern einer Auswertungseinrichtung zur Verfügung gestellt wird und die Geste von der Auswertungseinrichtung aus der zeitlichen Folge von Bildern erkannt wird,
• • von der Auswertungseinrichtung entsprechend der erkannten Geste zumindest einen Steuerbefehl erzeugt und der Steuerbefehl zu der Steuerung übertragen wird,
• • die Steuerung die Steuersignale abhängig von dem Steuerbefehl ausgibt.

The method is also suitable for gesture control of robots and machine tools. A method for controlling the operation of a robot or a machine tool is therefore also proposed, wherein

• • a control of the robot or the machine tool, which continuously outputs control signals for controlling the operation to at least one device to be controlled and evaluates control commands received via a command input at a first frequency and takes them into account in the generation and output of the control signals,
• • Body parts of a user are continuously recorded by at least one camera, so that a time sequence of images of at least one gesture by the user is generated at a second frequency that is greater than or at least equal to the first frequency,
• • the sequence of images is made available to an evaluation device and the gesture is recognized by the evaluation device from the temporal sequence of images,
• • at least one control command is generated by the evaluation device in accordance with the recognized gesture and the control command is transmitted to the controller,
• • the control outputs the control signals depending on the control command.

• • eine Steuerung, die ausgestaltet ist, fortlaufend Steuersignale zur Steuerung des Betriebes des Roboters oder der Werkzeugmaschine an zumindest eine zu steuernde Einrichtung auszugeben und mit einer ersten Frequenz über einen Befehlseingang empfangene Steuerbefehle auszuwerten und bei der Erzeugung und Ausgabe der Steuersignale zu berücksichtigen,
• • zumindest eine Kamera, die ausgestaltet ist, fortlaufend Körperteile eines Nutzers aufzunehmen, so dass mit einer zweiten Frequenz, die größer ist als oder zumindest gleich ist wie die erste Frequenz, eine zeitliche Folge von Bildern zumindest einer Geste des Nutzers erzeugt wird, und
• • eine Auswertungseinrichtung zur Auswertung der zeitlichen Folge von Bildern, wobei die Auswertungseinrichtung ausgestaltet ist, die zeitliche Folge von Bildern gemäß einem der vorhergehenden Verfahrensansprüche auszuwerten und zumindest einen Steuerbefehle zu erzeugen und zu der Steuerung zu übertragen, sodass die Steuerung die Steuersignale abhängig von dem Steuerbefehl ausgibt.

A robot or a machine tool is therefore also proposed, having:

• • a controller that is designed to continuously output control signals for controlling the operation of the robot or the machine tool to at least one device to be controlled and to evaluate control commands received via a command input at a first frequency and to take them into account when generating and outputting the control signals,
• • At least one camera that is designed to continuously record body parts of a user so that a time sequence of images of at least one gesture by the user is generated at a second frequency that is greater than or at least equal to the first frequency, and
• An evaluation device for evaluating the temporal sequence of images, the evaluation device being designed to evaluate the temporal sequence of images according to one of the preceding method claims and to generate at least one control command and transmit it to the controller, so that the controller uses the control command as a function of the control signals issues.

Refinements of the method in relation to the operation of a CMM have already been described above. Corresponding refinements of the method for operating the robot or the machine tool result from this. Furthermore, refinements of the robot or the machine tool result from this, since the method can be carried out in particular using the evaluation device and control of the robot or the machine tool. This also applies to the configurations which will be described as examples using the attached figures.

• 1 schematisch ein Koordinatenmessgerät in Portalbauweise,
• 2 schematisch eine Hand eines Nutzers im Erfassungsbereich eines Paars von Kameras, wobei die von der Hand ausgeführte Geste von den Kameras erfasst wird, um eine zeitliche Folge von dreidimensionalen Bildern zu erzeugen,
• 3 ein Flussdiagramm, das einen Ablauf bei der Aufnahme von Kamerabildern in die zeitliche Folge von Bildern, bei der wiederholten Auswertung von Bildern zur Gestenerkennung und bei der Ausgabe entsprechender Steuerbefehle darstellt,
• 4 ein Flussdiagramm eines Verfahrensablauf, bei dem zur Steuerung eines Betriebszustandes des Koordinatenmessgerätes eine wiederholte Auswertung von Bildern stattfindet,
• 5 schematisch die Ermittlung eines Bewegungsvektors aus zumindest zwei Bildern der zeitlichen Folge von Bildern und einen Verfahrensablauf für die wiederholte Erzeugung von Steuerbefehlen zur Steuerung der Bewegung eines beweglichen Teils des Koordinatenmessgerätes,
• 6 eine schematische Darstellung einer Mehrzahl von Bildern, die in einem Ringspeicher gespeichert werden und fortlaufend aktualisiert sowie ausgewertet werden,
• 7 schematisch eine Mehrzahl von Gelenken und die Gelenke verbindende Knochen eines Nutzers in einer ersten Anordnung und
• 8 die Mehrzahl von Gelenken und Knochen des Nutzers aus 7 in einer zweiten Anordnung, die eine Geste des Nutzers darstellt.

Further configurations and exemplary embodiments of the invention will now be described with reference to the accompanying drawings. The individual figures in the drawing show:

• 1 schematically a coordinate measuring machine in portal design,
• 2 schematically a hand of a user in the detection range of a pair of cameras, the gesture performed by the hand being detected by the cameras in order to generate a temporal sequence of three-dimensional images,
• 3 a flowchart that shows a sequence for the recording of camera images in the time sequence of images, for the repeated evaluation of images for gesture recognition and for the output of corresponding control commands,
• 4th a flowchart of a process sequence in which a repeated evaluation of images takes place to control an operating state of the coordinate measuring machine,
• 5 schematically the determination of a movement vector from at least two images of the time sequence of images and a process sequence for the repeated generation of control commands for controlling the movement of a movable part of the coordinate measuring machine,
• 6th a schematic representation of a plurality of images that are stored in a ring buffer and continuously updated and evaluated,
• 7th schematically a plurality of joints and bones connecting the joints of a user in a first arrangement and
• 8th the majority of the user's joints and bones 7th in a second arrangement, which represents a gesture by the user.

This in 1 Coordinate measuring machine shown (CMM) 211 in portal design is just one example of a CMM. It has a measuring table 201 on, above the pillars 202 , 203 are movably arranged in the Y direction of a Cartesian coordinate system. The columns 202 , 203 form together with a cross member 204 a portal of the KMG 211 . The cross member 204 is at its opposite ends with the pillars 202 or. 203 connected. Electric motors, not shown in detail, act as drives and cause the linear movement of the columns 202 , 203 in the Y direction, along the axis of movement which runs in the Y direction. It is z. B. each of the two pillars 202 , 203 assigned to an electric motor. The cross member 204 is with a cross slide 207 combined, which z. B. air bearings along the cross member 204 is movable in the X direction of the Cartesian coordinate system. The current position of the cross slide 207 relative to the cross member 204 can be based on a scale division 206 to be established. The movement of the cross slide 207 along the movement axis in the X direction, is driven by at least one further electric motor (not shown). On the cross slide 207 is a quill movable in the vertical direction (Z-direction) 208 stored at its lower end via a mounting device 210 and a rotating device 205 with a change interface 209 is connected to which a sensor 215 is coupled via an angled bracket. The turning device 205 with the change interface 209 can be driven by another electric motor relative to the cross slide 207 be moved in the Z direction, along the Z movement axis, of the Cartesian coordinate system. The KMG's electric motors can connect the 209 coupled sensor 215 in the area below the cross member 204 can be moved to almost any position. Furthermore, the rotating device 205 the sensor 215 Rotate around the Z axis so that the white light sensor 215 can be aligned in different directions. Alternatively, a rotating device can be used instead of the rotating device 205 can be used, which allows other degrees of freedom of movement, for example an additional rotational mobility about an axis of rotation that is perpendicular to the vertical (Z-direction). There is a workpiece to be measured on the measuring table 217 placed.

A controller is also shown 220 that the measurement signals of the sensor 215 receives and controls the operation of the CMM, especially the electric motors (ie the drives). The connection 230 the control 220 to the drives is shown schematically by a double line. When connecting 230 For example, it can be a light pipe such as a fiber optic cable.

Furthermore, in 1 schematically an evaluation device 222 of the KMG 211 shown by evaluating a in an image memory 221 Stored sequence of images recognizes gestures of a user, generates corresponding control commands and the control commands to the controller 220 transmits.

2 shows a hand 17th a user of the KMG, e.g. B. the in 1 CMM shown 211 . The hand 17th is in the detection range of a pair of digital cameras 15a , 15b that continuously record stereo images (and thus three-dimensional image information) of the detection area. The cameras preferably capture 15th not just the in 2 illustrated hand 11 , but also other groups of body parts of the user and in particular the entire user.

An image processing device 16 that is part of the evaluation facility 222 (e.g. the in 1 shown), created continuously from the image signals transmitted to it from the cameras 15th three-dimensional camera images. Gestures made by the user in particular with the hand 11 are captured by the chronological sequence of the three-dimensional camera images.

Unlike in 2 shown, the time sequence of images can be recorded by a single camera, which in addition to the two-dimensional image information for each pixel of the Image Generated distance information to the objects and body parts shown, so that a three-dimensional image of the scene recorded by the camera is generated. In this case, the image processing device 16 omitted.

2 shows as parts of the evaluation device 222 also a computing device 25th and an image memory 221 (e.g. the in 1 shown), in the case shown, via the image processing device 16 with the cameras 15th connected is. Also is the image memory 221 with the computing device 25th connected. The ones from the cameras 15th Images of the user recorded (or alternatively by the individual camera) are stored as a time sequence of images in the image memory 221 saved. The computing device 25th has access to the image memory 221 . On the one hand, it can use this to check by preprocessing whether individual images contain sufficient information for gesture recognition and, if necessary, to remove unsuitable images from the sequence of images. On the other hand, the computing device determines 25th by evaluating the sequence of images, at least one currently valid gesture by the user repeats and repeatedly issues a control command to the controller in accordance with the at least one recognized gesture 220 (e.g. the in 1 shown) of the KMG. It is possible, but not necessary, depending on the configuration of the method, that the computing device 25th outputs a control command as a result of each repetition of the evaluation of the sequence of images and / or after each recognition of a gesture. Rather, it is also sufficient to issue a control command only when a result of the evaluation interferes with the operation of the controller 220 indexed. For example, this is the case if, according to the last recognized gesture or according to the last recognized change in a gesture, a start of an operating mode of the CMM, an intervention in an ongoing operation of the CMM or a stop or change in an operating mode of the CMM is to be undertaken.

3 shows a flow chart with steps and decisions that are predominantly carried out by the evaluation device. The process begins with the start 31 the operation of the evaluation facility. In a following step 33 the evaluation device or a data storage device connected to it receives at least one camera image from the camera 15th . Either the at least one camera image is added directly to the sequence of images that forms the basis for gesture recognition as the latest, current image, or it is first checked whether the image is suitable for gesture recognition.

In a following step 35 an optional preprocessing of the at least one current camera image and / or a plurality of camera images including the at least one current camera image is carried out. If it is determined in the preprocessing that at least one image (in particular the current image) is unsuitable, then the unsuitable image is removed from the time sequence. Alternatively, the at least one inappropriate image is not added to the sequence. Is that at least one in the preprocessing according to step 35 checked image is suitable for gesture recognition, it either remains in the temporal sequence or is added to the temporal sequence.

In the next step 37 gesture recognition takes place. In a first case, only the at least one current camera image is evaluated and, in another case, a plurality of the camera images that are contained in the temporal sequence of images. The example of the ring memory has already been explained above. In this case, the gesture recognition can step 37 refer to all images in the ring buffer.

step 37 some repetitions of the execution of the in 3 illustrated loop of process steps 33 to 39 do not apply, namely if no new results are to be expected compared to a previously executed gesture recognition (e.g. because no usable current image has been added) and / or if it is certain that no control command is issued to the control of the CMM at the end of the loop . This can e.g. B. be the case when the recording of the camera images and thus also the execution of the loop in 3 takes place at a much higher frequency than the control can receive and process control commands. But even in the case of the much higher frequency, it is preferred to carry out the gesture recognition with this high frequency or at least with a higher frequency than the control can receive and process control commands. But it is possible, for. B. several times the step 33 and optionally the step 35 and then in a single step 37 to take into account several new camera images that have been added since the last gesture recognition in the gesture recognition.

In step 39 it is decided whether a control command is sent to the controller 220 of the KMG is issued. In particular, a control command is not issued when the control 220 since the last control command is not yet able to receive and process another control command. Alternatively or additionally, in step 39 but it can also be decided not to send a control command to the controller 220 output if the gesture recognition does not require output of a control command. For example, it may have been determined by the gesture recognition that the user is currently not making a gesture. Even in this case, however, a control command could still be issued if this means that the operation of the control must be influenced because a gesture was not displayed by the user or because no gesture could be captured by camera images due to a fault. An example of such a case is the continuous movement control through gestures by the user. If in this case no more gestures are detected, then step 39 decided to send a corresponding control command to the controller 220 output as soon as it is able to receive the control command and process it either immediately or later (e.g. with the next working cycle of the controller).

On step 39 if the evaluation device continues to operate, step follows again 33 , ie a new loop of the method steps carried out by the evaluation device begins.

4th shows a flow chart of a method sequence which is carried out by the evaluation device. In a first step 41 is by evaluating at least one image of the time sequence of images in the data memory 221 are saved, a gesture is recognized. If this is the first time this gesture is recognized, e.g. B. since the elapse of a period of time of predetermined length, and if this gesture is assigned to an operating mode of the CMM, the following step 49 decided that a control command to start this operating mode to the controller 220 is issued.

In particular, each time a new, current image has been added to the temporal sequence of images and optionally also when preprocessing has shown that this image is suitable for gesture recognition, the method continues with a renewed execution of step 41 away. In the next step 49 it is then checked whether it is the same gesture as previously recognized. If this is the case, either no new control command is sent to the controller 220 output or a control command is output which confirms the continuation of the started and current operating mode. Optionally, the control 220 in the absence of such a confirming control command, decide that the operating mode is ended. A time span of a predetermined length can optionally be defined. If there is no confirming control command from the evaluation device to the controller within this period of time 220 is transferred, the operating mode is determined by the controller 220 completed.

5 shows a chronological sequence of images in the illustration above. Two of the images are each marked with a rectangle and the reference number 51a or. 51 n denotes. The points between the images and to the right and left of the images 51a , 51 n symbolize that there can be more than these two images in the temporal sequence of images. The images are arranged in their chronological order of the recording period by the respective camera. However, it is also possible for images to be recorded by more than one camera at the same or to overlap recording time periods and to be inserted into the temporal sequence of images.
For example, two images can each be recorded at the same recording period. These are z. B. the pair of images from two cameras that record a scene from different viewing directions. Therefore, three-dimensional image information can be obtained from these two images. Either the two images are saved as a pair of images in the sequence or a three-dimensional image that was obtained from them.

By one at a time from the picture 51a and from the picture 51 The arrow pointing outwards indicates that the two images recorded at different recording time periods, that is to say one after the other 51a , 51 n by the evaluation device in one step 52 be evaluated. A motion vector of the movement of body parts of the user is determined. Optionally, the motion vector can be determined from more than two images in the sequence of images. In particular, the movement vector is determined from the movement of the same group of body parts of the user that took place between the two or more images. For example, the right hand of the user moves from the recording period of the picture 51a the period when the picture was taken 51 n from a first location to a second location. The motion vector has the direction of the straight line connection from the first location to the second location and the length of the distance between these two locations, which are e.g. B. can be the location of the center of mass (assuming a uniform local mass distribution) of the group of body parts. Alternatively, the location can be another location that can be clearly identified from the body parts and is preferably particularly reliably identifiable, such as. B. the intersection of two longitudinal axes of the bones of the group of body parts or the center of mass of a joint of the group of body parts.

In an optional step 53 who after the step 52 , before the step 52 and / or simultaneously with the step 52 or as part of the step 52 can be carried out, the evaluation device determines a further motion vector of a second group of body parts of the user from the same images. The second group of body parts is e.g. B. around the shoulder of the User from which the user's arm extends, which leads to the hand, which in turn forms the first group of body parts. As already described, the second motion vector can be subtracted from the first motion vector by forming the vector difference. The resulting vector of the result of the difference formation is in the case of the execution of the optional step 53 output as a result motion vector and in the following step 54 considered. In the other case if the optional step 53 is not executed, the resulting motion vector is the first motion vector.

In step 54 is decided, e.g. B. in a corresponding manner as to 3 it was explained whether a control command to the controller 220 is issued. When the control command is issued, it contains the information about the direction and length of the resulting motion vector. The control 220 generates control signals therefrom, which cause at least one drive of the CMM to bring about a movement of a movable part of the CMM that corresponds to the result movement vector. For example, in the step 54 and / or from the controller 220 the time base of the result motion vector must also be taken into account. In step 54 this means that the time base, in particular the time period between the recording periods of the evaluated images of the time sequence of images, is sent to the controller as information 220 is issued. In the case of the controller 220 This means that the controller generates the control signals in such a way that the movement of the moving part of the CMM is carried out at a speed which corresponds to the length of the resulting movement vector and the time base, i.e. in particular the length divided by the time difference of the time base. The time base can be set in the step 52 be determined in a simple manner that, for. B. the starting times of the recording periods of the two or more evaluated images are determined, z. B. by reading the corresponding information from the data memory in which the chronological sequence of images is stored.

In a combination of the procedures described in 4th and 5 are shown, the determination of the result motion vector can take place repeatedly and it can also be determined whether in successive loops of the execution of the steps 52 , optionally 53 and 54 the same gesture by the user from the images, in particular from all evaluated images (in the case of the example in 5 so the two pictures 51 a, 51 n), is recognized. The control command to continue the movement of the moving part of the CMM is given in step 54 only then to the controller 220 output if compared to the previously executed loop of the method steps that correspond to the control command last output in step 54 led, the same gesture was still recognized from the evaluated images. For example, the fist formed by the user was recognized as a gesture from all the images on which the evaluations were based in the successive or several loops. When the gesture is in a loop of steps 52 to 54 (with optional step 53 ) or was no longer recognized in several such loops is shown in step 54 decided that a control command to stop the movement of the moving part of the CMM was sent to the controller 220 is issued and this command is issued. The control 220 then stops the movement of the moving part. Alternatively, instead of a stop command, the command to move the movable part into a predefined, so-called zero position can be output.

If the same gesture was recognized from the evaluated images in the consecutive or several loops, but the resulting motion vector has a length of zero or the length of the resulting motion vector according to another embodiment is less than a predetermined limit value of the length (minimum length), then will be in step 54 decided to send a control command to the controller 220 output from the controller 220 causes corresponding control signals to be output, which in turn lead to the continuation of the movement of the movable part at the same speed and in the same direction. For example, in step 54 The control command outputted may be the same control command that was last output when the length of the result motion vector was still greater than zero or, according to the other exemplary embodiment, was greater than or equal to the minimum length. In particular, with the control command, the information about the same length and the same direction of the resulting motion vector as before is sent to the controller 220 issued.

Above the criterion was mentioned that the determined result motion vector must have at least a length that is greater than or equal to the minimum length in order to be used as a new result motion vector for the generation of a new control command. Alternatively, in the case in which a minimum length is specified, a modified criterion can consist in that the resulting motion vector has a length which is greater than the minimum length.

6th shows a special embodiment of the image data memory 221 as a ring memory in which a plurality of images (in the exemplary embodiment six images 61 to 66 ) are stored for the simultaneous recognition of a gesture or several gestures. By arrows within the image data memory 121 it is suggested that the pictures 61 to 66 were recorded one after the other in a chronological order. The information about the The recording period or a corresponding recording time (for example the beginning of the recording period) is preferably also in the image data memory 221 saved. As a result, when evaluating the images, it can be determined which of the stored images is the first in the time sequence of images, which of the stored images is the second in the time sequence of images, and so on 6th The condition shown is the picture 61 among all currently stored images, the image with the earliest recording period. Optionally, a plurality of images can be recorded simultaneously or in an overlapping manner and stored in the ring memory. Instead of six storage locations for images, the ring buffer can have any other number of storage locations, but in particular a number greater than three storage locations. In practice, it has been proven that the ring memory has ten storage locations and the up to ten stored images are used simultaneously for gesture recognition and / or generation of a control command.

Right in the 6th is a camera 15th shown schematically. From the camera 15th Recorded images are repeatedly stored in the ring memory. In the following, it is assumed that the recorded camera images are preprocessed and a decision is made as to whether the recorded image is suitable for gesture recognition. Only in the case of a suitable image is this stored in the ring memory. Alternatively, the preprocessing could be carried out after the storage of the respective image and a decision could then be made to delete the image again from the memory without it being used for the gesture recognition.

With the ring memory it is also possible in particular to store images that are not suitable for gesture recognition without preprocessing and to evaluate them together with other images for gesture recognition. It is assumed, for example, that more suitable than unsuitable images are recorded and stored. If, in particular, all images stored in the ring memory (for example, equivalent) are used for gesture recognition and / or the determination of a movement position of body parts to determine a movement vector, good results can nevertheless be achieved in the control.

When the camera 15th If a new, more up-to-date image is generated than is stored in the image data memory forced 21, this is optionally stored in the ring memory after preprocessing. If all the storage locations in the ring buffer are already occupied, the oldest image (or in the case of several cameras with simultaneous recording periods: the oldest images) is replaced by the new image (s). In 6th this is indicated by three arrows pointing from the camera 15th and lead to different images in the ring buffer. However, in the in 6th state represented by a new camera image is not one of the images 62 , 63 or 64 replaced, but picture 61 as this is the oldest picture.

For example, after each insertion of an image or several simultaneously recorded images into the ring memory, all images in the ring memory can be evaluated, in particular for gesture recognition and / or for determining a movement position of body parts of the user in each of the images. The latter in turn enables the formation of several position differences and thus several motion vectors. The evaluation is indicated by a broad arrow pointing down below the image data memory 121 indicated.

7th shows schematically a plurality of joints 70 to 75 of a user, each joint being connected to a further joint via at least one bone or an arrangement of bones. The joints are each represented by a circle and the bones are each represented by a straight line. This arrangement of joints in particular becomes from a camera image 70 to 75 especially taking into account the connections between the joints due to the bones shown 70 to 75 determined.

The joints are as follows: right sternum-collarbone joint 70a , left sternum-collarbone joint 70b , right shoulder joint 71a , left shoulder joint 71b , right elbow joint 72a , left elbow joint 72b , right wrist 73a , left wrist 73b and spinal joint 75 at the base of the skull.

This is only an example of a number of joints that can be determined from camera images of the user. In particular, a number of joints of the user's hands and / or the entire spine, the head and / or the hips and legs of the user can also be used for gesture recognition in practice.

In particular, a plurality of joints can be defined, which must be determined from a camera image so that it is decided that the camera image is a camera image suitable for gesture recognition. In the example of the 7th for example the joints 71a , 71b , 72a , 72b his. To identify the shoulder joints 71 It may be necessary in particular to identify the bone of the sternum-clavicle joint 70 to the shoulder joint 71 runs. If one of the four absolutely necessary joints is not identified from a camera image, it can be decided in particular as part of the preprocessing already mentioned that the camera image is not suitable for gesture recognition and is therefore not evaluated. Since the 7th If it is only an example of a number of joints, in practice other joints can be defined as mandatory for suitability for gesture recognition.

Alternatively or additionally, a number of joints can be defined. In this case, it is specified that it may not be any joints of the user, but rather certain, predetermined joints, for example all in 7th illustrated joints. During the preprocessing it can be determined whether a predetermined minimum number of joints, which can be any of the determined, predetermined joints, has been determined from the respective camera image. If the minimum number is reached, it is decided that the camera image is used for gesture recognition. Otherwise it is decided that the camera image is not used.

The two previously described examples of preprocessing can be combined with one another and / or modified. For example, instead of at least one of the joints, at least one bone can be specified which must have been determined from the camera image in order to use the camera image in the evaluation. This automatically includes in the event that a plurality of predetermined bones must have been determined.

8th shows the same bones and / or joints as in 7th which, however, were determined from a different camera image when the user was in a different posture. The one from the respective elbow joint 72a , 72b to the respective wrist 73a , 73b Running bones cross each other in this posture, with the middle areas of the bones crossing and the wrists 73 above the elbow joints 72 are positioned. In a specific embodiment, the evaluation device recognizes a corresponding, predefined gesture therefrom.

Claims (11)

A method for controlling an operation of a coordinate measuring machine (211), wherein A controller (220) of the coordinate measuring machine (211) which continuously outputs control signals for controlling (220) the operation of the coordinate measuring machine (211) to at least one device to be controlled of the coordinate measuring machine (211) and evaluates control commands received via a command input at a first frequency and taken into account in the generation and output of the control signals, • Body parts (70-75) of a user of the coordinate measuring machine (211) are continuously recorded by at least one camera (15), so that with a second frequency that is greater than or at least equal to the first frequency, a time sequence of Images of at least one gesture of the user are generated, • the sequence of images (51a, 51n) is made available to an evaluation device (222) and the gesture is recognized by the evaluation device (222) from the temporal sequence of images (51a, 51n), • at least one control command is generated by the evaluation device (222) in accordance with the recognized gesture and the control command is transmitted to the controller (220), • the controller (220) outputs the control signals as a function of the control command. Procedure according to Claim 1 , the evaluation device (222) repeatedly determining from the time sequence of images (51a, 51n) whether the recognized gesture is the same gesture that has already been repeated previously during a still ongoing period of a current operating state of the coordinate measuring machine (211) from the temporal sequence of images (51a, 51n) was recognized, and the evaluation device (222) outputs a control command to terminate the current operating state of the coordinate measuring machine (211) to the controller (220) if the same gesture is no longer recognized. Procedure according to Claim 2 , the evaluation device (222) recognizing a gesture at the beginning of the period and issuing a control command to start the current operating state. Method according to one of the Claims 1 - 3 , wherein the evaluation device (222) repeatedly determines a motion vector of a movement of body parts (70-75) of the user from at least two images (51a, 51n) of the temporal sequence of images (51a, 51n), and the evaluation device (222 ) repeatedly outputs a control command for executing a movement of a movable part of the coordinate measuring machine (211) corresponding to the movement of the body parts (70-75). Procedure according to Claim 4 wherein the evaluation device (222) determines the motion vector in each case as a motion vector of a first group of body parts relative to a second group of body parts. Procedure according to Claim 4 or 5 , wherein the evaluation device (222) continues to control commands to continue the movement of the Movable part of the coordinate measuring machine (211) outputs when the same gesture is still recognized as before from the temporal sequence of images (51a, 51n), although the determined motion vector has a length equal to zero or less than a predetermined limit value. Method according to one of the Claims 4 - 6th , the evaluation device (222) continuously determining a currently valid motion vector of the movement of body parts (70-75) of the user from a plurality of at least three images of the temporal sequence of images (61-66), the plurality of images being recorded one after the other contains the last recorded image of the at least one camera (15) and does not contain more than a predetermined number of images. Method according to one of the Claims 1 - 7th wherein the evaluation device (222) recognizes a currently valid gesture from a plurality of images (61-66) of the temporal sequence of images, each of the plurality of images (61-66) being recorded during a different time interval. Procedure according to Claim 8 , the evaluation device (222) continuously recognizing a currently valid gesture from a plurality of the images (61-66) of the temporal sequence of images, the plurality of images (61-66) being recorded one after the other, the last recorded and the temporal Sequence of inserted image (66) which contains at least one camera (15) and does not contain more than a predetermined number of images. Method according to one of the Claims 1 - 9 , wherein the evaluation device (222) tries from each image of the sequence of images (51a, 51n) to recognize a plurality of predetermined body parts (70-75) of the user, and from the recognized body parts (70-75) on the basis of a predetermined criterion decides whether there is sufficient information in the respective image for recognizing gestures by the user, the respective image not being taken into account for recognizing gestures by the user if it is decided based on the specified criterion that there is insufficient information in the respective image is. Having coordinate measuring machine A controller (220) which is designed to continuously output control signals for controlling (220) the operation of the coordinate measuring machine (211) to at least one device to be controlled of the coordinate measuring machine (211) and to evaluate control commands received via a command input at a first frequency and at take into account the generation and output of the control signals, • At least one camera (15) which is designed to continuously record body parts (70-75) of a user of the coordinate measuring machine (211), so that a second frequency that is greater than or at least equal to the first frequency, a temporal A sequence of images (51a, 51n) of at least one gesture by the user is generated, and • an evaluation device (222) for evaluating the time sequence of images (51a, 51n), the evaluation device (222) being designed to evaluate the time sequence of images (51a, 51n) according to one of the preceding method claims and to generate at least one control commands and to transmit to the controller (220) so that the controller (220) outputs the control signals in response to the control command.

Images