DE102022205569A1 - Device and method for recording a user's muscle activity using optically pumped magnetometers - Google Patents

Abstract

The invention relates to a sensor device for detecting muscle activity using magnetic field sensors, comprising at least one optically pumped magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d), which has at least one sensor medium (101, 102, 103, 104). optically excitable defect centers and is designed to detect a magnetic field signal at a measuring location (110, 112) by reading out a spin resonance in the sensor medium that is dependent on the magnetic field strength, a holder (260, 262) for attachment to a part of the user's body, wherein The at least one optically pumped magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d) is arranged in the holder (260, 262) in such a way that the magnetometer unit can detect a magnetic field caused by electrical activity during a muscle movement of the user; a processing unit which is set up to extract signal features of the magnetic field signal in time windows of a predetermined length; and a processing unit (136) which is set up to classify the extracted signal features of one or more time windows into two or more classes using at least one classification model and to output a classification result for the magnetic field signal in the one or more time windows.

Description

The present invention relates to a device and a method for detecting a user's muscle activity using optically pumped magnetometers, as well as a computing unit and a computer program for carrying out the method.

Background of the invention

The interpretation of neuromuscular activity, i.e. the detection and processing of signals caused by muscle movements and the associated nerve signals, is of interest in many areas. Electromyographic measurement methods (EMG) are known for this. This measures the secondary electrical activity that accompanies or precedes the contraction of muscle fibers. The main source of such electrical activities are action potentials along the membrane of muscle fibers. While needle electrodes or wire electrodes are usually used in the diagnostic field, which are inserted through the skin into the muscle tissue, surface electrodes can also be used for non-diagnostic purposes, which derive the electrical signals on the skin surface.

However, it is necessary to measure the electrical potentials via direct skin contact, so that the surface electrodes, for example, have to be glued to the skin at suitable locations. This method is therefore only suitable for use if, for example, medical personnel are available to attach the measuring electrodes. In addition, with conventional electromyographic measurement methods, only a very rough resolution of the electrical activity actually generated by the various muscle groups and nerves is possible; The surface signal is then composed of all electrical signals in the surrounding areas, so that the signals cannot be further resolved. In addition, the electrical signals recorded on the skin surface are distorted by the intervening tissue and different conduction conditions.

For the highly sensitive measurement of magnetic fields, which can be induced, for example, by electrical currents such as brain waves, optically pumped magnetometers have recently been proposed, such as those in DE 10 2018 203 845 A1 or DE 10 2020 207 200 A1 are revealed. These are based on measuring optically excitable and readable resonance frequencies on crystal defects that change depending on an existing magnetic field. In particular, such magnetometers can be realized based on diamonds with nitrogen vacancies (nitrogen vacancy, NV).

Disclosure of the invention

According to the invention, a device and a method for detecting a user's muscle activity as well as a computing unit and a computer program for carrying out the method are proposed with the features of the independent claims. Advantageous refinements are the subject of the subclaims and the following description.

In particular, a sensor device for detecting muscle activity using magnetic field sensors is proposed, which comprises at least one optically pumped magnetometer unit, which comprises at least one crystalline sensor medium with optically excitable defect centers and is set up to generate a magnetic field signal at a measuring location by reading out a spin resonance which is dependent on the magnetic field strength to detect the sensor medium. The sensor device also comprises a holder for attachment to a part of the body of a user, wherein the at least one optically pumped magnetometer unit is arranged in the holder so that the magnetometer unit can detect a magnetic field that is induced by electrical activity during a muscle movement of the user. In addition, the sensor device comprises a processing unit which is set up to extract signal features of the magnetic field signal in time windows of a predetermined length; and a processing unit which is set up to classify the extracted signal features of one or more time windows into two or more classes using at least one classification model and to output a classification result for the magnetic field signal in the one or more time windows. By using optically pumped magnetometer units, even very weak biomagnetic fields can be measured. With such a device, electrical signals that are triggered in a user's body by muscle activity can be detected indirectly via the weak magnetic field signals induced by the electrical signals. These signals can then be classified to draw conclusions about muscle activity, e.g. movements, gestures or moving muscle groups. In contrast to previously known methods, it is not necessary for the magnetic field sensors to be attached with direct skin contact.

The magnetometer unit can optionally be further set up to detect an orientation of the magnetic field at a measuring location, in which case the processing unit can further be set up to also take into account the orientation of the magnetic field for extracting signal features and/or for classifying the signal features. This allows a more precise localization of the field source, ie the location from which the electrical activity originates.

In exemplary embodiments, the one or more magnetometer units may be fixed at predetermined locations in the holder, the holder comprising at least one of the following: a garment, a band for attachment to a body part, a glove. For example, a bracelet can be used in which one or more magnetometer units are distributed at regular or irregular intervals along the band. The distances and locations can be specified in such a way that the muscle groups suitable for evaluation lie as close as possible to the respective magnetometer unit when carrying the holder.

It is also possible for the magnetometer unit to comprise at least two sensor media which are arranged at a predetermined distance from one another in a gradiometric configuration, wherein the magnetometer unit is set up to detect the magnetic field signal at the measuring location by combining the magnetic field signals of the at least two sensor media without background fields . As far as the background fields are large and relatively homogeneous, it can be assumed that they are identical in both sensor media; This allows the field gradient of a local magnetic field of interest to be determined, for example, as the difference between the two signals. In the case of inhomogeneous background fields, it can also be useful to use more than two sensor media within a magnetometer unit and evaluate them in combination in order to eliminate the background fields.

In particular, the magnetometer unit can comprise a nitrogen vacancy center magnetometer, wherein the sensor medium comprises a diamond crystal or a section of a diamond crystal with nitrogen vacancy centers. Especially for gradiometric magnetometer units, very compact magnetometer units can be realized by using several sections of the same diamond crystal. As an alternative to diamond crystals with nitrogen vacancies, other color centers in diamond or in other semiconductors can also be used, which allow optical excitation and readout of magnetic field-dependent spin resonances, such as various point defects in silicon carbide.

In addition, a method for detecting muscle activity of a user is proposed, which first comprises detecting a time profile of a magnetic field signal at at least one measuring location on a part of the user's body by at least one optically pumped magnetometer unit, wherein the at least one magnetometer unit has at least one sensor medium with optically excitable Comprises defect centers and is set up to detect at least one magnetic field strength by reading out a spin resonance in the sensor medium that is dependent on the magnetic field strength. Signal features are then extracted from the magnetic field signal in time windows of a predetermined length, which in turn can be entered into (at least) one classification model for one or more time windows. The classification model is set up to classify the signal features into two or more classes, so that a classification result for the magnetic field signal in the one or more time windows is then obtained.

The two or more classes can be defined, for example, by one or more of the following target variables: a muscle group, a muscle movement, a movement sequence, a nerve activity, a resting state, a time course of a muscle movement. A classification model can thus be provided in such a way that it directly evaluates a movement to be recognized, e.g. a gesture intended as a control command; Alternatively or additionally, more basic evaluations such as recognizing the muscle groups involved in a movement can also be carried out.

It is also possible for magnetic field signals to be recorded at several different measurement locations on parts of a user's body and then the signal characteristics of the several different measurement locations to be combined and entered into the classification model. The different measuring locations can be provided relatively close to one another, for example at different points on one hand, but also further apart (e.g. on the left and right arms, or on the hand and upper arm) with several suitable holders.

The at least one classification model can include, for example, a statistical classification method, a threshold check and/or a neural network or even suitable combinations of the same or different classification models.

The method can also include transmitting the extracted signal features and/or a classification result via a communication connection, for example to an external processing unit or to a device, which ches should be controlled using the detected signals.

Accordingly, the method can also include controlling a device based on the classification result obtained, such as a gesture control as an input interface for any device, a control of a robot or a similar actuator, or a control of a prosthesis on the body.

A sensor device as described above is particularly suitable for carrying out the method described above or its modifications and further developments.

The optically pumped magnetometers described here offer the advantage over classic (e.g. SQUID-based) magnetic field sensors that no complex cryogenic cooling of the sensors is required and a smaller distance to the signal source is possible, i.e. the sensors can be arranged very compactly on the body. Compared to conventional electromyographic methods with surface recording, a higher spatial resolution and thus a better distinction between different muscle activities can be achieved. At the same time, no direct skin contact and no invasive intervention are required. The magnetic field is hardly influenced by the tissue in between, so that a better signal-to-noise ratio can be achieved than in electromyographic methods. The magnetometers described can also be used to enable highly sensitive measurements in environments with strong magnetic background fields such as the earth's magnetic field, so that the systems described can also be used without any problems in everyday environments.

A computing unit according to the invention is set up, in particular in terms of programming, to carry out a method according to the invention.

The implementation of a method according to the invention in the form of a computer program or computer program product with program code for carrying out all method steps is also advantageous because this causes particularly low costs, especially if an executing control device is used for additional tasks and is therefore present anyway. Finally, a machine-readable storage medium is provided with a computer program stored thereon as described above. Suitable storage media or data carriers for providing the computer program are, in particular, magnetic, optical and electrical memories, such as hard drives, flash memories, EEPROMs, DVDs, etc. It is also possible to download a program via computer networks (Internet, intranet, etc.). Such a download can be wired or wired or wireless (e.g. via a WLAN network, a 3G, 4G, 5G or 6G connection, etc.).

Further advantages and refinements of the invention result from the description and the accompanying drawing.

The invention is shown schematically in the drawing using exemplary embodiments and is described below with reference to the drawing.

Brief description of the drawings

• 1 shows schematically possible components of a system with a sensor device according to one embodiment;
• 2a shows a possible embodiment of a sensor device as a bracelet;
• 2 B shows a possible embodiment of a sensor device as a glove;
• 3 shows schematically the sequence of a method according to a possible embodiment; and
• 4 shows two exemplary hand movements that can be recognized with embodiments of the invention.

Embodiments of the invention

One or more magnetometer units attached to appropriate locations on a user's body can detect signals of muscle activity in that area. In principle, activities of any muscle groups or parts can be recorded, such as movements of arms, legs, hands, feet, but also muscle groups that are in the torso or head area and can, for example, indicate certain movements. The muscle activity of individual fingers or phalanges can also be recorded, for example, in order to interpret finger gestures.

The devices and methods described here for measuring muscle activity are based on optically pumped magnetometers that enable highly sensitive magnetic field measurements. In particular, the principle described here can be implemented with magnetometers that are based on optically excitable and readable resonances at defect centers or color centers in crystals. As an example, magnetometers that detect nitrogen vacancy centers (nitrogen vacancy, NV) in single-crystalline diamond should be mentioned here based. By optically reading out the magnetic resonance of this color center and using the Zeeman effect, which describes the magnetic field-dependent splitting of the spin states, even very small magnetic fields can be measured at room temperature and with background fields of up to around 100mT, with sensitivities up to 1 pT/√Hz be achieved. Since the NV center in diamond has four possible orientations, a quartet of resonances is obtained for an ensemble of NV centers when an additional static bias magnetic field of, for example, 1mT is applied, from whose frequency position the full vector information of an additional unknown magnetic field is obtained by solving algebraically equations can be determined.

1 shows schematically essential elements of a sensor device for detecting muscle activity. Four individual sensor heads are provided here, which are intended to measure the unknown magnetic fields that emanate from two field sources 110, 112 (eg muscle areas with electrical activity).

A crystal with defect centers, for example a diamond 101, 102, 103, 104 with nitrogen defects, can initially be present as a sensor medium for each magnetometer. The optical excitation of the defect centers can be achieved by a suitable light source 120 such as a pump laser or a laser diode. An Nd:YAG laser in the green range at a frequency of 532 nm is suitable for NV centers in order to excite the respective transitions. The light 124 of the pump laser 120 can be formed into a focused excitation light beam via suitable optical elements 128 such as mirrors, beam splitters, lenses, polarizing elements and/or via fiber optic elements 122, 123 and irradiated into the diamonds. A single light source 120 is used here for all diamonds 101, 102, 103, 104, from which the excitation light is guided into the respective diamond via an optical fiber. In addition, the excitation light can be irradiated continuously by the laser or pulsed, so that, for example, time windows are kept free for interference-free fluorescent light measurement.

In addition, a microwave source 140 can be present for the sensor device, which is capable of generating an alternating electromagnetic field over a bandwidth that covers the desired resonance frequency in the sensor medium, i.e. in the area of the NV centers in the diamonds 101, 102, 103, 104 to generate. For example, a central microwave source 140 can be used, the field of which is irradiated via suitable high-frequency cables 142, 143 and conductor structures into a local microwave resonator (not shown) in the area of the sensor diamonds, so that a homogeneous resonant microwave field is generated at least in the sensor medium in the diamond. Such resonators can also be designed to be very compact, for example as planar resonators on a circuit board. A static bias magnetic field 150, 152 generated in the sensor diamond allows the direction of the detected magnetic field to be determined.

The resulting fluorescent light 125 can in turn be guided over the fiber optics and detected by a photodetector 130, which is sensitive at least in the range of the fluorescence wavelength; Alternatively, one or more photodetectors could also be arranged directly on the sensor diamonds, which receive the fluorescent light. Particularly in the case of several magnetometers or magnetometer units, separate photodetectors (not shown here) can be provided for each sensor head. Here too, suitable reflective or semi-transparent mirrors, filters, beam splitters, lenses for beam shaping, coupling lenses for fiber optics or for the diamond crystals, and other optical elements 126, 128 can be used on the optical path of the fluorescent light 125. The signal from each photodetector 130 can then be further processed by various elements, such as a preamplifier 132 and/or a computing unit 136 for signal evaluation.

The elements of the sensor unit are only shown schematically in the figure; It is understood that, for example, the microwave source is designed and arranged in such a way that the RF field is generated in the sensor diamonds. Also not shown are elements for generating the static magnetic field 150, 152, which can be implemented, for example, as Helmholtz coils. Such coils can then be arranged directly around the sensor diamonds in order to generate fields that are as homogeneous as possible. Other elements such as microwave resonators or housing components are not expressly shown here.

There are also alternative ways to read out the magnetic spin resonance electrically, for example in diamond. Charge carriers are detected that were raised into the conduction band of diamond by two-photon ionization of the NV centers. If such a method is used to read out the resonance effects, the parts for detecting the fluorescent light in the previous examples are not required and are replaced by suitable photocurrent detectors on the diamond. Apart from that, the method for magnetic field measurement can be transferred accordingly and in all embodiments with NV Center magnetometer units can be used.

The basic functionality and other exemplary structures of magnetometers based on NV centers in Diamant can, for example, be found in the registrations DE 10 2018 203 845 A1 , DE 10 2019 212 587 A1 or DE 10 2020 207 200 A1 The contents of which are expressly referred to here for further details. The magnetometer units mentioned below in a sensor device can be designed as described there, but also modified, for example with other variants for coupling the excitation light into the diamond or for detecting the fluorescent light, with different options for generating the microwave field and the static magnetic field at the measuring location in the sensor crystal, with changed arrangements of the various components of a sensor head relative to one another, and/or with modifications in the signal processing. For example, an additional photodetector can be used, which detects a portion of the excitation light beam and serves to eliminate fluctuations in the excitation light from the fluorescence signal using a differential amplification. Various optical elements can also be provided both in the excitation light path and in the fluorescent light path, such as deflecting elements such as mirrors, prisms or beam splitters, optical filters for specific wavelengths, polarizing elements and/or focusing or expanding elements.

In order to be usable in an everyday environment, magnetic fields that do not come from the desired weak source should be eliminated from the measurement as far as possible. For example, in the automotive sector or in the vicinity of electrical machines, comparatively high and barely shielded magnetic fields arise, which are approximately in the range of 10 ^-6 to 10 ^-9 Tesla (Nanotesla). In other applications, too, the earth's magnetic field is always in the range of 10 ^-5 Tesla (some micro Tesla) available. In contrast, the expected signal strengths of the magnetic fields, which correlate with the electrical potentials of muscle activity, are in the range of a few 10 ^-12 Tesla (Picotesla, pT) or even lower.

In order to eliminate these background fields, gradiometric magnetic field measurements are particularly suitable as an exemplary embodiment. Sensor units that are able to record not only the field strength but also the gradient of a field are generally referred to as gradiometers. For this purpose, two or more individual magnetometers 101, 102 or 103, 104 as described above can be used and evaluated simultaneously, each of which is arranged at a defined distance d1, d2 from one another. Two magnetometers then combine to form a gradiometric magnetometer unit. The distance d1, d2 between two sensor heads of a gradiometric magnetometer unit corresponds to the distance between the locations where magnetic field measurements are carried out at the same time. As long as the distance between the measurement locations is relatively small, it can be assumed that the strength of the additional (undesirable) background magnetic field is approximately the same at both locations. In contrast, the weak magnetic field of interest will decrease with increasing distance from the magnetic field source by a factor of 1/r ² . By arranging two sensors at different distances from the source, for example at different distances from the muscle fiber, the disturbing background field can be eliminated by forming a difference in the recorded sensor values and the small magnetic field of interest or its gradient can be extracted: Since the magnetic field weakens with the square of the distance, the largest magnetic field change is detected by the sensor near the source. For this purpose, for example, two magnetometers can be arranged one above the other in an axial gradiometer configuration and together form a gradiometric magnetometer unit.

The distance d1, d2 of the sensor heads within a gradiometric magnetometer unit, i.e. in this case the distance between the two sensor crystals 101, 102 and 103, 104 or crystal sections, can be in the range from a few mm to a few cm, for example between 0.1 cm and 2 cm. For values of this magnitude, good results could be achieved for measuring biomagnetic fields, which are in the picotesla range. As long as the signals are sufficiently independent of the background fields, larger or smaller distances or other gradiometer configurations can also be used, such as sensors arranged next to one another.

Through appropriate evaluation strategies, it is also possible to eliminate a large proportion of interference fields based on their magnetic field direction. If the background fields to be eliminated are not homogeneously distributed in space, it may make sense to use a gradiometric magnetometer unit with more than two individual magnetic field sensors (not shown) to obtain this spatial information. Each sensor head, which is part of a gradiometric magnetometer unit with several sensor heads, can therefore have a sensor crystal with excitable defect centers (or a partial section of a sensor crystal), optical filters, at least one photodetector and, if necessary, a microwave resonator and a coil order to generate a static bias magnetic field. The same light source and the same microwave source are used at least within a gradiometric magnetometer unit in order to achieve excitation conditions that are as identical as possible in the magnetometers of a magnetometer unit. However, a source can be brought to the respective sensor crystals, for example, through two separate resonators or two separate fibers.

For optically pumped magnetic field measurement in sensor crystals, usually only a measuring volume of approximately 1 mm ³ or even less is required in the crystal. This allows, for example, individual sensor crystals to be placed at a correspondingly small distance from one another, or for very small distances it is even possible to use two areas within a single sensor crystal for parallel measurement as a gradiometer. Since the required excitation light can be brought to the sensor crystal via optical fibers or photonic structures in a miniaturized design, and the microwave field can also be guided to the sensor crystal, for example along an optical fiber, a very compact but also flexible design of a sensor head is possible is particularly suitable for applications on the body and in everyday situations.

Such optically pumped magnetometer units can then be attached to a user's body in order to draw conclusions about the electrical activities and thus indirectly about the muscle activity and movement sequences of a user from the measured magnetic fields. It goes without saying that not all components of a complete magnetometer system have to be arranged directly at the measuring location or on the user's body; Essentially, only the sensor heads, i.e. the sensor crystals with appropriate means such as fibers, resonators or coils to generate the required excitation fields, need to be located directly at the measuring location. These elements are referred to here as a sensor device. Cables, optical fibers, signal conductors and other connections then make it possible for the other elements of a magnetometer system (e.g. pump laser, microwave source, signal processing units) to be accommodated away from it. In principle, however, other parts of the magnetometer system can also be integrated into the sensor device, especially if they can be implemented in a compact manner, for example by means of a chip element which includes the photodetector and processing units for processing the detected signals.

One or more magnetometer units should then be placed near the relevant muscle groups for measurement. In order to record certain hand movements, a position on the hand, wrist and/or forearm can be selected, for example. Depending on the nature of the movements that are to be recorded, other positions may be optimal because other muscle groups can be evaluated. The sensor device can include a holder in which one or more magnetometer units are arranged.

To attach the one or more magnetometer units, for example, a holder such as a band 260 can be used, which can be releasably attached (eg with a clasp or as an elastic band) around an arm joint, as shown in FIG 2a is shown. Four magnetometer units 270a, 270b, 270c, 270d are shown here only as an example, which are arranged at different distances along the bracelet 260. However, more or fewer magnetometer units can also be used or arranged at other locations. The individual parts are only shown schematically here; In particular, a sensor head of a magnetometer can be made very small compared to the holder. The magnetometer units can be connected to the required additional elements of a magnetometer system (e.g. light source, power source, microwave source) by suitable supply lines 280 such as optical fibers, high-frequency cables or power supplies. Alternatively, one or more magnetometer units can be placed in a garment such as a sleeve or a glove. For example, as in 2 B a finger glove 262 can be used as a holder, which covers the entire hand or (not shown) only a partial area of the hand. The magnetometer units can then be mounted in the glove in a suitable manner. Here too, four magnetometer units are shown as an example, with three units 272a, 272b, 272c, 272d each being shown in the area of the thumb, index finger and middle finger and another magnetometer unit 272d in the area of the palm. As in the example 2a Suitable supply lines 282 can also be used here to other parts of a magnetometer system such as a light source, power source or microwave field source. Since no direct skin contact is necessary to detect the magnetic field signal, the units can be embedded, for example, between two layers of a piece of clothing or bracelet, or e.g. B. be cast into a plastic material.

The position of the magnetometer units within such a holder 260, 262 can be selected so that the clearest possible signal for a given muscle movement can be achieved. This location information can can be determined at least partially from classic EMG methods and knowledge about the muscle groups used, but can also be further adapted and optimized. For this purpose, for example, tests can be carried out by the manufacturer in which magnetic field signals are recorded and evaluated at different locations along a user's limbs for a single movement sequence, so that the ideal attachment locations for a magnetometer unit can be found.

If a sensor holder such as a bracelet 260 is used, optional elements can be provided on the sensor holder that allow clear positioning relative to certain muscles or body parts of a user; For example, a bracelet could be provided with a finger loop, so that both the position along the arm and the rotational position of the sensor holder are at least roughly predetermined.

As already described, gradiometric magnetometer units consisting of two or more individual magnetometers arranged close to one another and evaluated together can be used as magnetometer units in order to eliminate background fields from the signal. In all of the examples described, a magnetometer unit can in particular comprise two or more individual magnetometers in a gradiometer configuration. This means that several gradiometric magnetometer units can be arranged at different locations in a sensor holder such as a glove or bracelet, each of this large number of gradiometric magnetometer units in turn being able to comprise several individual sensor heads. It goes without saying that “individual sensor heads” each refer to measuring areas in a sensor crystal; However, as already described above, several measuring ranges can also be used within a single sensor crystal, particularly for gradiometric measurements. Of course, a sensor device can also include individual magnetometer units and gradiometric magnetometer units in combination, or different designs of gradiometric magnetometer units, for example at different locations on the body. The microwave source and the excitation light source are also preferably used together at least for all magnetometers of a gradiometric magnetometer unit; However, it is also possible and simplifies the entire structure if the microwave source and the excitation light source are used together for all existing magnetometer units of a sensor device and are then guided through the holders (e.g. in a glove), for example by fiber optics. The signals can also be transmitted to a processing unit via suitable wired or wireless communication links. This processing unit can, for example, directly be the processing unit of a system that is to be controlled with the help of recognized muscle activities, e.g. a vehicle or a computer. Alternatively, a separate processing unit can also be present for evaluating the magnetic field signals, which then forwards its results to another processing unit.

An exemplary method for detecting muscle activity with such a sensor device is shown schematically in the flowchart 3 shown. The magnetic field is detected at one or more locations on a user's body via the one or more magnetometer units in step 310. As a result of the measurement, at least the magnetic field strength on a magnetometer unit is preferably also the orientation of the magnetic field on this magnetometer unit. The signals from individual magnetometer units can be processed separately or in combination. Signals from several sensor devices can optionally be recorded and processed at the same time, such as signals from two gloves for the right and left hand of a user.

The raw signals from the magnetometer units can then be preprocessed in a further step 320 by one or more signal processing units. For example, the raw signals from the magnetometer units can first be filtered via a bandpass so that all noise components with frequencies outside the expected frequencies of muscle activity are eliminated. Additionally or alternatively, by averaging the recorded signals or the filtered signals over a period of time significantly longer than typical hand movements, the bias drift of the sensors can be determined and subtracted from the filtered signal, so that a corrected signal is present.

Various features can then be extracted from the magnetometer signals preprocessed in this way in a step 330, which can later be used for evaluation and interpretation. For example, statistical parameters such as the average or the variance of the gradient can be determined from the time course of the magnetometer signals over defined time windows. Additionally or alternatively, the signals can be transformed into the frequency range so that a frequency signal is available for each defined time window, from which various statistical features can also be derived.

The extracted signal features can then optionally be made available to external processing units in step 340 using defined hardware and/or software interfaces or can be further processed directly.

The signal features thus obtained can then be classified in various ways in step 350. A classification of the activated muscle or the activated muscle group can be carried out directly on the basis of these characteristics. Additionally or alternatively, a specific movement, posture or movement sequence can also be classified directly without evaluating the specific individual muscle signals.

Various methods can be used for classification 350. The classification methods can be used separately or in combination or sequence.

A threshold value method can be used as a simple classification method, which uses, for example, upper and/or lower threshold values for certain signal features. The threshold values can, for example, be specified as a table that includes empirically collected characteristic values.

In addition, various machine learning methods can also be used to classify the captured signals. One possibility is to interpret the signals using a statistical classification method, such as a decision tree or a random forest method.

However, end-to-end methods based on deep neural networks can also be used, which receive the (if necessary pre-processed) signals from the magnetometer units as input values and carry out the classification according to muscle activity, muscle group, posture or movement sequence.

Several target variables can also be taken into account for the classification by specifying them in different tables or as different classification models (i.e., for example different trained neural networks). For example, one classification model can determine the muscle group, while another classifies the movement sequence.

Binary classifications can be used, which only distinguish between two states. For example, a distinction can be made between a voluntary hand or finger movement and a resting position of the hand or finger. In this case, the classification should be able to filter out involuntary movements, subtle tremors and the like and recognize them as a state of rest.

Multiclass classifications can also be used, so that the signal features recorded as input values are sorted into one of a large number of predetermined classes. For example, the distinction between different hand or finger movements can be used as classes here. 4 shows an example of a hand movement in the wrist, whereby the hand is moved either up/out (extension 410 of the wrist) or down/in (flexion 420 of the wrist). A classification can then, for example, include at least the states “hand at rest”, “extension” or “flexion”. In other embodiments, however, a more complex classification can also be specified, which, for example, additionally distinguishes between the speed of a movement sequence (fast and slow flexion) or analyzes the movement sequences of several movements. For example, additional classes can be specified for isolated movements (single flexion of the wrist) and combined movements (first flexion, then extension). For this purpose, the sensors do not necessarily have to be located on the wrist, but rather in the area of the muscles that trigger these movements. It is understood that the movements mentioned here are only to be understood as examples and any muscle activities, movements and gestures can be classified in this way. For example, simply tensing a muscle without any specific movement can trigger a signal and be classified accordingly.

An improved assessment can then be made by optionally combining the results. For example, in the examples mentioned above, the second classification could only be used to recognize a gesture if an arbitrary movement is recognized in the first classification. A multi-stage classification could also be used, which in a first step identifies the moving muscle groups and uses these - possibly with further signal features such as time courses - as input values for a further classification, which then classifies the specific gestures or movements. In other embodiments, however, the entire interpretation can also be carried out by a single classification model with several classes, which only receives the magnetometer signals or the signal features obtained therefrom as input values.

In all classification methods, a confidence level can also be added to each defined target variable value can be calculated that reflects the statistical certainty of the classification result. This value can be used, for example, to define another threshold for motion detection; If the confidence value is below a predetermined limit, it can be specified, for example, that the detected movement is not used for further control, or a user can be asked to repeat a movement again.

To form such classification models, measurements can be carried out in a training phase, which include, for example, predetermined movements or movement sequences, so that labeled training data can be generated based on these movements and the recorded measurement results. Additionally or alternatively, classic methods such as electromyography via surface electrodes or needle electrodes can be used and correlated with the recorded magnetic field signals in order to create labeled training data or target variables for classification models. The results can then be saved in a suitable manner and delivered together with the sensor device and the associated evaluation software.

It is also possible to incorporate further information into the classification process. For example, information such as a user's age, height, or weight may be collected or obtained from other sources and used to improve user-specific results. Personalization can also be done by the user adapting a classification model through a calibration process. For example, the user can be asked to carry out certain movement sequences or gestures as specified, which are then recorded and evaluated. Optionally, all classification models can also be continuously improved, for example by asking a user for feedback on the correct recognition of movements and postures during active use.

The results of the classification of muscle movements obtained in this way can then be further processed in any way.

For example, the results can be made available to other processing units, which can use them in step 360 as control information for a user interface. For example, it can be used to implement gesture-based or motion-based control for a PC, a tablet or a smartphone or any hardware connected to it. This allows, for example, one or more finger and/or hand gestures to be assigned to a specific system behavior (e.g. scrolling up or down on a displayed page, jumping back and forth in a user interface such as a playlist) and the assigned system behavior can be carried out as soon as the respective one gesture is recognized. It is also possible that the same gestures or muscle movements trigger different reactions for different systems, or that one movement controls several different systems at the same time. Alternatively, the movement can also be evaluated by a single central processing unit for control purposes, whereupon this processing unit can in turn control one or more other devices.

In order to enable motion detection, the measurement of the magnetic fields can in particular be repeated continuously or at predetermined intervals so that all relevant movements are recorded. The signal processing and classification steps can be carried out at the same intervals or at larger intervals, for example whenever a predetermined number of measured signals are present in successive time windows. It is also possible for the measurement and/or the associated evaluation of the magnetic field signals to be triggered or stopped by other conditions, for example by corresponding switches or user interfaces or by signals from other sensors such as a proximity sensor.

Such a user interface, which is able to recognize muscle movements, can be used in particular in neurorehabilitation and to control (partial) prostheses.

Of course, it is also possible to use such a motion-controlled user interface to take over or at least support the control of vehicles, aircraft, drones, machines, robots or other moving elements via magnetometers, if the assignment of the magnetic field signals to the respective movement sequences can be ensured precisely enough .

It is understood that the exemplary embodiments described and illustrated only serve to illustrate the proposed principle and can also be designed differently. For example, differently designed holders can be used to attach the sensors to a user's body; The holder can also be integrated into a device used for another purpose (e.g. wristwatch).

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102018203845 A1 [0004, 0031]
• DE 102020207200 A1 [0004, 0031]
• DE 102019212587 A1 [0031]

Claims (15)

Sensor device for detecting muscle activity using magnetic field sensors, comprising at least one optically pumped magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d), which comprises at least one sensor medium (101, 102, 103, 104) with optically excitable defect centers and is designed to transmit a magnetic field signal to a to detect the measuring location (110, 112) by reading out a spin resonance in the sensor medium that is dependent on the magnetic field strength, a holder (260, 262) for attachment to a part of the body of a user, the at least one optically pumped magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d) being arranged in the holder (260, 262). that it can sense a magnetic field induced by electrical activity when the user's muscles move; a processing unit which is set up to extract signal features of the magnetic field signal in time windows of a predetermined length; and a processing unit (136) which is set up to classify the extracted signal features of one or more time windows into two or more classes using at least one classification model and to output a classification result for the magnetic field signal in the one or more time windows. Sensor device after Claim 1 , wherein the at least one magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d) is further configured to detect an orientation of the magnetic field at a measurement location, and wherein the processing unit is further configured to determine the orientation of the Magnetic field must be taken into account for extracting signal features and/or for classifying the signal features. Sensor device after Claim 1 or 2 , wherein the one or more magnetometer units (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d) are fixed at predetermined locations in the holder (260, 262), and wherein the holder comprises at least one of the following: a garment , a band (260) for attaching to a body part, a glove (262). Sensor device according to one of the preceding claims, wherein the at least one magnetometer unit comprises at least two sensor media (101, 102; 103, 104) which are arranged at a predetermined distance (d1, d2) from one another in a gradiometric configuration, the magnetometer unit being set up for this purpose to detect the magnetic field signal (110, 112) at the measuring location by combining the magnetic field signals of the at least two sensor media without background fields. Sensor device according to one of the preceding claims, wherein the magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d) comprises a nitrogen vacancy center magnetometer, the sensor medium comprising a diamond crystal (101, 102, 103, 104 ) or a section of a diamond crystal with nitrogen vacancy centers. Method for detecting muscle activity of a user, comprising: Detecting (310) a time course of a magnetic field signal at at least one measuring location on a part of a user's body by at least one optically pumped magnetometer unit (270a, 270b, 270c, 270d, 272a, 272b, 272c, 272d), which has at least one sensor medium with optically excitable defect centers comprises and is set up to detect at least one magnetic field strength by reading out a spin resonance in the sensor medium that is dependent on the magnetic field strength, extracting (330) signal features of the magnetic field signal in time windows of a predetermined length; Entering (350) the extracted signal features of one or more time windows into at least one classification model, which is designed to classify the signal features into two or more classes, and obtaining a classification result for the magnetic field signal in the one or more time windows. Procedure according to Claim 6 , where the two or more classes are defined by one or more of the following target variables: a muscle group, a muscle movement, a movement sequence, a nerve activity, a resting state, a time course of a muscle movement. Procedure according to Claim 6 or 7 , wherein magnetic field signals are detected at several different measurement locations on parts of a user's body, and wherein the signal characteristics of the several different measurement locations are combined and entered into the classification model. Procedure according to one of the Claims 6 until 8th , wherein the at least one classification model comprises at least one of the following: a statistical classification method, a threshold check, a neural network. Procedure according to one of the Claims 6 until 9 , wherein the method further comprises: transmitting (340) the extracted signal features and/or a classification result via a communication connection. Procedure according to one of the Claims 6 until 10 , wherein the method further comprises: controlling (360) a device based on the classification result obtained. Procedure according to one of the Claims 6 until 11 , wherein the magnetic field signal with a sensor device according to one of Claims 1 until 6 is recorded. Computing unit that is set up to process all process steps of a method according to one of the Claims 7 until 12 to carry out. Computer program that causes a computing unit to carry out all process steps of a process according to one of the Claims 7 until 12 to be carried out when it is executed on the computing unit. Machine-readable storage medium with a computer program stored on it Claim 14 .

Images